You have an Azure ML real-time inference model deployed to Azure Kubernetes Service.
While running the model, clients sometimes experience a HTTP 503 (Service Unavailable) error.
As a data engineer, you started to investigate the problem and you found that the error occurs when there are spikes in the number of requests.
Which two things can you do to prevent the problem?
Click on the arrows to vote for the correct answer
A. B. C. D. E.Answers: C and D.
Option A is incorrect becausethe utilization level used to trigger creating new replicas is set to 70%, by default, meaning that the “buffer” to handle fluctuations is the remaining 30%
By increasing the limit, this margin narrows, further decreasing the resistance against peak demands.
Option B is incorrect because creating new replicas is quick and responsive, with the time needed to create a new instance being around 1 second.
There are no settings to control the speed of creation.
Option C is CORRECT because setting the minimum number of autoscaling replicas will result in a larger space for handling sudden performance needs.
Option D is CORRECT because the default setting for autoscale target utilization 70%
By decreasing it, the flexibility increases, i.e.
the infrastructure can accommodate higher fluctuations without running out of capacity.
Option E is incorrect because increasing the timeout (which is 1s, by default) might be a solution in the case of timeout (HTTP 504) error.
It won't cure unavailability problems.
Reference:
When clients experience a HTTP 503 (Service Unavailable) error while running an Azure ML real-time inference model deployed to Azure Kubernetes Service, it indicates that the service is overloaded and unable to handle the incoming requests. This error can occur when there are spikes in the number of requests, and the current resources are not sufficient to handle the load.
To prevent this problem, there are a few options:
A. Increase the utilization level at which autoscaling creates new replicas: Autoscaling is a feature in Azure Kubernetes Service that automatically adjusts the number of replicas (instances of a container) based on resource utilization. By increasing the utilization level at which autoscaling creates new replicas, more replicas will be created earlier in response to a spike in the number of requests, which can help to handle the increased load.
B. Set creating autoscale replicas faster: When autoscaling is enabled, the time it takes to create new replicas can affect how quickly the service can respond to increased load. By setting creating autoscale replicas faster, new replicas can be created more quickly, which can help to handle the increased load.
C. Increase the minimum number of autoscaling replicas: By increasing the minimum number of autoscaling replicas, there will be more replicas available to handle the increased load, even before autoscaling kicks in. This can help to prevent the service from becoming overloaded and unable to handle requests.
D. Decrease the utilization level at which autoscaling creates new replicas: Decreasing the utilization level at which autoscaling creates new replicas can cause new replicas to be created earlier in response to a lower level of resource utilization. This can help to ensure that there are enough replicas available to handle the increased load.
E. Increase the service's timeout: When a client sends a request to the service, the service may take some time to process the request and send a response. By increasing the service's timeout, clients will wait longer for a response, which can help to prevent HTTP 503 errors caused by overloaded resources. However, this option may not be ideal if clients require a fast response time.
In summary, to prevent HTTP 503 errors caused by spikes in the number of requests in an Azure ML real-time inference model deployed to Azure Kubernetes Service, options A, B, and C are recommended. These options involve increasing the resources available to handle the increased load, either by creating more replicas earlier or by having more replicas available at all times. Option D is also a possibility, but it may result in more replicas being created than necessary, which can increase costs. Option E may be a temporary solution, but it may not be ideal for clients who require a fast response time.