Large Increase in Traffic to Your Serverless Architecture: Identifying the Root Cause |

Troubleshooting AWS API Gateway, AWS Lambda, and AWS DynamoDB Failures |

Prev Question Next Question

Question

Your serverless architecture using AWS API Gateway, AWS Lambda, and AWS DynamoDB experienced a large increase in traffic to a sustained 3000 requests per second, and the failure rates increased dramatically.

The usage of your DynamoDB table did not exceed 50% of the provisioned throughput, and the primary key is also designed properly.

What is the most likely reason to cause the issue?

Answers

Explanations

Click on the arrows to vote for the correct answer

A. B. C. D.

Answer - C.

Every Lambda function is allocated with a fixed amount of specific resources regardless of the memory allocation.

Each function is allocated with a fixed amount of code storage per function and per account.

By default, AWS Lambda limits the total concurrent executions across all functions within a given region to 1000

For more information on Concurrent executions, please visit the below URL:

http://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html

The most likely reason for the increased failure rates is that the AWS API Gateway is throttling your requests. API Gateway has default throttling limits that apply to all requests made to your APIs. If you exceed these limits, API Gateway will return a 429 Too Many Requests error response code.

The default throttling limits for API Gateway are 10,000 requests per second per account and 5,000 requests per second per API stage. If your traffic exceeds these limits, you may need to request a limit increase from AWS.

Bottlenecking on request (de)serialization may also be a possible cause of increased failure rates. API Gateway serializes the request payload before sending it to your Lambda function, and deserializes the response payload before returning it to the client. If you have large payloads or a large number of requests, this serialization and deserialization process may become a bottleneck.

However, in the given scenario, it is stated that the DynamoDB table usage did not exceed 50% of the provisioned throughput and the primary key is properly designed. Therefore, it is less likely that the issue is caused by the semaphore lock.

Additionally, while it is important to request a limit increase on concurrent Lambda function executions, this is not the most likely reason for the increased failure rates in this scenario.

Therefore, the most likely reason for the increased failure rates in this scenario is that the AWS API Gateway is throttling your requests.