Challenges with AWS Security Using VPC

A common scenario in cloud computing is an AWS serverless lambda function reading or writing from a serverless storage, namely S3. If you are using a lambda to connect to call out directly to s3, the packets are actually going out over the open internet, even if the S3 bucket is not publicly accessible. This point is easy to overlook in a serverless architecture. This scenario, in a simple illustration:

More accurate view:

In order to prevent packets from going over the public internet, a VPC, which stands for Virtual Private Cloud, may be used. A VPC is defined as a service that lets you launch AWS resources in a logically isolated virtual network that you define. In using a VPC, there are complexities involved such as security groups, routing tables, and endpoints. Endpoints are used for each AWS service that needs to be accessible within the VPC. On top of this, in some cases, the documentation on how to use these services within the VPC is different from the documentation that is publicly available. such as with the SQS service.

In illustrating a potential pitfall, take an example in which a lambda behind a VPC which writes to S3, but in addition, publishes a message to an SQS Queue. In switching from vanilla lambda to S3 to lambda to S3 via the VPC, once the lambda is attached to the VPC and the SQS service endpoint is attached to the VPC, the SQS implementation needed to change. After a call to AWS customer service, it turns out there is an undocumented parameter that need to be used in the call to the sending a message to the SQS queue when the lambda is behind a VPC. The initialization of the SQS client in Python using the boto3 library went from:

    sqs = boto3.client(‘sqs’)

To:

   sqs = boto3.client(‘sqs’, endpoint_url = sqs_endpoint)

In using a VPC, the endpoint URL needed to be a parameter to the SQS boto3 constructor. This scenario is not mentioned in the publicly available SQS boto3 python documentation and illustrates the need to be extra vigilant when applying best practice security in AWS. If the endpoint URL is omitted when initializing the SQS client for sending an SQS message when behind a VPC, the message does not reach the SQS queue, and there are no error logs to help assist in troubleshooting the issue.