Deploying a Python Lambda with Requirements using CDK

In order to deploy a Python Lambda function with additional dependencies, we must use something that will install those dependencies. With NodeJS, we can use something like esbuild to do this. In a Typescript CDK, we can use npm or yarn to install this as a development dependency. However, we do not have this option with a Python Lambda. To have this built during deployment, we can use the bundlingOptions interface.

this.queryLambda = new Function(this, "QueryLambda", {
  code: Code.fromAsset(path.join(__dirname, "resources/query_lambda"), {
    bundling: {
      image: Runtime.PYTHON_3_9.bundlingImage,
      command: [
        "bash",
        "-c",
        "pip install -r requirements.txt -t /asset-output && cp -au . /asset-output",
      ],
    },
  }),
  runtime: Runtime.PYTHON_3_9,
  vpc: props.vpc,
  vpcSubnets: { subnetType: SubnetType.PRIVATE_WITH_EGRESS },
  architecture: Architecture.ARM_64,
  handler: "index.handler",
  timeout: Duration.minutes(5),
  role: props.role,
  layers: [props.powerToolsLayer],
  environment: {
    RDS_SECRET_NAME: props.dataBase.secret?.secretName!,
  },
});

In this example from the cdk-private-rds-with-lambda repository, we can see the QueryLambda function is being created using Code from the /resources/query_lambda directory and bundled using a Docker image. This does require Docker desktop to be run locally or from the pipeline that is deploying this function.

When this Lambda function is deployed, pip install will be used with the requirements.txt file in the /query_lambda directory to create the zip file that will be uploaded as the Lambda function.

Note: If you're working with packages that have binary dependencies like psycopg2 for PostgreSQL connections, this basic approach may not work due to platform compatibility issues. For a detailed solution to this specific challenge, see my post on using psycopg2 with AWS Lambda and CDK.

resources/
├── initialize_lambda/
│   ├── index.py
│   └── requirements.txt
└── query_lambda/
    ├── index.py
    └── requirements.txt