📝 Introduction
💡 Brief overview of what you’re setting up and its purpose
This guide will walk you through setting up Datadog monitoring for AWS Lambda functions deployed as Docker container images. By integrating Datadog, you gain comprehensive observability into your Lambda functions, including traces, metrics, and logs, which is vital for troubleshooting and performance optimization.
👥 Target audience
This guide is intended for developers, DevOps engineers, and system administrators who are deploying AWS Lambda functions using container images and want to leverage Datadog for monitoring.
✅ Prerequisites and requirements
- An AWS account with permissions to create and manage Lambda functions and IAM roles.
- A Datadog account with an API key.
- Basic understanding of Docker and AWS Lambda.
- Python 3.11 Lambda function code.
- Docker installed on your local machine.
📋 Planning and Preparation
🏗️ System architecture or design
The architecture involves deploying a Python-based Lambda function as a container image, with Datadog’s Lambda library and extension integrated directly into the Dockerfile. Datadog will collect and visualize metrics, traces, and logs from the Lambda function.
🔍 Gathering necessary information
- Datadog API key.
- AWS account ID and region.
- IAM role ARN for the Lambda function.
- Lambda function handler name.
🛡️ Backup considerations
Ensure your Lambda function code and Dockerfile are version-controlled. Store your Datadog API key securely using AWS Secrets Manager or other secure methods.
🚀 Step-by-Step Setup
➡️ Step 1: Create or Update Your Dockerfile
🖼️ Detailed instructions and screenshots
Start with the AWS Lambda Python 3.11 base image:
FROM public.ecr.aws/lambda/python:3.11
Install necessary dependencies from requirements.txt:
COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt
Install the Datadog Lambda Extension:
COPY --from=public.ecr.aws/datadog/lambda-extension:latest /opt/. /opt/
Add Datadog environment variables:
ENV DD_SERVICE="your-service-name" ENV DD_ENV="your-environment" ENV DD_VERSION="1.0.0" ENV DD_TRACE_ENABLED="true" ENV DD_LOGS_INJECTION="true" ENV DD_LOG_LEVEL="INFO" ENV DD_API_KEY="YOUR_DATADOG_API_KEY" # Securely inject your API key ENV DD_SITE="datadoghq.com" # Or your Datadog site ENV DD_LAMBDA_HANDLER="lambda_function.lambda_handler" # Original handler
Set the entry point:
CMD ["datadog_lambda.handler.handler"]
Explanation of CMD and DD_LAMBDA_HANDLER:
CMD ["datadog_lambda.handler.handler"]tells Lambda to execute Datadog’s handler function. This function acts as a wrapper, intercepting your Lambda invocation.ENV DD_LAMBDA_HANDLER="lambda_function.lambda_handler"specifies your original Lambda handler function. Datadog’s handler uses this to call your actual code, allowing Datadog to instrument and monitor it.
➡️ Step 2: Create requirements.txt
🖼️ Detailed instructions and screenshots
- Create a
requirements.txtfile in the same directory as yourDockerfile. - Add the following dependencies:
boto3 python-dotenv pyodbc datadog-lambda
Full guide example for Dockerfile config:
# Use AWS Lambda Python 3.11 base image
FROM public.ecr.aws/lambda/python:3.11
# Set working directory
WORKDIR /opt/python/
# Install only required system packages (no MSSQL, no unixODBC)
RUN yum update -y && \
yum install -y \
gcc gcc-c++ make unzip python3-devel wget && \
yum clean all && \
rm -rf /var/cache/yum /var/tmp/* /tmp/*
# Install Python dependencies
RUN pip install --no-cache-dir awslambdaric -t .
# Optional: Copy extracted layer contents if used
COPY layers/layer-content/python /opt/python
# Install Datadog Lambda Extension
COPY --from=public.ecr.aws/datadog/lambda-extension:latest /opt/. /opt/
# Switch to app directory
WORKDIR /var/task
# Install Python requirements
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy Python source code
COPY src/*.py .
COPY .env .
# Set Datadog and Lambda environment variables
ARG DD_API_KEY
ARG ENV_NAME
ENV DD_SERVICE="yourfunctioname-lambda"
ENV DD_ENV=${ENV_NAME}
ENV DD_VERSION="1.0.0"
ENV DD_TRACE_ENABLED="true"
ENV DD_LOGS_INJECTION="true"
ENV DD_LOG_LEVEL="INFO"
ENV DD_API_KEY=${DD_API_KEY}
ENV DD_SITE="datadoghq.com"
ENV DD_LAMBDA_HANDLER="lambda_function.lambda_handler"
# Lambda entrypoint
CMD ["datadog_lambda.handler.handler"]
➡️ Step 3: Build and Push the Docker Image
🖼️ Detailed instructions and screenshots
- Build the Docker image:
- Tag and push the image to Amazon ECR.
docker buildx build --platform linux/amd64 --provenance=false -t your-image-name .
➡️ Step 4: Create or Update the Lambda Function
🖼️ Detailed instructions and screenshots
- In the AWS Lambda console, create or update your function.
- Specify the container image from ECR.
- Ensure the Lambda function’s IAM role has necessary permissions.
🧪 Verification and Testing
✅ How to confirm the setup is successful
- Invoke the Lambda function.
- Check Datadog for metrics, traces, and logs.
🔬 Basic testing procedures
- Create test events and invoke the Lambda function.
- Monitor Datadog dashboards for real-time data.
🔧 Post-Setup Configuration
⚙️ Optional configuration steps
- Configure custom Datadog dashboards and alerts.
- Adjust Datadog environment variables for specific needs.
🔒 Security hardening
- Use AWS Secrets Manager for the Datadog API key.
- Restrict IAM role permissions to the minimum required.
📈 Performance tuning
- Monitor Datadog metrics and traces to identify performance bottlenecks.
- Adjust Lambda function memory and timeout settings as needed.
🎉 Conclusion
🏁 Summary of the setup process
You have successfully set up Datadog monitoring for your containerized AWS Lambda function. This integration provides valuable insights for troubleshooting and performance optimization.
⏭️ Next steps and further resources
- Explore advanced Datadog features for Lambda monitoring.
- Refer to the official Datadog documentation for more details: