Delete Your AWS Lambda Log Management with Python

As developers, we often face the daunting task of sifting through massive amounts of log data to isolate the information we truly need. AWS Lambda functions add another layer to this challenge by generating a significant volume of logs that are pushed to CloudWatch Log Groups.

Introducing a Python script for better log management! This utility will enable you to delete unnecessary log streams from your specified AWS Lambda CloudWatch Log Group, thereby allowing you to focus on the logs that matter most.

Why Use This Script?

• Targeted Deletion: Remove only the logs you don’t need without manual intervention.
• Clarity in Testing: Easily spot issues during your debugging and testing phases.
• Simplicity: A straightforward approach that integrates seamlessly into your workflow.

How Does It Work?

The following Python script uses boto3, the AWS SDK for Python, to interact with CloudWatch Logs. By automating the retrieval and deletion of log streams, this script makes managing your logs a breeze.

import boto3

# Initialize the boto3 client for CloudWatch Logs
logs_client = boto3.client('logs', region_name='us-east-1')

# Specify the log group ARN (or simply its name) we want to work with
log_group_name = '/aws/lambda/<your-lambda-log-group-name>'

def delete_log_streams(client, log_group_name):
    # Retrieve all log streams from the specified log group
    paginator = client.get_paginator('describe_log_streams')
    for page in paginator.paginate(logGroupName=log_group_name):
        # Extract log stream names from the current page
        log_streams = page['logStreams']
        
        # Iterate and delete each stream
        for log_stream in log_streams:
            try:
                # Retrieve the log stream name
                log_stream_name = log_stream['logStreamName']
                print(f"Deleting log stream: {log_stream_name}")
                
                # Delete the log stream
                client.delete_log_stream(
                    logGroupName=log_group_name,
                    logStreamName=log_stream_name
                )
            except Exception as e:
                print(f"Error deleting log stream: {log_stream_name}. Error: {e}")

if __name__ == '__main__':
    print(f"Deleting all log streams in the log group: {log_group_name}")
    delete_log_streams(logs_client, log_group_name)
    print("Done")

Getting Started

1. Install Boto3: If you haven’t already, install boto3 using pip pip install boto3.
2. AWS Credentials: Make sure your AWS credentials are configured correctly. The script requires access permissions to CloudWatch Logs.
3. Setting the Region: Replace 'us-east-1' with the AWS region you’re working in.
4. Specify Your Log Group: Set log_group_name to the name of the Log Group you’re targeting.
5. Run the Script: Execute the script. It will paginate through all the log streams within the given log group and delete them.
6. Monitoring: After running the script, you can verify the deletions in the AWS Management Console under CloudWatch Logs.

Benefits of Using the Script

• Automation: Automate a task that would otherwise be time-consuming and tedious.
• Cost Savings: Reduce storage costs incurred by excessive log data.
• Focus: Maintain a clean logging environment, focusing on the logs that are important.

Incorporate this script into your AWS environment to keep your Lambda function logs organized and manageable. Say goodbye to clutter and hello to a more efficient way to track down the issues that really matter.

⚠️ Warning: Executing this script will permanently delete all the log streams within the specified log group, and the data will not be recoverable. Use with caution.

Tags: aws, cloudwatch