Retrieve AWS Secrets Manager Keys Across Accounts with a Python Script on EC2

In this guide, we’ll walk you through how to securely retrieve secrets stored in AWS Secrets Manager across different AWS accounts using a Python script running on an EC2 instance.

Scenario Overview:

AWS Account A: Hosts the secret in Secrets Manager.
- Account ID: [AccountA-ID]
- Secret Name: dev/Project/ExampleSecretName
- Secret ARN: arn:aws:secretsmanager:[Region]:[AccountA-ID]:secret:dev/Project/ExampleSecretName-abc123
AWS Account B: Hosts the EC2 instance that will retrieve the secret.
- Account ID: [AccountB-ID]

Step 1: Configure Permissions in AWS Account A

AWS Account A, navigate to the Secrets Manager console and update the resource policy for the secret:

Modify Secrets Manager Resource Permissions

{
  "Version": "2012-10-17", 
  "Statement": [
    {
      "Sid": "CrossAccountReadSecretsManagerKeys",
      "Effect": "Allow", 
      "Principal": {
        "AWS": "*"  // Allows any AWS account to access the secret
      },
      "Action": "secretsmanager:GetSecretValue",  // Grants permission to read the secret's value
      "Resource": "*",  // Applies to all secrets
      "Condition": {
        "StringEquals": {
          "aws:PrincipalOrgID": "o-s3xxxxx"  // Restricts access to a specific AWS Organization ID
        }
      }
    }
  ]
}

Note: Please remove the temporary comments within the JSON content to avoid syntax error.

Create an IAM role named EC2ReadSecretsManagerRole with the following trust policy:

Create an IAM Role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow", 
      "Principal": {
        "AWS": "arn:aws:iam::[AccountB-ID]:role/EC2InstanceAssumeRole"  // Specifies the role in Account B that can assume the role
      },
      "Action": "sts:AssumeRole"  // Grants permission to assume the specified role
    }
  ]
}

Attach the following IAM policy to the role created above:

Attach an IAM Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowAccountBAccess", 
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",  // Grants permission to retrieve the secret's value
        "secretsmanager:DescribeSecret"  // Grants permission to view the secret's details
      ],
      "Resource": "arn:aws:secretsmanager:[Region]:[AccountA-ID]:secret:dev/Project/ExampleSecretName-abc123"  // Specifies the exact secret this policy applies to
    }
  ]
}

Provide your IAM Policy name e.g. EC2ReadSecretsManagerPolicy and save.

Step 2: Configure Permissions in AWS Account B

In AWS Account B, create an IAM role named EC2InstanceAssumeRole with the following trust policy:

Add EC2 in Trust Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow", 
      "Principal": {
        "Service": [
          "ec2.amazonaws.com"  // Allows EC2 instances to assume the role
        ]
      },
      "Action": "sts:AssumeRole"  // Grants permission to assume the role
    }
  ]
}

Attach the following IAM policy to the role:

Attach an IAM Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow", 
      "Action": "sts:AssumeRole",  // Grants permission to assume the specified role
      "Resource": "arn:aws:iam::[AccountA-ID]:role/EC2ReadSecretsManagerRole"  // Specifies the role that can be assumed
    }
  ]
}

Provide your IAM Policy name e.g. EC2AssumeSecretsManagerRole and save.

Attach the Role to the EC2 Instance

Ensure this IAM role (EC2InstanceAssumeRole) is attached to your EC2 instance.

Step 3: Python Script to Retrieve Secrets

Now, create a Python script to retrieve the secrets from AWS Secrets Manager:

import os
import json
import boto3
from dotenv import load_dotenv
from botocore.exceptions import ClientError

# Load environment variables
load_dotenv()

def assume_role(role_arn):
    sts_client = boto3.client('sts')
    try:
        response = sts_client.assume_role(
            RoleArn=role_arn,
            RoleSessionName='EC2InstanceSession'
        )
        return response['Credentials']
    except ClientError as e:
        raise Exception(f"Error assuming role: {e}")

# Function to get secrets for EC2
def get_secrets(key):
    secret_name = os.getenv('SECRET_NAME')
    role_arn = os.getenv('ROLE_ARN')

    if not secret_name or not role_arn:
        raise ValueError("SECRET_NAME and ROLE_ARN must be set in the environment variables.")

    # Assume role to get temporary credentials
    credentials = assume_role(role_arn)

    # Create a Secrets Manager client with the temporary credentials
    client = boto3.client(
        'secretsmanager',
        region_name='us-east-1',
        aws_access_key_id=credentials['AccessKeyId'],
        aws_secret_access_key=credentials['SecretAccessKey'],
        aws_session_token=credentials['SessionToken']
    )

    try:
        response = client.get_secret_value(SecretId=secret_name)
        secret_value = response['SecretString']
        secret_dict = json.loads(secret_value)
        return secret_dict.get(key, None)
    except ClientError as e:
        raise Exception(f"Error retrieving secret: {e}")
    except json.JSONDecodeError:
        raise Exception("Error decoding the secret JSON.")

if __name__ == '__main__':
    # Retrieving the keys on Secrets Manager API keys or database credentials
    try:
        app_api_key = get_secrets('APP_API_KEY')
        database_credentials = get_secrets('DATABASE_CREDENTIALS')

        # Display Sample Output
        print("App API Key:", app_api_key)
        print("Database Credentials:", database_credentials)
    except Exception as e:
        print(f"An error occurred: {e}")

Save the above script on your EC2 instance and install the required libraries like boto3, dotenv, and Python. Then, run python script.py to validate access to Secrets Manager key values.

Step 4: Set Up Environment Variables

Ensure to create a .env file in your project directory before running the python script with the following required variables:

ROLE_ARN=arn:aws:iam::[AccountA-ID]:role/EC2ReadSecretsManagerRole
SECRET_NAME=dev/Project/ExampleSecretName

Conclusion

With this setup, your EC2 instance in Account B can securely retrieve secrets stored in Secrets Manager in Account A. This approach ensures that your sensitive information remains protected while allowing cross-account access where needed.