Authenticating to AWS Redshift using Ephemeral Credentials

Source: Abdul R. Wahab

Background

With any cloud service, it is important to secure your credentials to protect your data/service from unauthorized access.

In this segment, I will demonstrate how to use Python to generate ephemeral credentials for authenticating to Amazon Redshift.

And also explain why ephemeral credentials are safer than static credentials.

What are Ephemeral Credentials? 🤔

Ephemeral credentials are temporary credentials that provide limited access to AWS services.

They can be used to improve security by reducing the risk of long-term credential exposure.

Ephemeral credentials typically have a short lifetime, usually between 15 minutes to 1 hour, and are automatically rotated by the AWS service. This makes it more difficult for attackers to steal and abuse the credentials.

Why are Ephemeral Credentials safer than Static Credentials? 🤔

Static credentials, such as access key IDs and secret access keys are long-term credentials that can be used to access AWS services.

These credentials are typically stored in configuration files or environment variables on your local machine.

The problem with static credentials is that if they are compromised, they can be used to gain access to your AWS resources indefinitely.

Ephemeral credentials, on the other hand, are temporary and have a limited scope of access.

They are typically generated by a trusted third party, such as AWS Security Token Service (STS), and can be used to access specific AWS resources for a short period of time.

This significantly reduces the risk of long-term credential exposure and makes it more difficult for attackers to gain access to your AWS resources.

Generating Ephemeral Credentials

To generate ephemeral credentials for authenticating to Redshift, you can use the AWS Security Token Service (STS) API.

The following codeblock in Python demonstrates how to use the STS API to generate ephemeral credentials for authenticating to Redshift:

import boto3

def generate_redshift_credentials(database_user, cluster_identifier, duration_seconds):
    sts_client = boto3.client('sts')
    redshift_client = boto3.client('redshift')
    
    # Get the cluster endpoint and port
    cluster_data = redshift_client.describe_clusters(ClusterIdentifier=cluster_identifier)['Clusters'][0]
    endpoint = cluster_data['Endpoint']['Address']
    port = cluster_data['Endpoint']['Port']
    
    # Generate temporary credentials for the specified database user
    response = sts_client.get_cluster_credentials(
        DbUser=database_user,
        ClusterIdentifier=cluster_identifier,
        DurationSeconds=duration_seconds
    )
    
    return {
        'host': endpoint,
        'port': port,
        'user': response['DbUser'],
        'password': response['DbPassword'],
        'dbname': response['DbName'],
        'sslmode': 'require',
        'sslrootcert': '/path/to/redshift-ca-cert.pem'
    }

This function takes following three parameters:

• The database user to authenticate with
• The cluster identifier of your Amazon Redshift cluster
• The duration in seconds that the credentials will be valid for

The function then uses the STS API to generate temporary credentials for the specified database user, and returns the credentials in a dictionary format.

Using Ephemeral Credentials to Authenticate to Redshift

Once you have generated the ephemeral credentials, you can use them to authenticate to Redshift.

Here is an example Python codeblock that does this using the psycopg2 library:

import psycopg2

def connect_to_redshift(credentials):
    conn = psycopg2.connect(
        host=credentials['host'],
        port=credentials['port'],
        dbname=credentials['dbname'],
        user=credentials['user'],
        password=credentials['password'],
        sslmode=credentials['sslmode'],
        sslrootcert=credentials['sslrootcert']
    )
    
    cur = conn.cursor()
    cur.execute('SELECT COUNT(*) FROM my_table')
    count = cur.fetchone()[0]
    print(f'Total records in my_table: {count}')
    
    conn.close()

This script uses the psycopg2 library to connect to the Redshift cluster using the ephemeral credentials returned by the generate_redshift_credentials function. It then executes a simple SQL query to count the number of records in the my_table table and prints the result.

Unit Testing the Ephemeral Credentials Function

To ensure that the generate_redshift_credentials function works as expected, we can write a quick unit test using PyTest.

Here's an example unit test:

import pytest
from moto import mock_redshift
import boto3

from redshift_credentials import generate_redshift_credentials

@pytest.fixture
def redshift_cluster(request):
    with mock_redshift():
        redshift = boto3.client('redshift')
        cluster = redshift.create_cluster(
            ClusterIdentifier='test-cluster',
            NodeType='dc2.large',
            MasterUsername='testuser',
            MasterUserPassword='Test1234',
            NumberOfNodes=2,
        )
        yield cluster
        redshift.delete_cluster(ClusterIdentifier='test-cluster', SkipFinalClusterSnapshot=True)

def test_generate_redshift_credentials(redshift_cluster):
    credentials = generate_redshift_credentials('testuser', 'test-cluster', 900)
    assert credentials['user'] == 'testuser'
    assert credentials['host'] == redshift_cluster['Endpoint']['Address']
    assert credentials['port'] == redshift_cluster['Endpoint']['Port']
    assert credentials['sslmode'] == 'require'

This test uses the moto library to mock a Redshift cluster, and then calls the generate_redshift_credentials function to generate ephemeral credentials for the testuser database user.

It then checks that the generated credentials match the expected values, such as the correct user, host, port, and SSL mode.

Thanks for following along! 👏

Ephemeral credentials are far safer than static credentials because they have a limited scope of access and are automatically rotated by the AWS service.

By using the AWS Security Token Service (STS) API, you can generate temporary credentials that provide secure access to your Amazon Redshift cluster.

If you have any questions / inputs, feel free to add a comment below.