Using Vault Agent Caching to authenticate to Amazon RDS

Photo by Mika Baumeister on Unsplash

Problem statement

I was recently dealing with a problem where my API was taking a long time fetching credentials from Vault to connect to Amazon RDS.

Vault is a pretty popular tool for managing secrets such as API keys, passwords, and security tokens. It is commonly used with other AWS services to manage access to resources.

When using Vault to retrieve credentials for AWS, there are several steps involved that can add latency to your application.

Solution

I used the Vault Agent Caching functionality to use leased credentials to authenticate to RDS via my API.

Here is how I implemented this:

Using the Vault Agent

Step 1: Understand how it all works

Using a Vault agent is just one way to speed up the credential retrieval process.

A Vault agent runs in the background and automatically retrieves credentials from Vault and caches them locally. This means that your application can simply read the cached credentials instead of going through the entire Vault authentication process each time.

To use a Vault agent, you need to first install it. Once you have installed the agent, you need to configure it to retrieve the AWS credentials you need.

Step 2: Configure the Vault agent

To configure the Vault agent, you need to create a configuration file that specifies which credentials to retrieve and how often to refresh them.

Here’s an example configuration file:

pid_file = "./vault-agent.pid"
exit_after_auth = true

auto_auth {
  method "aws" {
    mount_path = "auth/aws"
    config = {
      type = "iam"
      role = "my-app-role"
    }
  }
  sink "file" {
    config = {
      path = "/tmp/awscreds"
    }
  }
}
cache {
  use_auto_auth_token = true
}
listener "tcp" {
  address = "127.0.0.1:8200"
  tls_disable = 1
}

This configuration file specifies that the Vault agent should use the AWS authentication method to retrieve credentials for the my-app-rolerole.

It also specifies that the credentials should be cached in a file located at /tmp/awscreds.

Step 3: Connect to RDS

Once you have configured the Vault agent, you can connect to Amazon RDS using the cached credentials.

Example Python script that demonstrates how to do this:

import boto3

# Retrieve the credentials from the Vault agent cache
with open('/tmp/awscreds', 'r') as f:
    creds = f.read()

# Use the credentials to create an RDS client
client = boto3.client('rds', aws_access_key_id=creds['AccessKeyId'], aws_secret_access_key=creds['SecretAccessKey'], aws_session_token=creds['SessionToken'])

# Use the client to perform operations on the RDS instance
response = client.describe_db_instances()

This script retrieves the cached credentials from the file /tmp/awscreds and uses them to create an Amazon RDS client. It then uses the client to perform an operation (in this case, listing the available database instances).

Closing thoughts 👏

By using a Vault agent to cache your AWS credentials, you can significantly reduce the time it takes to authenticate with Vault and retrieve the credentials you need.

This is simply one way to help speed up your application and reduce latency for your users. Meaning, there could be other ways to achieve the same! 🙂

Source: HashiCorp Vault Agent Caching docs