Connect to DynamoDB

Connect to a DynamoDB database through Cyral

These instructions assume you have tracked your DynamoDB database in Cyral and associated it with a Cyral sidecar.

Set up your connection details and credentials

To connect to your DynamoDB database through the sidecar, follow these steps:

1. Get the sidecar endpoint address and port number associated with your DynamoDB data repository:

   • As a Cyral admin, open the Cyral control plane UI.
   • Click Sidecars and click the name of your sidecar.
   • In the Data Repositories list, find your DynamoDB repository.
   • Copy its hostname and port number from the Sidecar Endpoint column.

2. Store the endpoint address as the proxy address:

   • In a shell session, set the HTTP_PROXY and HTTPS_PROXY environment variables to the hostname and port of your sidecar. For example, for a sidecar reachable at http://www.mysidecar.example.com, and a DynamoDB repository bound to it at port 453, we might assign:

     export HTTP_PROXY=http://www.mysidecar.example.com:453
     export HTTPS_PROXY=$HTTP_PROXY

   • If your sidecar runs as an AWS EC2 instance, you must also set the NO_PROXY variable to the IP address of the AWS EC2 instance metadata service, 169.254.169.254. This address does not vary.

     export NO_PROXY=169.254.169.254

   Detailed information and settings for configuring the proxy endpoint for different systems is available in the AWS documentation section, Using an HTTP proxy

3. In your AWS credentials file, add a profile containing a valid (aws_access_key_id) and secret key (aws_secret_access_key) associated with the IAM username that you'll use to connect to DynamoDB. This file is usually found at ~/.aws/credentials.

   In the example below, we create an example profile called mysidecar. You can also use an existing profile that has appropriate credentials.

   [profile mysidecar]
   aws_access_key_id=AKIAIOSFODNCYRALEXAMPLE
   aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYRALEXAMPLEKEY

   tip

   By default, Cyral treats the aws_access_key_id as the username for logging and policy enforcement.

4. Add Cyral's certificate authority bundle to the CLI tool. The sidecar intercepts TLS communications between your data users and the DynamoDB instance. To keep the connection secure, the sidecar signs the messages using its own certificate. CLI clients and tools must be able to validate the sidecar's certificate, and this step provides the CA certificate required for that.

   • Download the certificate bundle from the Cyral control plane. The cyral_ca_bundle.pem file is used by the AWS CLI tool to validate the certificate sent by the sidecar. Use the following command to download it, replacing $CYRAL_CONTROL_PLANE_DOMAIN with your control plane domain:

     curl https://$CYRAL_CONTROL_PLANE_DOMAIN/v1/templates/ca_bundle -o cyral_ca_bundle.pem

   • Next, provide the CA bundle to the AWS CLI by setting an environment variable or using a profile configuration. Choose one of these approaches:

     • Set the AWS_CA_BUNDLE environment variable:

       export AWS_CA_BUNDLE=/path/to/cyral_ca_bundle.pem

     • Set it in the profile of your AWS credentials file. Use the profile you created or modified earlier. Add a line setting the ca_bundle parameter:

       [profile mysidecar]
       ca_bundle = /path/to/cyral_ca_bundle.pem

     • Note that if you're using an AWS SDK, e.g. Ruby or Python SDKs, you will be provided with customized configurations for both proxy and certificate bundle inside your client application. See the SDK sections below for some examples.

Connect to DynamoDB

1. Make sure you have set up your connection details and credentials as shown above.

2. Connect to DynamoDB using your preferred client. User authentication relies on the credentials you collected from the Cyral control plane UI. If present, Cyral policies are enforced. All traffic is sent through the Cyral sidecar, which logs all data activity to the log location configured in your Cyral installation.

CLI

Access DynamoDB using the profile you created earlier. Here's an example using the mysidecar profile:

aws dynamodb --region us-east-1 --profile mysidecar list-tables

CLI + Plugins

Throughout this guide, we've used environment variables to configure the proxy settings used by the AWS CLI tool. An alternative approach is to add the proxy settings to the configuration profile instead. To do this, we need to add a third-party plugin, awscli-plugin-s3-proxy, to the AWS CLI tool. The tool is also compatible with S3 and DynamoDB Streams.

For the steps below, we are assuming the profile aws_sidecar_plugin_profile will be used.

Procedure

1. Install the plugin, awscli-plugin-s3-proxy, using python pip:

python -m pip install awscli-plugin-s3-proxy --user

2. If you are using the AWS CLI v2 (aws --version), you will need to define the appropriate location for the plugin:

   a. Get the plugin location: python -m pip show awscli-plugin-s3-proxy

   It is expected to have a Location key in the generated output, for example:

   Linux: Location: /home/<username>/.local/lib/python3.8/site-packages
   Mac: Location: /Users/<username>/.local/lib/python3.8/site-packages
   Windows: Location: c:\Users\<username>\Application Data\python\python38\site-packages

   b. Set the plugin location with the command:

   aws configure set plugins.cli_legacy_plugin_path <LOCATION>

   which for our Mac output above, would be, for example:

   aws configure set plugins.cli_legacy_plugin_path /Users/<username>/.local/lib/python3.8/site-packages

3. Configure using the AWS CLI:

aws configure set plugins.s3-proxy awscli_plugin_s3_proxy

4. Add the sidecar as a proxy for DynamoDB:

aws configure --profile aws_sidecar_plugin_profile set dynamodb.proxy http://<YOUR_SIDECAR_ADDRESS>:<PORT>

replacing <YOUR_SIDECAR_ADDRESS> with the actual address.

5. Add the Cyral CA bundle to the same profile:

aws configure --profile aws_sidecar_plugin_profile set ca_bundle /path/to/cyral_ca_bundle.pem

6. Connect to DynamoDB:

aws dynamodb --region us-east-1 --profile aws_sidecar_plugin_profile list-tables

Ruby SDK

Connect using the AWS SDK for Ruby:

Set http_proxy to the hostname of your sidecar and the port where the DynamoDB repository is bound. This example assumes a sidecar is reachable at example-sidecar-endpoint.com, and a DynamoDB repository bound to it at port 453.

Set ssl_ca_bundle to the path to the Cyral CA bundle.

require 'aws-sdk-dynamodb'

def sidecar_example
    sidecar_certificate_bundle = '/path/to/cyral_ca_bundle.pem'
    sidecar_endpoint = 'example-sidecar-endpoint.com'
    sidecar_port = 453

    region = 'us-east-2'

    # static credentials:
    credentials = Aws::Credentials.new('YOUR_AWS_ACCESS_KEY_ID', 'YOUR_AWS_SECRET_ACCESS_KEY')
    # alternative credential configuration available at:
    # https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html

    dynamodb_client = Aws::DynamoDB::Client.new(
        credentials: credentials,
        region: region,
        http_proxy: sprintf("http://%s:%d", sidecar_endpoint, sidecar_port),
        ssl_ca_bundle: sidecar_certificate_bundle,
    )

    puts 'Running List Tables command through the sidecar'
    result = dynamodb_client.list_tables()
    puts result
end

sidecar_example if $PROGRAM_NAME == __FILE__

Alternatively, the following environment variables are applicable to the Ruby AWS SDK http_proxy, https_proxy, and AWS_CA_BUNDLE. If a shell session is configured with these variables, all data traffic will go through the sidecar, potentially including traffic not related to DynamoDB itself, so the code snipped presented above is preferable in favor of environment variables.

Python SDK

Connect using the AWS SDK For Python

In proxies, specify the hostname of your sidecar and the port where the DynamoDB repository is bound. This example assumes a sidecar is reachable over both http and https at example-sidecar-endpoint.com, and a DynamoDB repository bound to it at port 453.

Set verify to the path to the Cyral CA bundle.

import boto3
from botocore.config import Config

# reference AWS documentation:
# https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html

def sidecar_example():
    sidecar_certificate_bundle = '/path/to/cyral_ca_bundle.pem'
    sidecar_endpoint = 'example-sidecar-endpoint.com'
    sidecar_port = 453 # port selected when binding the dynamodb repo to the sidecar
    region = 'us-east-2'

    dynamodb_client = boto3.client('dynamodb',     
        aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID",
        aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY",
        region_name=region,
        verify=sidecar_certificate_bundle,
        config=Config(
            proxies={
                "http": f"http://{sidecar_endpoint}:{sidecar_port}",
                "https": f"http://{sidecar_endpoint}:{sidecar_port}",
            }),
    )

    print('Running List Tables command through the sidecar')
    result = dynamodb_client.list_tables()
    print(result)

if __name__ == "__main__":
    sidecar_example()

Alternatively, the following environment variables are applicable to the Python AWS SDK HTTP_PROXY, HTTPS_PROXY, and AWS_CA_BUNDLE. If a shell session is configured with these variables, all data traffic will go through the sidecar, potentially including traffic not related to DynamoDB itself, so the code snipped presented above is preferable in favor of environment variables.

Java SDK

Connect using the AWS SDK For NodeJS

In HttpsProxyAgent, specify the hostname of your sidecar, the port where the DynamoDB repository is bound, and the certificate file.

import { DynamoDBClient, ListTablesCommand } from '@aws-sdk/client-dynamodb';
import { NodeHttpHandler } from '@aws-sdk/node-http-handler';
import { HttpsProxyAgent } from 'hpagent';
import fs from 'fs';
import util from 'util';

const sidecar_certificate_bundle = '/path/to/cyral_ca_bundle.pem'
const sidecar_endpoint = 'example-sidecar-endpoint.com'
const sidecar_port = 453
var region = 'us-east-2'

// load the certificate bundle file
const certs = [
    fs.readFileSync(sidecar_certificate_bundle)
];

// create a proxy agent pointing to the sidecar that uses the custom certificate bundle
const agent = new HttpsProxyAgent({
    proxy: util.format('http://%s:%d', sidecar_endpoint, sidecar_port),
    ca: certs
});

(async () => {
    const client = new DynamoDBClient({
        region: region,
        requestHandler: new NodeHttpHandler({
            httpAgent: agent,
            httpsAgent: agent
        }),
    });
    
    const command = new ListTablesCommand({});
    try {
        console.log("Running List Tables command through the sidecar")
        const results = await client.send(command);
        console.log(results.TableNames.join("\n"));
    } catch (err) {
        console.error(err);
    }
})();

A proxy agent is required for a proper configuration and can be installed with the command: npm install hpagent --save. Different SDK versions might require different proxy modules. Please refer to the official AWS documentation at: https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/node-configuring-proxies.html

NodeJS SDK

Connect using the AWS SDK For Java

In ClientConfiguration, specify the hostname of your sidecar, the port where the DynamoDB repository is bound, and the proxy protocol, HTTP. This example assumes a sidecar is reachable at example-sidecar-endpoint.com, and a DynamoDB repository bound to it at port 453.

In the AWS SDK for JAVA, the certificate bundle needs to be directly included in the JVM certificate trust store.

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.Protocol;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.TableCollection;
import com.amazonaws.services.dynamodbv2.model.ListTablesResult;


public class ExampleSidecarConn {

    static String  sidecar_certificate_bundle = "/path/to/cyral_ca_bundle.pem";
    static String  sidecar_endpoint = "example-sidecar-endpoint.com";
    static Integer sidecar_port = 443; // port selected when binding the dynamodb repo to the sidecar
    static String  region = "us-east-2";

    public static void main(String[] args) throws Exception {
        BasicAWSCredentials basic = new BasicAWSCredentials("YOUR_AWS_ACCESS_KEY_ID", "YOUR_AWS_SECRET_ACCESS_KEY");
        AWSCredentialsProvider credentials = new AWSStaticCredentialsProvider(basic);

        // sidecar proxy configuration
        ClientConfiguration config = new ClientConfiguration();
        config.setProxyHost(sidecar_endpoint);
        config.setProxyPort(sidecar_port);
        config.setProxyProtocol(Protocol.HTTP);

        AmazonDynamoDB client = AmazonDynamoDBClientBuilder.
            standard().
            withCredentials(credentials).
            withRegion(region).
            withClientConfiguration(config).build();
        DynamoDB dynamoDB = new DynamoDB(client);


        System.out.println("Running List Tables command through the sidecar");

        TableCollection<ListTablesResult> result = dynamoDB.listTables();
        for (Table a: result) {
            System.out.println(a.getTableName());
        }
    }
}

Golang SDK

Connect using the AWS SDK For Golang

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "net/url"
    "os"

    awshttp "github.com/aws/aws-sdk-go-v2/aws/transport/http"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb"
)

// For more details, please refer to the official AWS documentation:
// https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/custom-http/#configuring-a-proxy

const (
    sidecar_certificate_bundle = "/path/to/cyral_ca_bundle.pem"
    sidecar_endpoint           = "example-sidecar-endpoint.com"
    sidecar_port               = 453 // port selected when binding the dynamodb repo to the sidecar
    region                     = "us-east-2"
)

func assertNil(err error) {
    if err != nil {
        log.Fatal(err)
    }
}

func main() {
    httpClient := awshttp.NewBuildableClient().WithTransportOptions(func(tr *http.Transport) {
        proxyURL, err := url.Parse(fmt.Sprintf("http://%s:%d", sidecar_endpoint, sidecar_port))
        assertNil(err)
        tr.Proxy = http.ProxyURL(proxyURL)
    })

    ctx := context.TODO()
    certReader, err := os.Open(sidecar_certificate_bundle)
    assertNil(err)

    cfg, err := config.LoadDefaultConfig(
        ctx,
        config.WithRegion(region),
        config.WithHTTPClient(httpClient),
        config.WithCustomCABundle(certReader),
    )
    assertNil(err)

    client := dynamodb.NewFromConfig(cfg)
    log.Println("Running List Tables command through the sidecar")
    result, err := client.ListTables(ctx, &dynamodb.ListTablesInput{})
    assertNil(err)

    log.Println(result)
}

Alternatively, the following environment variables are applicable to the Golang AWS SDK HTTP_PROXY, HTTPS_PROXY, and AWS_CA_BUNDLE. If a shell session is configured with these variables, all data traffic will go through the sidecar, potentially including traffic not related to DynamoDB itself, so the code snipped presented above is preferable in favor of environment variables.

Note on other AWS SDKs

AWS offers SDKs for a wide range of programming languages, while here we only show configuration examples for a subset of them. If examples for your programming language are missing from this page, we encourage you to either look for examples in AWS official documentation at https://aws.amazon.com/developer/tools/, or contact Cyral support for help.

Connect to DynamoDB Streams

1. Make sure you have set up your connection details and credentials as shown above.

2. Access DynamoDB Streams using the profile you created earlier. Here's an example using the mysidecar profile:

   > aws dynamodbstreams list-streams --table-name MySampleTable --profile mysidecar

This uses the credentials associated with the profile mysidecar, sending all traffic through the sidecar. Logs are available at the log location configured in your Cyral installation.

Applications leveraging the AWS SDK can also be modified to directly talk to the sidecar by following similar examples as previously presented for DynamoDB.

warning

Cyral policy-based features (for example, access blocking, masking, and alerting) are not available for DynamoDB Streams.