Cross account Secrets Access in AWS EKS pod

2 min readOct 21, 2020

This covers a use case where you have several applications in an enterprise, and they are deployed in different AWS account based on environment such dev, int, prod. You also have a centralized security account which stores credentials for various dependent services an app needs.

The AWS secrets manager can hosts the secrets. You can create secrets with uri ‘s so as to distinguish between environments and have the application in a given environment only access the secret for that specific environment.

This is the mechanism which would help it achieve :

Secret Hosting Account : grants permission to the Accessing Account

Role : "arn:aws:iam::<Hosting Account>:role/testsecrets-dev"Trust relationship
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "1",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<Accessing Account>:role/testsecrets-dev"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}Policy giving access to "dev" secrets only
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:DescribeKey",
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "arn:aws:secretsmanager:us-east-1:<Hosting Account>:secret:dev/testsecrets/*",
                "arn:aws:kms:us-east-1:<Hosting Account>:key/*"
            ]
        }
    ]
}

Accessing Account : seeks permission

Role : "arn:aws:iam::<Accessing Account>:role/testsecrets-dev"Policy{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Resource": "arn:aws:iam::<Hosting Account>:role/testsecrets-dev",
            "Action": "sts:AssumeRole"
        }
    ]
}

Following IAM roles for service account pattern, ensure to define following Trust relationship.

Trust relationship
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<Accessing Account>:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/<cluster id>"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-1.amazonaws.com/id/<cluster id>:sub": "system:serviceaccount:<eks ns>:<service account>"
        }
      }
    }
  ]
}

Also set the role in the service account and deploy the application.

AWS injects above role in the pod as environment variable AWS_ROLE_ARN. The token file is also available from environment variable AWS_WEB_IDENTITY_TOKEN_FILE. Behind the scenes, EKS is using a Mutating Webhook to mount an OIDC token ( ID token ) signed by the cluster into the pod through a volume projection.

String role_ARN = "arn:aws:iam::<Hosting Account>:role/testsecrets-dev";/*
In this code, AssumeRoleWithWebIdentity takes place behind the scenes by using role and the token from the environment variable. The mounted OIDC jwt token on the pod is exchanged to get AWS temporary credentials by STS Service, those credentials are then used to create instance of AWSSecurityTokenService to prepare for next assume role.
*/AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder.standard()                .withCredentials(WebIdentityTokenCredentialsProvider.create()).withRegion("us-east-1").build();AssumeRoleRequest roleRequest = new AssumeRoleRequest()                    .withRoleArn(role_ARN) 
.withRoleSessionName("iam");  /*
Now use AWSSecurityTokenService to assume role of hosting account and get new set of temporary credentials 
*/AssumeRoleResult roleResponse = stsClient.assumeRole(roleRequest);         Credentials credentials = roleResponse.getCredentials();BasicSessionCredentials awsCredentials = new BasicSessionCredentials(credentials.getAccessKeyId(),                credentials.getSecretAccessKey(), credentials.getSessionToken());  AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()                .withCredentials(new AWSStaticCredentialsProvider(awsCredentials))                .withRegion("us-east-1").build();//now access the secret through client

Note : This was written when AWS documentation did not mention cross account use case. This article uses “Chained AssumeRole” mechanism as mentioned in the docs now.

References

Cross account Secrets Access in AWS EKS pod

Written by Neeraj Agrawal