Amazon S3 authentication methods

P4 Server supports three methods for authenticating to Amazon S3.

• Amazon S3 authentication methods
  • Access key and secret key
  • AWS credentials profile
  • EC2 instance role (recommended for AWS)
  • Required Amazon S3 permissions
  • Supported keys

Access key and secret key

This is the only authenticated method available for non-AWS S3. Use this approach when you need to supply static credentials directly.

Examples

Different storage providers might have different requirements. For the most complete, correct, and current requirements, check with your storage provider.

AWS

Address: s3,region:us-west-2,bucket:my-s3bucket,accessKey:******,secretKey:******

DigitalOcean

Address: s3,url:https://my-s3bucket.sfo3.digitaloceanspaces.com,bucket:my-s3bucket,accessKey:******,secretKey:******

Google Cloud Storage

Address: s3,url:https://storage.googleapis.com/my-s3bucket,bucket:my-s3bucket,accessKey:******,secretKey:******

MinIO

Address: s3,url:https://server:port/my-s3bucket,bucket:my-s3bucket,accessKey:******,secretKey:******

AWS credentials profile

Use this approach to read credentials from the AWS credentials file (~/.aws/credentials) on the server machine.

Address: s3,region:us-west-2,bucket:my-s3bucket,profile:my-profile-name

EC2 instance role (recommended for AWS)

This method automatically retrieves temporary credentials from the IAM role attached to the EC2 instance hosting the P4 Server. No credentials are stored on disk and no keys or files are required.

Address: s3,region:us-west-2,bucket:my-s3bucket,role:my-ec2-role-name

This method follows AWS best practices and is the recommended approach when running P4 Server on Amazon EC2.

IAM role name format

When using the role: parameter, specify only the IAM role name, not the full ARN.

For example, if the role ARN in the AWS console is:

arn:aws:iam::123456789012:role/my-ec2-role-name

Use only the short name:

role:my-ec2-role-name

Using the full ARN will fail, typically with an error similar to:

Failed to retrieve role credentials! AWS rejected command: 404

This occurs because P4 Server queries the EC2 instance metadata service, which recognizes only the role name, not the full ARN.

To verify the role name that the server can see, run the following commands on the EC2 instance hosting the P4 Server:

TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/

The output is the role name exactly as the server expects it.

Required Amazon S3 permissions

The S3 data source requires the following AWS Identity and Access Management (IAM) permissions of the Amazon S3 API Reference:

• s3:GetObject

• s3:PutObject

• s3:DeleteObject

• s3:ListBucket

Example IAM policy

The following minimal policy has been tested and is sufficient for an S3-backed depot:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::my-s3bucket",
                "arn:aws:s3:::my-s3bucket/*"
            ]
        }
    ]
}

Both resources are required as:

• arn:aws:s3:::my-s3bucket

  • This grants access to the bucket itself, which is required for s3:ListBucket.

• arn:aws:s3:::my-s3bucket/*

  • This grants access to objects within the bucket, required for GetObject, PutObject, and DeleteObject.

This distinction is an S3-specific convention. Not including either of these resource entries will result in permission errors that can be difficult to diagnose.

Supported keys

The keys required depend on the method chosen. The supported keys are:

Only one of the credential mechanisms can be specified:

• accessKey/secretKey/token

• profile

• role