AWS CLI

The AWS Command Line Interface is a tool that can be used to interact with Amazon Web Services in a command-line shell, but only the S3 commands are relevant to the USS S3 gateway.

The awscli tool is available for Windows, macOS and Linux. The steps below show how the tool can be installed and used on Rocky Linux. Installation and usage instructions for other operating systems can be found here:

• https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

Install awscli

[user@localhost ~]$ sudo dnf install awscli

Configure awscli

The awscli utility's configuration can be associated with a dedicated profile. The utility must be configured to use path-style bucket addressing, signature version 4, and to not require trailing checksums:  

[user@localhost ~]$ aws configure --profile my-uss-profile
AWS Access Key ID [None]: <username>
AWS Secret Access Key [None]: <password>
Default region name [None]: [Enter]
Default output format [None]: [Enter]
[user@localhost ~]$ aws configure --profile my-uss-profile set s3.addressing_style path
[user@localhost ~]$ aws configure --profile my-uss-profile set s3.signature_version s3v4
[user@localhost ~]$ aws configure --profile my-uss-profile set request_checksum_calculation when_required

More configuration options can be found here: 

• https://docs.aws.amazon.com/cli/latest/topic/s3-config.html

By default, the configuration is maintained within ~/.aws/config and ~/.aws/credentials. Using environment variables may be another option for specifying the configuration settings. When running the utility, it's necessary to specify the configured profile and the endpoint URL for the USS S3 gateway. The credentials, bucket name, and endpoint URL should be provided by the USS admins. Creating an alias including these arguments may be helpful. For example: 

[user@localhost ~]$ alias aws="aws --endpoint-url <https://uss-endpoint-url> --profile my-uss-profile"

Run awscli

Below are a few basic command examples. The --profile and --endpoint-url arguments have been omitted.  

List all buckets:

[user@localhost ~]$ aws s3 ls

List the top-level contents of a bucket (add --recursive to list all contents within the specified bucket and prefix) 

[user@localhost ~]$ aws s3 ls s3://mybucket

Note: The forward slash character is used as the delimiter to define hierarchy (specifying a different arbitrary delimiter is not supported)
Note: The returned list of keys from the USS S3 gateway is not guaranteed to be in sorted order, but the returned order is guaranteed to be consistent 

Copy a local file to a specific bucket and prefix:

[user@localhost ~]$ aws s3 cp /home/user/downloads/myfile.bin s3://mybucket/mydownloads/

Synchronize a bucket prefix to a local directory:

[user@localhost ~]$ aws s3 sync /home/user/downloads s3://mybucket/mydownloads/

Note: The awscli utility assumes keys will be returned in sorted order but this is not guaranteed by the USS S3 gateway. The result is that the aws s3 sync command may re-transfer files to the bucket prefix that already exist.  
Note: The --delete flag should not be used with the aws s3 sync command as it may result in objects being purged from the bucket prefix that exist in the local directory tree

Generate a pre-signed URL that may be shares with others; allowing them to access the object for a limited time without exposing your credentials (the command will return a temporary link that can be used to download the object):

[user@localhost ~]$ aws s3 presign s3://mybucket/mydownloads/myfile.bin --expires-in 3600

More examples can be found here:

• https://docs.aws.amazon.com/cli/latest/userguide/cli-services-s3-commands.html

Access Buckets Anonymously

A bucket can be configured to allow anonymous unauthenticated read-only access (USS administrators will need to enable the permissions). Once the bucket is configured to allow anonymous requests, the awscli utility can be run with the --no-sign-request flag to issue the commands without loading credentials. 

[user@localhost ~]$ aws s3 ls s3://mybucket/publicobjects/ --no-sign-request --endpoint-url <https://uss-endpoint-url>

Note: Anonymous requests will also be evaluated against the underlying file system's permissions. If a file or directory does not grant read access to the 'others' classification or 'Everyone' group, anonymous requests will not be allowed to read the object.