Create AWS S3 Bucket using CLI

Last Updated : 27 Nov, 2024

Amazon S3 is a Simple Storage Service in AWS that stores files of different types like Photos, Audio, and videos as objects providing more scalability and security. It allows users to store and retrieve any amount of data at any point in time from anywhere on the web. It facilitates features such as extremely high availability, security, and simple connections to other AWS Services. In this article, we will see how to create s3 buckets using AWS CLI.

Prerequisites:

Set up an Amazon Web Console. Refer to the article - AWS Console
IAM user with appropriate access. Refer to the article - IAM User
Configure a CLI profile using your access key, secret key, default region, and output format. Refer to the article - AWS CLI

Creating S3 Bucket Using CLI

The AWS S3 command is part of the AWS Command Line Interface (CLI) and serves as a tool for interacting with Amazon Simple Storage Service (S3). The AWS S3 make bucket command creates a new bucket in Amazon S3. It's a straightforward way to initiate the creation of a storage container within AWS. mb stands for make bucket The bucket name should be globally unique

aws s3 mb s3://<bucket-name> [--options]

Step 1: We will create a S3 bucket named gfg-example with a default region. Before command execution

aws s3 mb s3://gfg-example

Step 2: The above command will create a s3 bucket with default region

Step 3: Here, we are creating a s3 bucket with specific region using AWS CLI.

The --region flag in the aws s3 mb command allows us to specify the region of the S3 bucket. By using this flag we can ensure that the s3 bucket is belongs to specific AWS region. Here I am creating a bucket in the Mumbai (ap-south-1) region.

aws s3 s3://gfg-example2 --region ap-south-1

S3 bucket with specific region — s3 bucket with specific region

How to List All AWS S3 Buckets?

Use the following command to list all AWS S3 Buckets:

aws s3 ls

Use the following command to list all the files within a bucket:

aws s3 ls s3://my-bucket-name

Uploading Or Downloading Objects From S3 Bucket using AWS CLI Commands

The aws s3 cp command can be used for both uploading files to S3 bucket and downloading files from S3 bucket.

aws s3 cp <source> <target> [options]

source:
- For uploading files to s3 bucket this option set to local path that you want to upload to s3.
- For downloading files from s3 bucket this set to S3 URN.
target:
- For uploading files to s3 bucket this set to URN of S3 bucket.
- For downloading the files from s3 bucket this is set to local path where you want to save the downloaded file.
options: These are optional parameters to modify the original behavior of the command. Here are some commonly used optionsal parameters
- --recursive: If <source> or <target> is a directory then this parameter will recursively upload/downloads the files.
- --acl: This parameter specifies the access control list for object
- --sse: Specifies the server-side encryption method for the uploaded object.

Uploading Objects in S3

Step 1: Here, we are uploading the all the files present in directory to s3 bucket(gfg-example)

aws s3 cp <directory-name> s3://gfg-example --recursive

Downloading Objects From S3

Step 1: In this example we just download the one file from above s3 bucket.

aws s3 cp s3://gfg-example/1.here-there-and-everywhere.pdf C:\Users\ravis\Desktop

Step 2: The above command downloads the 1.here-there-and-everywhere.pdf from gfg-example bucket to local Desktop.

The aws s3 rb command is used to delete s3 bucket and aws s3 rm is used to delete the objects from s3 bucket.

aws s3 rb <S3 URN>

The aws s3 rb deletes the s3 bucket if and only if bucket is empty

aws s3 em <S3 URN>

We can delete all the objects from s3 using --recursive option parameter.

Deleting S3 Bucket Using S3 Command

Step 1: Deleting the one object from s3 bucket(gfg-example)

aws s3 rm s3://gfg-example/1.here-there-and-everywhere.pdf

Step 2: The above command will delete object named with 1.here-there-and-everywhere.pdf

Step 3: First we try to delete the s3 bucket which is empty(gfg-example) then we will empty the bucket and then delete the s3 bucket(gfg-example).

aws s3 rb s3://gfg-example

Step 4: If the bucket is empty then above command directly delete bucket without any error else it will give BucketNotEmptyError.

aws s3 rm s3://gfg-example --recursive  #first delete all the objects present in given bucket
aws s3 rb s3://gfg-example   #delete the bucket

Creating S3 Bucket with Low Level S3 API Command

The s3api is a command that allows users to interact with Amazon Simple Storage Service (S3) through the AWS CLI. With s3api, users can perform advanced operations on S3 buckets and objects such are configure bucket policies, set access control lists (ACLs), manage versioning, configure lifecycle policies, and more.

Creating Bucket

aws s3api create-bucket --bucket <bucket-name> --region <region-name>

create-bucket: This subcommand initiates the process of creating a new S3 bucket.
--bucket: This option is used to specify the name of the bucket you want to create.
--region: This option allows you to specify the AWS region where you want to create the bucket

Step 1: The command will create a s3 bucket with name gfg-eample3 in Europe (Ireland) region

aws s3api create-bucket --bucket gfg-example3 --region eu-west-1

Step 2: Attaching life cycle policy to bucket

aws s3api put-bucket-lifecycle-configuration --bucket <bucket-name> --lifecycle-configuration file://<lifecycle-config-file.json>

put-bucket-lifecycle-configuration: This is the subcommand used to set the lifecycle configuration for the specified bucket.
--bucket: This option specifies the name of the S3 bucket. s3 bucket should be present in s3.
--lifecycle-configuration: This option specifies the lifecycle configuration for the bucket.

First we will create json file which contains the life cycle rules for s3 bucket. Refer the document to know how to write life cycle policy for s3 bucket.
Then we will attach the Life cycle poliy to gfg-example3 bucket

aws s3api put-bucket-lifecycle-configuration --bucket gfg-example3 --lifecycle-configuration file://C:/Users/ravis/Desktop/life-cycle-policy.json

Applying ACL Property of S3

Step 1: We can use the put-bucket-acl of s3api for specifying the ACL property of s3 bucket.

aws s3api put-bucket-acl --bucket <bucket-name> --acl <acl>

put-bucket-acl: This is the command used to set the access control list for the specified bucket.
--bucket: This option specifies the name of the S3 bucket
--acl: This option specifies the desired access control list for the bucket. The <acl> parameter can be set to one of the following values:
- private: Bucket and objects are accessible only to the bucket owner.
- public-read: Bucket is accessible to anyone, but object access is still controlled by individual object ACLs.
- public-read-write: Bucket and objects are accessible to anyone. Use with caution.

aws s3api put-bucket-acl --bucket gfg-example3 --acl private

Step 2: Carefully review the bucket policies and IAM Policies in order to execute the commands properly.

Conclusion

Creating an AWS S3 bucket using the CLI is a straightforward process that involves configuring the AWS CLI with your credentials and executing the aws s3api create-bucket command with the necessary parameters, such as the bucket name and region. This method is efficient for automating tasks and managing resources directly from the command line, especially in large-scale environments or automated workflows. By using the CLI, you gain flexibility and control over your AWS resources while ensuring consistent and repeatable operations across your infrastructure.

How to Create Subfolders in S3 Bucket using AWS CLI

ravi_sadam

Improve

Article Tags :