What is the maximum size of the S3 bucket?

246    Asked by DanielCameron in AWS , Asked on Dec 28, 2023

I am an architect and I am designing a data storage solution by using the Amazon S3 for a high-volume media streaming service. How can I plan my bucket configuration considering what is the maximum size of an S3 bucket? What strategies can I employ in handling potential scaling needs if I go beyond this limit? 

Answered by Unnati gautam

In the context of AWS, if you do not know what is the maximum size of a S3 bucket. It is unlimited. Thus, it allows you to store a big amount or volume of data, however, there is a limit set for a single object stored within an S3 bucket which is 5 terabytes.

If you want to manage storage sizes after exceeding this limit, then you can utilize a process which is known as “multiple uploads” in the AWS SDK or AWS Management Console. This process would certainly provide you assistance in uploading large objects by breaking them into smaller components and then uploading them concurrently. Once all parts are created all parts would be combined into a single object.

Here is the example given by using the AWS SDK:-

Import com.amazonaws.AmazonServiceException;
Import com.amazonaws.SdkClientException;
Import com.amazonaws.auth.profile.ProfileCredentialsProvider;
Import com.amazonaws.client.builder.AwsClientBuilder;
Import com.amazonaws.services.s3.AmazonS3;
Import com.amazonaws.services.s3.AmazonS3ClientBuilder;
Import com.amazonaws.services.s3.model.*;
Import java.io.File;
Import java.util.ArrayList;
Import java.util.List;
Public class S3MultipartUpload {
    Public static void main(String[] args) {
        String bucketName = “your-bucket-name”;
        String keyName = “large-object”;
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                .withCredentials(new ProfileCredentialsProvider())
                .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(“s3.amazonaws.com”, “us-west-2”))
                .build();
        // Initialize the InitiateMultipartUploadRequest
        InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(bucketName, keyName);
        // Start multipart upload
        InitiateMultipartUploadResult initResponse = s3Client.initiateMultipartUpload(initRequest);
        Final File file = new File(“path-to-your-large-file”);
        Long contentLength = file.length();
        Long partSize = 5 * 1024 * 1024; // Set part size to 5 MB
        Try {
            // Upload parts
            List partETags = new ArrayList<>();
            Long filePosition = 0;
            For (int I = 1; filePosition < contentLength xss=removed xss=removed xss=removed xss=removed xss=removed>

Your Answer