Skip to content
AWS aws storage 6 min read

Multipart Upload & Transfer Acceleration

Uploading a single large file to Amazon S3 (Simple Storage Service, AWS’s object storage) in one shot is slow and fragile. If your connection drops at 90%, you start over from zero. Multipart upload fixes this by splitting one big object into many small parts that upload in parallel, and Transfer Acceleration fixes slow long-distance uploads by routing your data through the nearest AWS edge location. This page covers both, plus a sneaky cost leak that catches almost everyone: incomplete uploads you keep paying for.

What is multipart upload

Multipart upload breaks a large object into a set of parts (each between 5 MB and 5 GB, except the last part which can be smaller). Each part uploads independently, in parallel, and S3 reassembles them into one object when you finish. If a part fails, you retry only that part, not the whole file.

It works in three stages:

  1. Initiate — you tell S3 you are starting a multipart upload. S3 returns an UploadId that ties all the parts together.
  2. Upload parts — you upload each part with its UploadId and a part number (1 to 10,000). S3 returns an ETag (an entity tag, a checksum-like identifier) for each part.
  3. Complete — you send the list of part numbers and their ETag values. S3 stitches them into the final object.

When to use this (and when not)

Object sizeRecommendationWhy
Under 100 MBSingle PutObjectSimpler, fast enough, no overhead
100 MB – 5 GBMultipart (optional but recommended)Parallelism and retry resilience
Over 5 GBMultipart (required)Single PutObject has a hard 5 GB limit

Use multipart for large files, unreliable networks, or when you want to upload while the file is still being produced (such as streaming a backup). Do not bother for small files — the extra round trips add latency for no benefit.

Good news: the AWS CLI and SDKs do multipart automatically. The CLI’s aws s3 cp switches to multipart once a file crosses the multipart_threshold (8 MB by default). You rarely call the low-level API by hand.

Upload a large file with the CLI

The high-level command handles initiate, parts, and complete for you:

aws s3 cp ./backup-2026.tar.gz s3://my-app-backups/backup-2026.tar.gz

Output:

Completed 2.1 GiB/4.0 GiB (118.3 MiB/s) with 1 file(s) remaining
upload: ./backup-2026.tar.gz to s3://my-app-backups/backup-2026.tar.gz

You can tune the chunk size and parallelism to match your bandwidth:

aws configure set default.s3.multipart_chunksize 64MB
aws configure set default.s3.max_concurrent_requests 20

The low-level API (when you control parts yourself)

# 1. Initiate — capture the UploadId
aws s3api create-multipart-upload \
  --bucket my-app-backups --key backup-2026.tar.gz

# 2. Upload part 1 (repeat for each part, incrementing --part-number)
aws s3api upload-part \
  --bucket my-app-backups --key backup-2026.tar.gz \
  --part-number 1 --upload-id "VXBsb2FkIElEIGZvc..." \
  --body part-001.bin

# 3. Complete the upload
aws s3api complete-multipart-upload \
  --bucket my-app-backups --key backup-2026.tar.gz \
  --upload-id "VXBsb2FkIElEIGZvc..." \
  --multipart-upload file://parts.json

Output:

{
    "Location": "https://my-app-backups.s3.amazonaws.com/backup-2026.tar.gz",
    "Bucket": "my-app-backups",
    "Key": "backup-2026.tar.gz",
    "ETag": "\"9bb58f26192e4ba00f01e2e7b136bbd8-2\""
}

The -2 suffix on the ETag tells you the object was assembled from 2 parts.

The hidden cost: incomplete multipart uploads

Here is the gotcha that wastes real money. When a multipart upload fails or is abandoned (the app crashes, someone hits Ctrl+C, a Lambda times out), the parts already uploaded stay in your bucket. They are invisible in the normal object list, but you are billed for that storage every month, forever, until you remove them.

Find them with:

aws s3api list-multipart-uploads --bucket my-app-backups

Output:

{
    "Uploads": [
        {
            "UploadId": "VXBsb2FkIElEIGZvc...",
            "Key": "backup-2026.tar.gz",
            "Initiated": "2026-01-12T03:14:00.000Z",
            "StorageClass": "STANDARD"
        }
    ]
}

That orphaned upload from January is still costing you. The fix is a lifecycle rule that automatically aborts incomplete multipart uploads after N days.

Add the lifecycle rule (console)

  1. Open the S3 console and click your bucket name.
  2. Go to the Management tab.
  3. Under Lifecycle rules, choose Create lifecycle rule.
  4. Name it abort-incomplete-uploads and choose Apply to all objects in the bucket.
  5. Under Lifecycle rule actions, tick Delete expired delete markers or incomplete multipart uploads.
  6. Check Delete incomplete multipart uploads and set Number of days to 7.
  7. Choose Create rule.

Add the lifecycle rule (CLI)

Create a lifecycle.json:

{
  "Rules": [
    {
      "ID": "abort-incomplete-uploads",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

Apply it:

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-backups \
  --lifecycle-configuration file://lifecycle.json

Or as Terraform:

resource "aws_s3_bucket_lifecycle_configuration" "backups" {
  bucket = "my-app-backups"

  rule {
    id     = "abort-incomplete-uploads"
    status = "Enabled"
    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }
}

Cost tip: This single rule is one of the highest-value lines of config in all of S3. Add it to every bucket that receives uploads. It costs nothing and silently stops a leak that can quietly add gigabytes of phantom storage charges.

Transfer Acceleration

Transfer Acceleration speeds up uploads from clients far away from your bucket’s region. Instead of sending data across the public internet straight to the region, your client uploads to the nearest CloudFront edge location (AWS’s global network of cache servers), which then forwards the data to S3 over Amazon’s fast private backbone.

When to use this (and when not)

Use it when users upload from across the world to a single-region bucket, or when you upload large files over long, congested network paths. Do not use it when clients are in the same region as the bucket (it adds cost for no speed gain) — AWS even offers a Speed Comparison tool so you can check before committing.

Transfer Acceleration costs an extra $0.04–$0.08 per GB transferred (region-dependent) on top of normal data transfer. AWS does not charge the acceleration fee if it did not actually speed up your transfer.

Enable it (console)

  1. Open the bucket in the S3 console.
  2. Go to the Properties tab.
  3. Find Transfer acceleration and choose Edit.
  4. Select Enabled and Save changes.

Enable it (CLI)

aws s3api put-bucket-accelerate-configuration \
  --bucket my-app-backups \
  --accelerate-configuration Status=Enabled

Then upload using the accelerate endpoint:

aws s3 cp ./big-video.mp4 s3://my-app-backups/big-video.mp4 \
  --endpoint-url https://s3-accelerate.amazonaws.com

Bucket names must be DNS-compliant (no dots) to use acceleration.

Best practices

  • Let the AWS CLI and SDKs handle multipart automatically; only drop to the low-level API when you need fine control over individual parts.
  • Tune multipart_chunksize and max_concurrent_requests to match your available bandwidth for the fastest transfers.
  • Add an AbortIncompleteMultipartUpload lifecycle rule (3–7 days) to every upload bucket — this is the single most overlooked cost saver in S3.
  • Periodically audit list-multipart-uploads to catch orphaned parts before the lifecycle rule sweeps them.
  • Enable Transfer Acceleration only after the Speed Comparison tool confirms a real gain, since it adds per-GB cost.
  • Use the larger of 5 MB or total size / 10,000 as your part size so you never exceed the 10,000-part limit on very large objects.
Last updated June 15, 2026
Was this helpful?