Skip to main content

Multipart Uploads

Multipart upload lets you upload large objects in parts, enabling parallelism, retries of failed parts, and resuming interrupted transfers.

When to use multipart upload​

  • Objects larger than 100 MB (recommended)
  • Unreliable networks where retrying parts is beneficial
  • Parallelizing uploads to increase throughput
Select S3 Endpoint
Choose preferred region.

Using AWS CLI​

Simple approach (aws s3)​

The aws s3 cp command automatically uses multipart for large files:

aws --profile aurora --endpoint-url https:// s3 cp ./bigfile.tar s3://my-bucket/bigfile.tar

Fine-grained control (aws s3api)​

# 1) Create the multipart upload
aws --profile aurora --endpoint-url https:// s3api create-multipart-upload --bucket my-bucket --key bigfile.tar

# Save the returned UploadId as UPLOAD_ID

# 2) Upload parts (example shows 3 parts)
aws --profile aurora --endpoint-url https:// s3api upload-part --bucket my-bucket --key bigfile.tar --part-number 1 --body ./part1.bin --upload-id $UPLOAD_ID

aws --profile aurora --endpoint-url https:// s3api upload-part --bucket my-bucket --key bigfile.tar --part-number 2 --body ./part2.bin --upload-id $UPLOAD_ID

aws --profile aurora --endpoint-url https:// s3api upload-part --bucket my-bucket --key bigfile.tar --part-number 3 --body ./part3.bin --upload-id $UPLOAD_ID

# 3) List uploaded parts (optional)
aws --profile aurora --endpoint-url https:// s3api list-parts --bucket my-bucket --key bigfile.tar --upload-id $UPLOAD_ID

# 4) Complete the upload (provide ETags and part numbers)
aws --profile aurora --endpoint-url https:// s3api complete-multipart-upload --bucket my-bucket --key bigfile.tar --multipart-upload file://parts.json --upload-id $UPLOAD_ID

# Abort a stuck multipart upload
aws --profile aurora --endpoint-url https:// s3api abort-multipart-upload --bucket my-bucket --key bigfile.tar --upload-id $UPLOAD_ID

Best practices​

  • Prefer at least 5–25 MB part sizes; 8–64 MB is a common sweet spot.
  • Upload parts in parallel; 4–16 workers fit many networks.
  • Keep a manifest of part ETags to safely resume and complete.
  • Regularly list and abort stale multipart uploads.
  • Incomplete multipart uploads are automatically deleted after 24 hours.