Skip to main content

Data Migration Guide

This guide covers moving data from existing S3-compatible systems to Aurora.

Select S3 Endpoint
Choose preferred region.

Strategy overview​

  • Inventory and segment data by bucket/prefix.
  • Decide on transfer tooling (AWS CLI, rclone, SDKs).
  • Use multipart uploads for large files.
  • Validate integrity (size, ETag, checksums, counts).
  • Cutover with delta sync, then switch writers.

Option 1: rclone​

# Initial bulk copy
rclone copy s3:source-bucket aurora:dest-bucket --progress --s3-no-check-bucket --transfers 16 --checkers 32

# Delta sync just before cutover
rclone sync s3:source-bucket aurora:dest-bucket --progress --s3-no-check-bucket --transfers 16 --checkers 32

Integrity validation​

  • Count objects per prefix and compare.
  • Compare total bytes transferred.
  • Spot-check object downloads and verify checksums.
  • Use list-object-versions if migrating versioned buckets.

Permissions and security​

  • Create minimally privileged keys per migration job.
  • Avoid Delete permissions during migration.
  • If using Object Lock, ensure retention allows copying required metadata.