dynobase-icon
Dynobase

How to Import Data from S3 to DynamoDB

Rafal Wilinski

Written by Rafal Wilinski

Published: November 20th, 2026

Working with DynamoDB every day? Dynobase is a desktop DynamoDB client for macOS, Windows and Linux for browsing, querying and editing your tables. Try it free for 7 days. Learn more

To import data from S3 to DynamoDB, stage CSV, DynamoDB JSON, or Amazon Ion files in S3, then call ImportTable with the source prefix and the destination key schema. DynamoDB Import from S3 always creates a new table, so an existing table cannot be the target. The import runs asynchronously and consumes no write capacity. AWS Import from S3 guide

Prepare your S3 files

Put the data under a dedicated prefix such as orders/seed/. DynamoDB attempts to read every matching object, so keep manifests and other metadata outside the prefix. All matching files must use the same format and the same compression setting: NONE, GZIP, or ZSTD. The request settings determine how DynamoDB reads the objects, regardless of filename extension. Keep the source objects unchanged until the job finishes. Import validation, requesting an import

CSV

By default, the first row of each file supplies the attribute names and commas separate the fields. For headerless files, pass InputFormatOptions.Csv.HeaderList with one name per column. With that option, DynamoDB reads the first row as data. Set InputFormatOptions.Csv.Delimiter if your files use another supported separator. Wrap fields containing commas in double quotes, and escape a literal double quote by doubling it. AWS CSV format guide, CSV options

CSV key columns take their types from the table or index key definitions. In this example, LineId becomes a Number because it is declared as a numeric sort key. Other columns become Strings, so Quantity is stored as "2" and Gift as "false". Empty fields are omitted. Use a typed format if those values need to be a DynamoDB Number and Boolean. AWS import formats

DynamoDB JSON

Set InputFormat to DYNAMODB_JSON. Write one marshalled Item object per line, with no array around the records or commas between them. Use DynamoDB attribute descriptors such as S, N, and BOOL. The payload of N is a JSON string, even though DynamoDB stores it as a Number. The DynamoDB JSON converter can help inspect a typed item before you wrap it in Item. AWS import formats, AttributeValue reference

Amazon Ion

Set InputFormat to ION. Write one text record per line with the $ion_1_0 version marker followed by an Item struct. Use Ion decimals for DynamoDB Numbers, such as 1. and 2.. Annotate an Ion list with $dynamodb_SS, $dynamodb_NS, or $dynamodb_BS if it represents a DynamoDB set. AWS Ion import mapping, Ion record layout

Size limits and import cost

The default quota is 50,000 S3 objects per job and 50 concurrent imports per account per Region. The combined size of the source objects for in-flight imports is limited to 15 TB in us-east-1, us-west-2, and eu-west-1, and 1 TB in other Regions. These quotas use the S3 object sizes as stored, so compression helps you fit more data. Split large inputs across multiple objects for parallel processing. Each resulting DynamoDB item must fit within the 400 KB item limit, including attribute names and values. Import quotas, import best practices, DynamoDB item limits

DynamoDB bills per GB of uncompressed source data processed, including records it processes but fails to load. The rate in US East (N. Virginia) is $0.15 per GB, so 100 GB of uncompressed input processed there costs $15 for the import itself. Compression affects the source size quota and leaves the import charge per uncompressed GB unchanged. The job consumes no write capacity on the new table. Budget separately for S3 storage and requests and for storing and using the resulting table. DynamoDB pricing, US East price list, AWS import billing details

Define the new table

In TableCreationParameters, set TableName to an unused name and define the partition key (HASH) and optional sort key (RANGE) in KeySchema. Every item needs a value for each base-table key. In AttributeDefinitions, declare the types of the table and index keys: String (S), Number (N), or Binary (B). Ordinary attributes such as Quantity and Gift need no definitions. Table creation parameters, key attribute types

Set each GSI's name, key schema, and projection in GlobalSecondaryIndexes. Check that any supplied index key has the declared type. An item missing a GSI key stays in the base table and is absent from that index. ImportTable supports GSIs at creation and does not support local secondary indexes. Import validation, sparse indexes, AWS import guide

Use BillingMode: PAY_PER_REQUEST for on-demand capacity. With PROVISIONED, set ProvisionedThroughput with read and write capacity units for the table and each GSI. Size capacity for application traffic after the load, since the import itself uses none of it. ImportTable API, AWS import walkthrough

Grant access to S3, DynamoDB, and CloudWatch

Grant the user or role that starts the import the following permissions. Use the source bucket and prefix for S3 access and the destination account and Region for the table and logs. AWS import IAM example

ResourcePermissions
New table and importdynamodb:ImportTable and dynamodb:DescribeImport. Add dynamodb:ListImports if the operator needs to list jobs.
Source bucket and objectss3:ListBucket on the bucket and s3:GetObject on the objects under the import prefix.
CloudWatch Logslogs:DescribeLogGroups, logs:CreateLogGroup, logs:CreateLogStream, logs:DescribeLogStreams, logs:PutLogEvents, and logs:PutRetentionPolicy. Give the operator logs:GetLogEvents to read the error stream.

For a cross-account bucket, allow the importing principal to list the source bucket and read the input objects in the bucket policy, in addition to its IAM permissions. Set S3BucketSource.S3BucketOwner to the source bucket owner's account ID. Omitting it assumes the caller's account. The source and destination may be in different Regions. AWS cross-account import permissions, CloudWatch Logs permissions

For SSE-KMS source objects, grant the importing principal kms:Decrypt on the source key. Cross-account access requires a customer managed key and authorization in both the key policy and the caller's IAM policy. Import does not support SSE-C source objects. If you also specify a customer managed key for the new DynamoDB table, grant the separate DynamoDB encryption permissions on that key, including kms:CreateGrant. S3 SSE-KMS, cross-account KMS access, DynamoDB key permissions, import permissions

Run the import

For any of the examples below, upload the uncompressed DynamoDB JSON sample to orders/seed/part-0001.jsonl in your source bucket. Export SOURCE_BUCKET with the bucket name, TABLE_NAME with an unused table name, and AWS_REGION with the destination Region. Each example creates an on-demand table keyed by OrderId and LineId, with a BySku GSI. If you prefer a form to a command, Dynobase's S3 import workflow starts the same ImportTable job from the desktop app.

AWS CLI v2

The aws dynamodb import-table command returns the import ARN. Save it for the status check.

Python with boto3

Use the low-level import_table client method.

JavaScript SDK v3

Run this as an ES module with @aws-sdk/client-dynamodb installed. ImportTableCommand reference

CloudFormation

Deploy this template with SourceBucket and TableName set. The stack's execution role needs the import permissions. Changing ImportSourceSpecification requires table replacement. For a named table, a replacement requires a new physical table name. ImportSourceSpecification reference, DynamoDB table resource

Monitor the import and diagnose bad rows

Open Imports from S3 in the DynamoDB console or call DescribeImport with the import ARN. Wait for COMPLETED before switching application traffic. Compare ProcessedItemCount, ImportedItemCount, and ErrorCount against what you expected to load.

DynamoDB skips invalid items and continues processing. If an object cannot be parsed, it may skip the remainder of that object too. An item validation error leaves the job in FAILED with FailureCode: ItemValidationError. A failed import can leave a partially loaded table, so check the destination before retrying with corrected input and an unused table name. Duplicate full primary keys overwrite each other in an unspecified order and produce no error, so deduplicate first if the surviving version matters. AWS import validation and troubleshooting

For failures, inspect FailureMessage and CloudWatchLogGroupArn. In the destination Region, the group is /aws-dynamodb/imports and the stream is <import-id>/error. The import ID is the last segment of the import ARN. Item errors can include the source S3 key, item index, and validation message. AWS CloudWatch troubleshooting

Round-trip a full DynamoDB export

Start with a full DynamoDB export to S3 in DynamoDB JSON format. For the new import, set the prefix to your-prefix/AWSDynamoDB/<export-id>/data/, the format to DYNAMODB_JSON, and compression to GZIP. The data files already contain one typed Item per line. Keep the manifest and checksum files out of the import prefix, and define the destination keys and indexes separately. Incremental exports contain change records that can include deletes, so process them as changes to apply. For a spreadsheet copy, see exporting DynamoDB to CSV. AWS export output format, importing an export

Load into an existing DynamoDB table

For an existing table, AWS also offers bulk put items through StartBulkPutItems. It reads DynamoDB JSON or Ion from S3, consumes no table write capacity, and supports per-item conditions. Its puts replace existing items unless a condition prevents the write. Convert CSV to a supported typed format first, or use a loader that handles CSV directly. Bulk put input formats and conditions

Use a custom loader when you need transformations at ingestion, partial updates, or custom handling for rejected records. With BatchWriteItem, send up to 25 puts or deletes per request and retry UnprocessedItems with exponential backoff. See the JavaScript bulk insert example. Batch puts replace entire items and have no per-item conditions. For an in-place merge, use UpdateItem and add a condition when the update depends on the item's current state. For a file-based GUI workflow, see importing CSV into DynamoDB or importing JSON into an existing table. BatchWriteItem API, UpdateItem API

For parallel processing across many S3 objects, a Step Functions Distributed Map can read CSV or JSON Lines, then run child workflows that transform and write records. Set MaxConcurrency to keep the workers within the target table and GSI write limits. Distributed Map, S3 ItemReader

ApproachDestinationLoad chargesChoose it for
ImportTableNew tablePer uncompressed GB processed, no table write capacityCreating a table from a snapshot or staged CSV
StartBulkPutItemsExisting tableBulk put fees, no table write capacityManaged loads of typed data with optional per-item conditions
Custom BatchWriteItem loaderExisting tableNormal DynamoDB write chargesSimple batch puts or deletes with custom parsing
Step Functions Distributed MapExisting tableWorkflow and worker costs plus DynamoDB writesParallel loads with transformations, partial updates, or custom retry handling

DynamoDB pricing, Step Functions pricing

Frequently Asked Questions

Can DynamoDB ImportTable write to an existing table?

No. ImportTable always creates a new table. For a managed load into an existing table, use bulk put items. Use a custom loader for partial updates or transformations during ingestion.

Does DynamoDB infer data types from CSV?

Table and index key columns use the types declared for those keys. Other populated fields are imported as strings. Use DynamoDB JSON or Ion to preserve types such as numbers and booleans in non-key attributes.

Does importing to DynamoDB consume write capacity?

ImportTable uses no write capacity. AWS charges for the uncompressed input processed, including processed records that fail to load.

Can I import a DynamoDB export directly from S3?

Yes. For a full export in DynamoDB JSON format, point the import at the export's data prefix and choose DynamoDB JSON with GZIP compression. Define the destination table's schema in the import request.

What happens when an import finds invalid rows?

DynamoDB skips invalid items and marks the job FAILED for item validation errors. Check the import status and the error stream in CloudWatch Logs. A failed import can leave a partially loaded table.

Dynobase is a Professional GUI Client for DynamoDB

Start your 7-day free trial today

Product Features

Download
/
Changelog
/
Pricing
/
Member Portal
/
Privacy
/
EULA
/
Twitter
© 2026 Dynobase