{
    "componentChunkName": "component---src-templates-mdx-content-page-js",
    "path": "/dynamodb-cdc/",
    "result": {"data":{"mdx":{"body":"var _excluded = [\"components\"];\n\nfunction _extends() { _extends = Object.assign || function (target) { for (var i = 1; i < arguments.length; i++) { var source = arguments[i]; for (var key in source) { if (Object.prototype.hasOwnProperty.call(source, key)) { target[key] = source[key]; } } } return target; }; return _extends.apply(this, arguments); }\n\nfunction _objectWithoutProperties(source, excluded) { if (source == null) return {}; var target = _objectWithoutPropertiesLoose(source, excluded); var key, i; if (Object.getOwnPropertySymbols) { var sourceSymbolKeys = Object.getOwnPropertySymbols(source); for (i = 0; i < sourceSymbolKeys.length; i++) { key = sourceSymbolKeys[i]; if (excluded.indexOf(key) >= 0) continue; if (!Object.prototype.propertyIsEnumerable.call(source, key)) continue; target[key] = source[key]; } } return target; }\n\nfunction _objectWithoutPropertiesLoose(source, excluded) { if (source == null) return {}; var target = {}; var sourceKeys = Object.keys(source); var key, i; for (i = 0; i < sourceKeys.length; i++) { key = sourceKeys[i]; if (excluded.indexOf(key) >= 0) continue; target[key] = source[key]; } return target; }\n\n/* @jsxRuntime classic */\n\n/* @jsx mdx */\nvar _frontmatter = {\n  \"path\": \"/dynamodb-cdc/\",\n  \"publishedDate\": \"2026-11-20\",\n  \"updatedDate\": \"2026-11-20\",\n  \"metaTitle\": \"DynamoDB CDC: Streams, Kinesis, Zero-ETL and S3\",\n  \"metaDescription\": \"Choose a DynamoDB change data capture option. Compare Streams, Kinesis, EventBridge Pipes, zero-ETL and S3 exports by replay, latency and cost.\",\n  \"title\": \"DynamoDB Change Data Capture (CDC): Which Option Should You Use?\",\n  \"authorName\": \"Rafal Wilinski\",\n  \"authorLink\": \"https://x.com/rafalwilinski\",\n  \"authorImage\": \"/images/authors/rafal-wilinski.jpg\"\n};\nvar layoutProps = {\n  _frontmatter: _frontmatter\n};\nvar MDXLayout = \"wrapper\";\nreturn function MDXContent(_ref) {\n  var components = _ref.components,\n      props = _objectWithoutProperties(_ref, _excluded);\n\n  return mdx(MDXLayout, _extends({}, layoutProps, props, {\n    components: components,\n    mdxType: \"MDXLayout\"\n  }), mdx(\"p\", null, \"DynamoDB change data capture (CDC) gives downstream systems a way to react to item inserts, updates, and deletes. Start with \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-streams/\"\n  }, \"DynamoDB Streams\"), \" for near-real-time events, add EventBridge Pipes for managed filtering and routing, or choose Kinesis Data Streams for longer replay and independent consumers. Use zero-ETL to keep a supported search index or analytics target in sync, and incremental S3 exports when your pipeline can process batches of changes. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/streamsmain.html\"\n  }, \"AWS streaming CDC comparison\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-dynamodb.html\"\n  }, \"Pipes with DynamoDB\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport.HowItWorks.html\"\n  }, \"AWS export documentation\")), mdx(\"h2\", null, \"Compare DynamoDB CDC options\"), mdx(\"p\", null, \"Start with what the destination needs to recover after an outage. A product index can be rebuilt from current state. A billing event or audit trail may need each individual change and a way to replay it. For export-based options, DynamoDB point-in-time recovery (PITR) determines how far back you can request source data. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/PointInTimeRecovery_Howitworks.html\"\n  }, \"AWS PITR documentation\")), mdx(\"table\", null, mdx(\"thead\", {\n    parentName: \"table\"\n  }, mdx(\"tr\", {\n    parentName: \"thead\"\n  }, mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Option\"), mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Retention or replay\"), mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Ordering and duplicates\"), mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Consumers or target\"), mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Latency\"), mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Pricing model\"), mdx(\"th\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Delete and TTL visibility\"))), mdx(\"tbody\", {\n    parentName: \"table\"\n  }, mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html\"\n  }, \"DynamoDB Streams\")), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"24 hours\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Source records appear once and are ordered per item. Lambda can redeliver.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Lambda, Pipes, Flink, custom readers\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Near real time\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Stream \", mdx(\"inlineCode\", {\n    parentName: \"td\"\n  }, \"GetRecords\"), \" requests where billable, plus consumer costs. Lambda trigger reads are exempt outside Lambda Managed Instances.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"inlineCode\", {\n    parentName: \"td\"\n  }, \"REMOVE\"), \" for deletes. TTL service identity is visible in the Region that performs the deletion.\")), mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/kds.html\"\n  }, \"Kinesis Data Streams for DynamoDB\")), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"24 hours by default, configurable up to 365 days\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"DynamoDB changes can arrive out of order or more than once.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Lambda, Pipes, Amazon Data Firehose, Flink, custom readers\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Near real time\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"DynamoDB CDC units per 1 KB of the larger item image, plus mode-specific Kinesis charges.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Delete records include the old item. TTL service deletes are also captured.\")), mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-dynamodb.html\"\n  }, \"EventBridge Pipes\")), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Inherits the source's retention\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Preserves source ordering when sending records to its target. Design for retries and duplicates.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"One target per pipe, such as SQS or an EventBridge event bus\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Polling and optional batching, plus target time\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Requests after filtering, billed in 64 KB payload chunks, plus applicable source and target costs.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Forwards matching deletes and any TTL metadata supplied by the source.\")), mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/OpenSearchIngestionForDynamoDB.html\"\n  }, \"OpenSearch Service zero-ETL\")), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"The live feed uses the 24-hour DynamoDB stream. Search documents live in the target.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Managed index updates by document ID and version\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"OpenSearch Service domain or OpenSearch Serverless collection\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Within seconds in AWS's launch description\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"OpenSearch Ingestion compute, destination charges, and applicable Streams and snapshot costs\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Source deletes become index deletes, including physical TTL removals.\")), mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/RedshiftforDynamoDB-zero-etl.html\"\n  }, \"Redshift zero-ETL\")), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Managed replication from PITR exports. Query the target tables.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Batched state replication\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Amazon Redshift\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Incremental replication every 15 to 30 minutes\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"DynamoDB PITR and exports, plus Redshift compute and storage\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Replicates deletes. Redshift history mode can retain inactive target versions.\")), mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/amazon-sagemaker-lakehouse-for-DynamoDB.html\"\n  }, \"SageMaker Lakehouse zero-ETL\"), \", including S3 Tables via AWS Glue\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Managed replication from PITR exports. Query the target tables.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Batched state replication\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Lakehouse tables for analytics and ML\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Typically 15 to 30 minutes. AWS Glue refresh interval is configurable.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"DynamoDB PITR and exports, plus target storage and processing charges\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Applies net deletes in the refresh. Export records carry no TTL service marker.\")), mdx(\"tr\", {\n    parentName: \"tbody\"\n  }, mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, mdx(\"a\", {\n    parentName: \"td\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport.HowItWorks.html\"\n  }, \"Incremental export to S3\")), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Request windows within configured PITR history. Keep S3 files for as long as your retention policy allows.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"One compacted result per changed key per window. Make reprocessing safe.\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Glue, Athena, or a custom batch loader\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"Windows of 15 minutes to 24 hours, plus asynchronous export and load time\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"PITR, export volume processed with a 10 MB minimum per incremental export, and S3 charges\"), mdx(\"td\", {\n    parentName: \"tr\",\n    \"align\": null\n  }, \"A net delete contains keys, optionally with an old image. There is no TTL service marker in the export format.\")))), mdx(\"p\", null, \"Pricing and record details are documented separately. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://aws.amazon.com/dynamodb/pricing/\"\n  }, \"DynamoDB pricing\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/streams/latest/dev/kinesis-extended-retention.html\"\n  }, \"Kinesis retention\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://aws.amazon.com/kinesis/data-streams/pricing/\"\n  }, \"Kinesis pricing\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://aws.amazon.com/eventbridge/pricing/\"\n  }, \"EventBridge pricing\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-streams.html\"\n  }, \"DynamoDB Streams TTL records\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://aws.amazon.com/blogs/database/archive-data-from-amazon-dynamodb-to-amazon-s3-using-ttl-and-amazon-kinesis-integration/\"\n  }, \"Kinesis TTL records\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/redshift/latest/mgmt/zero-etl-history-mode.html\"\n  }, \"Redshift history mode\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://aws.amazon.com/glue/pricing/\"\n  }, \"AWS Glue pricing\")), mdx(\"h2\", null, \"Choose a stream or a managed consumer for DynamoDB events\"), mdx(\"p\", null, mdx(\"strong\", {\n    parentName: \"p\"\n  }, \"DynamoDB Streams\"), \" is the simplest starting point for application reactions. Choose \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"KEYS_ONLY\"), \" for key-based invalidation, \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"NEW_IMAGE\"), \" for the resulting item, \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"OLD_IMAGE\"), \" for the previous item, or \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"NEW_AND_OLD_IMAGES\"), \" to compare both. If the event lacks an image you expect, see \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-errors/dynamodb-stream-missing-fields/\"\n  }, \"missing stream fields\"), \". Ordering applies to changes to the same full primary key, so an order item and its separate outbox item can arrive independently. See the \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-streams/\"\n  }, \"DynamoDB Streams setup guide\"), \" and \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-triggers/\"\n  }, \"Lambda triggers\"), \" for setup. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html\"\n  }, \"AWS stream views and ordering\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/transaction-apis.html\"\n  }, \"Transaction propagation\")), mdx(\"p\", null, mdx(\"strong\", {\n    parentName: \"p\"\n  }, \"Kinesis Data Streams for DynamoDB\"), \" is a better fit when consumers need their own read positions and the ability to catch up more than a day later. Set the retention period explicitly and include a per-item version in your data when an older update arriving late could overwrite newer state. DynamoDB sends changes to a Kinesis stream in the same account and Region, and you can enable both native streaming options on the same table. See the \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-stream-vs-kinesis-stream/\"\n  }, \"DynamoDB Streams vs. Kinesis comparison\"), \". \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/kds.html\"\n  }, \"Kinesis destination behavior\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/streamsmain.html\"\n  }, \"AWS streaming CDC comparison\")), mdx(\"p\", null, mdx(\"strong\", {\n    parentName: \"p\"\n  }, \"EventBridge Pipes\"), \" is a consumer of either stream. Use a pipe to filter records, shape or enrich the payload, and forward it to a single target. For example, route outbox records to an event bus when several services need their own rules, or \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-sqs/\"\n  }, \"to an SQS queue\"), \" when one service needs to process them. Use \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"TRIM_HORIZON\"), \" if a new DynamoDB pipe needs all available records. AWS warns that polling can take time to start, so \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"LATEST\"), \" can \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-errors/dynamodb-streams-missing-events/\"\n  }, \"miss events during creation or updates\"), \". \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-dynamodb.html\"\n  }, \"DynamoDB source settings\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-kinesis.html\"\n  }, \"Kinesis source settings\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html\"\n  }, \"Pipes targets\")), mdx(\"h2\", null, \"Which DynamoDB zero-ETL targets are generally available?\"), mdx(\"p\", null, \"As of November 2026, AWS offers these generally available DynamoDB integrations. Check the AWS launch and setup documentation for availability in your target Region.\"), mdx(\"ul\", null, mdx(\"li\", {\n    parentName: \"ul\"\n  }, mdx(\"strong\", {\n    parentName: \"li\"\n  }, \"Amazon OpenSearch Service.\"), \" Use OpenSearch Ingestion for full-text search or filtering on an indexed copy. Enable PITR for the initial S3 snapshot and DynamoDB Streams for ongoing replication. If you only need new changes, the pipeline can start without a snapshot. Budget enough ingestion capacity for the initial load to finish before source changes expire. \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://aws.amazon.com/blogs/aws/amazon-dynamodb-zero-etl-integration-with-amazon-opensearch-service-is-now-generally-available/\"\n  }, \"GA announcement\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-ddb.html\"\n  }, \"Pipeline setup and limitations\")), mdx(\"li\", {\n    parentName: \"ul\"\n  }, mdx(\"strong\", {\n    parentName: \"li\"\n  }, \"Amazon Redshift.\"), \" Use the DynamoDB integration for SQL analytics and joins with warehouse data. It starts with a full export and applies periodic incremental exports, so enable PITR on the source table. Redshift history mode can track versions that make it to the warehouse. Multiple changes compacted within an export window can still collapse into one result. \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://aws.amazon.com/about-aws/whats-new/2024/10/amazon-dynamodb-zero-etl-integration-redshift/\"\n  }, \"GA announcement\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/RedshiftforDynamoDB-zero-etl.html\"\n  }, \"Redshift replication\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport_Requesting.html\"\n  }, \"Incremental export semantics\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://aws.amazon.com/blogs/big-data/get-started-with-amazon-dynamodb-zero-etl-integration-with-amazon-redshift/\"\n  }, \"Redshift cost breakdown\")), mdx(\"li\", {\n    parentName: \"ul\"\n  }, mdx(\"strong\", {\n    parentName: \"li\"\n  }, \"Amazon SageMaker Lakehouse, including Amazon S3 Tables via AWS Glue.\"), \" Use the Lakehouse integration to make DynamoDB data available to analytics and ML tools. Storage options include a general-purpose S3 bucket, S3 Tables for managed Apache Iceberg tables, and Redshift Managed Storage. Check the configured refresh interval against your freshness requirements. \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-zero-etl-integration-with-amazon-sagemaker-lakehouse/\"\n  }, \"Lakehouse launch\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://aws.amazon.com/about-aws/whats-new/2025/07/aws-glue-zero-etl-integrations-amazon-dynamodb-applications-s3-tables/\"\n  }, \"S3 Tables launch\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://docs.aws.amazon.com/glue/latest/dg/zero-etl-target.html\"\n  }, \"Glue target setup\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"li\",\n    \"href\": \"https://docs.aws.amazon.com/glue/latest/dg/zero-etl-configuring-integration.html\"\n  }, \"Glue refresh behavior\"))), mdx(\"h2\", null, \"Use incremental export to S3 for batch synchronization\"), mdx(\"p\", null, \"For an hourly reporting copy, seed the destination with a full export, then request contiguous incremental windows. The start is inclusive and the end is exclusive, so the window after 10:00\\u201311:00 starts at 11:00. Use the completed job's file manifest to select inputs. Upsert rows that have a new image and remove rows that have keys without a new image. Advance the completed window in your loader only after the destination writes succeed, and make reapplying a window safe. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport_Requesting.html\"\n  }, \"Requesting an export\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport.Output.html\"\n  }, \"Export manifests and record shapes\")), mdx(\"p\", null, \"If a product changes price three times in a window, the export keeps the final result for that key. An item inserted and deleted within the same window produces no row. Use export for a replica or batch reconciliation, and capture stream records separately if each intermediate change matters. See the \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/export-dynamodb-data-to-s3/\"\n  }, \"DynamoDB-to-S3 export walkthrough\"), \". \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport.Output.html\"\n  }, \"AWS incremental export examples\")), mdx(\"h2\", null, \"Five CDC patterns with examples\"), mdx(\"h3\", null, \"Search indexing\"), mdx(\"p\", null, \"For a product catalog, use OpenSearch Ingestion and map the full DynamoDB primary key to a stable document ID. A price update replaces the same search document, and deleting the product removes it from search. For a custom search consumer, make document writes conditional on the source version and keep version information for deletions so a late update cannot recreate a removed product. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-ddb.html\"\n  }, \"AWS OpenSearch document actions and versioning\")), mdx(\"h3\", null, \"Transactional outbox and event publishing\"), mdx(\"p\", null, \"When an order is paid, use \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"TransactWriteItems\"), \" to update the order and create an immutable outbox item carrying \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"OrderPaid\"), \", a stable event ID, and the order ID. Have a pipe match the outbox item type in \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"dynamodb.NewImage\"), \" and forward it to an event bus, or have Lambda publish it if the envelope needs custom logic. Consumers deduplicate on the outbox event ID. Keep outbox items long enough for a separate republisher to recover missed events by querying them. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/transactional-outbox.html\"\n  }, \"AWS transactional outbox pattern\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-filtering.html\"\n  }, \"Pipes filter syntax\")), mdx(\"h3\", null, \"Audit history\"), mdx(\"p\", null, \"To retain every account status change, \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-faq/how-to-track-changes-to-items-in-dynamodb/\"\n  }, \"capture both item images\"), \" and write the stream events to S3 with the item key, operation, and event ID. Add actor and reason fields to an explicit audit event when the business needs that context. For a TTL-only archive, filter \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"REMOVE\"), \" records where \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"userIdentity.type\"), \" is \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"Service\"), \" and \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"userIdentity.principalId\"), \" is \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"dynamodb.amazonaws.com\"), \". DynamoDB Streams adds that identity in the Region that performed the deletion. A deleted item's fields require an old image. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.html\"\n  }, \"AWS audit trail example\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-streams.html\"\n  }, \"TTL archive filtering\")), mdx(\"h3\", null, \"Cache invalidation\"), mdx(\"p\", null, \"For a cache keyed by product ID, have Lambda evict \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"product:123\"), \" on an item change. The next cache miss reloads from DynamoDB. Repeating an eviction is safe, but a read that started before the change can refill the cache after the eviction. Use a short cache TTL to bound stale entries and use version-aware fills or direct table reads where the application requires stricter freshness. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/prescriptive-guidance/latest/dynamodb-elasticache-integration/introduction.html\"\n  }, \"AWS DynamoDB and ElastiCache patterns\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html\"\n  }, \"DynamoDB read consistency\")), mdx(\"h3\", null, \"Cross-Region replication\"), mdx(\"p\", null, \"Use \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-global-tables/\"\n  }, \"DynamoDB global tables\"), \" when applications in two Regions need replicas of the same operational table. For a different data model, such as a customer summary table in another Region, use a CDC consumer to write a one-way projection with idempotent upserts and delete handling. If consuming streams from several global table replicas, account for replicated writes and use a shared application event ID for side effects. MREC replicas can coalesce changes, so one replica's stream may differ from another's. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/V2globaltables_HowItWorks.html\"\n  }, \"AWS global tables behavior\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-global-table-design.prescriptive-guidance.checklist-and-faq.html\"\n  }, \"Replica stream records\")), mdx(\"h2\", null, \"Production pitfalls\"), mdx(\"h3\", null, \"Make writes and side effects idempotent\"), mdx(\"p\", null, \"Lambda stream delivery is at least once, even though a record appears once in DynamoDB Streams. For a native stream retry, use its \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"eventID\"), \" as an identity. For events that can be republished, use the application event ID that travels with the payload. Where possible, commit the deduplication entry and the state change together. For an external side effect such as a payment request, pass the same idempotency key on every retry. A deduplication marker written before an unprotected side effect can cause a retry to skip work that never finished. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html\"\n  }, \"Lambda delivery semantics\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_streams_Record.html\"\n  }, \"Stream record fields\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/transactional-outbox.html\"\n  }, \"AWS outbox guidance\")), mdx(\"h3\", null, \"Isolate poison records and retain failed payloads\"), mdx(\"p\", null, \"A permanently invalid record can hold up an affected shard while the consumer retries. For \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"/dynamodb-errors/aws-lambda-dynamodb-stream-error/\"\n  }, \"Lambda stream processing errors\"), \", enable \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"BisectBatchOnFunctionError\"), \" to split batches that throw, and use \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"ReportBatchItemFailures\"), \" to return the sequence number of failed records. Lambda uses the lowest reported failure as the retry checkpoint. Set retry and record-age bounds and select an on-failure destination. Choose S3 if you need the full failed invocation payload for replay after stream expiry. With Pipes, configure source retries, \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"AUTOMATIC_BISECT\"), \" where appropriate, and a dead-letter queue with a maximum record age below source retention. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/lambda/latest/dg/services-ddb-batchfailurereporting.html\"\n  }, \"Lambda partial batch responses\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/lambda/latest/dg/services-dynamodb-errors.html\"\n  }, \"Lambda failure destinations\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-error-troubleshooting.html\"\n  }, \"Pipes retry handling\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/pipes-reference/API_UpdatePipeSourceDynamoDBStreamParameters.html\"\n  }, \"Pipes batch bisection\")), mdx(\"h3\", null, \"Monitor iterator age and plan for the 24-hour gap\"), mdx(\"p\", null, \"Alarm on Lambda's \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"IteratorAge\"), \" and track errors and throttling alongside it. Use maximum age to surface a slow batch that an average could hide. For Kinesis, monitor \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"GetRecords.IteratorAgeMilliseconds\"), \" for polling consumers or \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"SubscribeToShardEvent.MillisBehindLatest\"), \" for enhanced fan-out. For Pipes, watch \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"ExecutionFailed\"), \" and measure the age of the source record at the destination. Choose a lag threshold that leaves enough time to fix the issue and drain the backlog. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/lambda/latest/dg/monitoring-metrics-types.html\"\n  }, \"Lambda metrics\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/streams/latest/dev/monitoring-with-cloudwatch.html\"\n  }, \"Kinesis metrics\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-monitoring.html\"\n  }, \"Pipes metrics\")), mdx(\"p\", null, \"If a caught-up consumer stops for 26 hours while writes continue, changes from the first two hours are already outside the DynamoDB Streams replay window. \", mdx(\"inlineCode\", {\n    parentName: \"p\"\n  }, \"TRIM_HORIZON\"), \" starts at the oldest record still available. If the target needs current state, reconcile with a snapshot or use contiguous incremental exports when PITR covers the gap. If the system must recover every event, arrange longer Kinesis retention, a durable event archive, or a recoverable outbox before the failure. For OpenSearch, a pipeline that has fallen past retention may need a fresh index build. \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html\"\n  }, \"Streams retention\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/PointInTimeRecovery_Howitworks.html\"\n  }, \"PITR recovery window\"), \" \\xB7 \", mdx(\"a\", {\n    parentName: \"p\",\n    \"href\": \"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/OpenSearchIngestionForDynamoDB.html\"\n  }, \"OpenSearch recovery\")), mdx(FAQ, {\n    questions: [{\n      question: \"What DynamoDB events does CDC capture?\",\n      answer: \"DynamoDB Streams and Kinesis capture item inserts, updates, and deletes, including removals performed by TTL. With native DynamoDB Streams, the selected stream view controls whether the event includes item images or keys only.\"\n    }, {\n      question: \"What is the difference between DynamoDB Streams and EventBridge Pipes?\",\n      answer: \"DynamoDB Streams captures item changes. EventBridge Pipes consumes those records and can filter, transform, or enrich them before sending them to a target. A pipe inherits its source's replay window.\"\n    }, {\n      question: \"Can you replay DynamoDB changes after 24 hours?\",\n      answer: \"Native stream records expire after 24 hours. For longer event replay, configure Kinesis retention or archive changes before they expire. PITR exports can recover state changes within the backup window, with intermediate changes compacted.\"\n    }, {\n      question: \"Which DynamoDB zero-ETL targets are generally available?\",\n      answer: \"Amazon OpenSearch Service, Amazon Redshift, and Amazon SageMaker Lakehouse are available, including managed ingestion into Amazon S3 Tables via AWS Glue in supported Regions.\"\n    }, {\n      question: \"Does an incremental export include deleted items?\",\n      answer: \"A net delete appears with its keys and, if requested, its old image. If the item was inserted and deleted within the same export window, it produces no row.\"\n    }],\n    mdxType: \"FAQ\"\n  }));\n}\n;\nMDXContent.isMDXComponent = true;","frontmatter":{"publishedDate":"2026-11-20","updatedDate":"2026-11-20","path":"/dynamodb-cdc/","metaTitle":"DynamoDB CDC: Streams, Kinesis, Zero-ETL and S3","metaDescription":"Choose a DynamoDB change data capture option. Compare Streams, Kinesis, EventBridge Pipes, zero-ETL and S3 exports by replay, latency and cost.","title":"DynamoDB Change Data Capture (CDC): Which Option Should You Use?","authorName":"Rafal Wilinski","authorLink":"https://x.com/rafalwilinski","authorImage":"/images/authors/rafal-wilinski.jpg","image":null}}},"pageContext":{}},
    "staticQueryHashes": ["3649515864"]}