DynamoDB + CloudFormation - The Ultimate Guide w/ Examples

Cloud services are widely used in modern web applications, and it is common to have multiple cloud services used for a single application. Although these services bring many advantages, handling and provisioning them individually is a challenging task.

As a result, various tools and services were introduced to provision cloud resources easily. As developers, it is essential to understand these services to increase development productivity. So, in this article, I will discuss how to manage AWS DynamoDB using CloudFormation with examples.

What is CloudFormation in AWS?

AWS CloudFormation is a service that allows developers to create, provision, and manage collections of AWS services easily. It can manage the entire infrastructure of your application using a single file and quickly replicate the required resources in any environment with a few clicks.

How CloudFormation Works with DynamoDB

CloudFormation can be used with DynamoDB to automate operations, including table creation and autoscaling. For example, DynamoDB global tables are used to seamlessly replicate data over multiple regions. With CloudFormation, we can enable automated deployments for global tables and replicate the changes over multiple regions in real-time.

You can use a JSON file or a YAML file to define the CloudFormation template. AWS CloudFormation provides a set of properties to create and manage DynamoDB tables and its services.

The below example shows the properties available in a DynamoDB CloudFormation template.

Type: AWS::DynamoDB::Table 
Properties: 
  AttributeDefinitions: 
    - AttributeDefinition
  BillingMode: String
  ContributorInsightsSpecification: ContributorInsightsSpecification
  GlobalSecondaryIndexes: 
    - GlobalSecondaryIndex
  KeySchema: 
    - KeySchema
  KinesisStreamSpecification: KinesisStreamSpecification
  LocalSecondaryIndexes: 
    - LocalSecondaryIndex
  PointInTimeRecoverySpecification: PointInTimeRecoverySpecification
  ProvisionedThroughput: ProvisionedThroughput
  SSESpecification: SSESpecification
  StreamSpecification: StreamSpecification
  TableClass: String
  TableName: String
  Tags: 
    - Tag
  TimeToLiveSpecification: TimeToLiveSpecification
Type: AWS::DynamoDB::Table 
Properties: 
  AttributeDefinitions: 
    - AttributeDefinition
  BillingMode: String
  ContributorInsightsSpecification: ContributorInsightsSpecification
  GlobalSecondaryIndexes: 
    - GlobalSecondaryIndex
  KeySchema: 
    - KeySchema
  KinesisStreamSpecification: KinesisStreamSpecification
  LocalSecondaryIndexes: 
    - LocalSecondaryIndex
  PointInTimeRecoverySpecification: PointInTimeRecoverySpecification
  ProvisionedThroughput: ProvisionedThroughput
  SSESpecification: SSESpecification
  StreamSpecification: StreamSpecification
  TableClass: String
  TableName: String
  Tags: 
    - Tag
  TimeToLiveSpecification: TimeToLiveSpecification

Benefits of Using CloudFormation with DynamoDB

Deployment Speed Is High

We can deploy multiple instances of the same resource within seconds using a CloudFormation template. Thus, this method is much faster than manually setting up the deployments.

Enhance Scalability

CloudFormation templates are much more efficient when scaling up the resources and applications. So even if there was no idea to deploy multiple instances initially, you can use CloudFormation templates to scale the resources later.

Uplift Security

CloudFormation templates help to reduce the human errors that can happen when manually configuring resources. Also, CloudFormation templates contribute to the security of the applications and resources when designed with security in mind.

Consistency Across Environments

Using CloudFormation ensures that your infrastructure is consistent across different environments. This is particularly useful for development, testing, and production environments, where you want to ensure that the infrastructure behaves the same way.

Version Control

CloudFormation templates can be stored in version control systems like Git. This allows you to track changes, roll back to previous versions, and collaborate with other team members more effectively.

Code Examples

DynamoDB Create Table Using CloudFormation

CloudFormation templates can be used to create replicas of the same table across multiple environments within seconds. The below example shows a CloudFormation template for creating a DynamoDB table.

AWSTemplateFormatVersion: 2010-09-09
Description: Creating DynamoDB tables with CloudFormation
Resources:
  myTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Student
      AttributeDefinitions:
        - AttributeName: ID
          AttributeType: S
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: ID
          KeyType: HASH
Outputs:
  Student:
    Description: Student table created with this template
    Value: !Ref myTable
AWSTemplateFormatVersion: 2010-09-09
Description: Creating DynamoDB tables with CloudFormation
Resources:
  myTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Student
      AttributeDefinitions:
        - AttributeName: ID
          AttributeType: S
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: ID
          KeyType: HASH
Outputs:
  Student:
    Description: Student table created with this template
    Value: !Ref myTable

DynamoDB Global Table Using CloudFormation

DynamoDB global tables replicate the same table over multiple regions to ensure uninterrupted and fast accessibility to data. The below example shows how to create DynamoDB global tables replicas in 2 regions (us-west-1 & us-east-1) using a CloudFormation template.

However, it is essential to have at least one table replica in the same region where we deploy the CloudFormation stack.

AWSTemplateFormatVersion: 2010-09-09
Description: Creating global tables with CloudFormation

Parameters: {}
Resources:
  myGlobalTable:
    Type: 'AWS::DynamoDB::GlobalTable'
    Properties:
      TableName: mgtable
      AttributeDefinitions:
        - AttributeName: ID
          AttributeType: S
      KeySchema:
        - AttributeName: ID
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES
      SSESpecification:
        SSEEnabled: true
        SSEType: "KMS"
      Replicas:
        - Region: us-west-1
          PointInTimeRecoverySpecification:
            PointInTimeRecoveryEnabled: true
          SSESpecification:
            KMS
            MasterKeyId: alias/dynamodb-key-west
          Tags:
            - Key: Name
              Value: mgtable
            - Key: Region
              Value: west
        - Region: us-east-1
          PointInTimeRecoverySpecification:
            PointInTimeRecoveryEnabled: true
          SSESpecification:
            KMS
            MasterKeyId: alias/dynamodb-key-east
          Tags:
            - Key: Name
              Value: mgtable
            - Key: Region
              Value: east
AWSTemplateFormatVersion: 2010-09-09
Description: Creating global tables with CloudFormation

Parameters: {}
Resources:
  myGlobalTable:
    Type: 'AWS::DynamoDB::GlobalTable'
    Properties:
      TableName: mgtable
      AttributeDefinitions:
        - AttributeName: ID
          AttributeType: S
      KeySchema:
        - AttributeName: ID
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES
      SSESpecification:
        SSEEnabled: true
        SSEType: "KMS"
      Replicas:
        - Region: us-west-1
          PointInTimeRecoverySpecification:
            PointInTimeRecoveryEnabled: true
          SSESpecification:
            KMS
            MasterKeyId: alias/dynamodb-key-west
          Tags:
            - Key: Name
              Value: mgtable
            - Key: Region
              Value: west
        - Region: us-east-1
          PointInTimeRecoverySpecification:
            PointInTimeRecoveryEnabled: true
          SSESpecification:
            KMS
            MasterKeyId: alias/dynamodb-key-east
          Tags:
            - Key: Name
              Value: mgtable
            - Key: Region
              Value: east

DynamoDB GSI Using CloudFormation

DynamoDB has 2 types of secondary indexes: Global secondary index and Local secondary index. These indexes allow developers to perform queries on non-key attributes.

Resources:
  MyTable:
    Type: AWS::DynamoDB::Table
    Properties:
      GlobalSecondaryIndexes:
        - IndexName: student-index
          KeySchema:
            - AttributeName: studentId
              KeyType: HASH
          Projection:
            ProjectionType: ALL
          ProvisionedThroughput: 
            ReadCapacityUnits: 5
            WriteCapacityUnits: 5
Resources:
  MyTable:
    Type: AWS::DynamoDB::Table
    Properties:
      GlobalSecondaryIndexes:
        - IndexName: student-index
          KeySchema:
            - AttributeName: studentId
              KeyType: HASH
          Projection:
            ProjectionType: ALL
          ProvisionedThroughput: 
            ReadCapacityUnits: 5
            WriteCapacityUnits: 5

The "IndexName" is the name of the global secondary index of the table. It is crucial to make this index name unique from all the table's other indexes. "KeySchema" contains all the attribute names and key types of the complete key schema of the GSI. Key type "HASH" represents the partition key, where "RANGE" indicates the sort key.

DynamoDB Autoscaling Using CloudFormation

DynamoDB auto scaling allows changing the provisioned capacity for the tables and GSIs dynamically. The below example shows how to configure autoscaling using an AWS CloudFormation template.

The first step is to set up the scalable target to set boundaries for writes on the table.

MyTableWriteCapacityScalableTarget: 
  Type: "AWS::ApplicationAutoScaling::ScalableTarget"
  Properties: 
    MaxCapacity: 100
    MinCapacity: 5   
    ResourceId: !Sub table/${myTableName}
    RoleARN: !Sub arn:aws:iam::${AWS::AccountId}:role/aws-service-role/dynamodb.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_DynamoDBTable
    ScalableDimension: "dynamodb:table:WriteCapacityUnits"
    ServiceNamespace: dynamodb
    DependsOn: studentTable
MyTableWriteCapacityScalableTarget: 
  Type: "AWS::ApplicationAutoScaling::ScalableTarget"
  Properties: 
    MaxCapacity: 100
    MinCapacity: 5   
    ResourceId: !Sub table/${myTableName}
    RoleARN: !Sub arn:aws:iam::${AWS::AccountId}:role/aws-service-role/dynamodb.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_DynamoDBTable
    ScalableDimension: "dynamodb:table:WriteCapacityUnits"
    ServiceNamespace: dynamodb
    DependsOn: studentTable

Then we have to create the scaling policy.

MyTableWriteScalingPolicy: 
  Type: "AWS::ApplicationAutoScaling::ScalingPolicy"
  Properties: 
    PolicyName: WriteAutoScalingPolicy
    PolicyType: TargetTrackingScaling
    ScalingTargetId: 
      Ref: MyTableWriteCapacityScalableTarget
    TargetTrackingScalingPolicyConfiguration: 
      TargetValue: 70
      ScaleInCooldown: 60
      ScaleOutCooldown: 60
      PredefinedMetricSpecification: 
        PredefinedMetricType: DynamoDBWriteCapacityUtilization
MyTableWriteScalingPolicy: 
  Type: "AWS::ApplicationAutoScaling::ScalingPolicy"
  Properties: 
    PolicyName: WriteAutoScalingPolicy
    PolicyType: TargetTrackingScaling
    ScalingTargetId: 
      Ref: MyTableWriteCapacityScalableTarget
    TargetTrackingScalingPolicyConfiguration: 
      TargetValue: 70
      ScaleInCooldown: 60
      ScaleOutCooldown: 60
      PredefinedMetricSpecification: 
        PredefinedMetricType: DynamoDBWriteCapacityUtilization

DynamoDB Local Secondary Index Using CloudFormation

There are instances where we need to query data using an alternative sort key instead of the primary key. In such instances, we can create local secondary indexes on a DynamoDB table and execute queries over these indexes.

LocalSecondaryIndexes:
  - IndexName: StudentIndex
    KeySchema:
      - AttributeName: ModuleCode
        KeyType: HASH
      - AttributeName: StudentName
        KeyType: RANGE
    Projection:
      ProjectionType: KEYS_ONLY
LocalSecondaryIndexes:
  - IndexName: StudentIndex
    KeySchema:
      - AttributeName: ModuleCode
        KeyType: HASH
      - AttributeName: StudentName
        KeyType: RANGE
    Projection:
      ProjectionType: KEYS_ONLY

Just like with the GSI, the "IndexName" carries the unique name of the local secondary index with key types, "HASH" and "RANGE." Finally, the "Projection" indicates the attributes copied to the local secondary index from the table. The primary key attributes and index key attributes get projected by default, so this represents any additional attributes to project to the local secondary index.

DynamoDB Stream Using CloudFormation

DynamoDB Stream captures and logs the information on item-level modifications made to a DynamoDB table. The below example of a CloudFormation template shows how to set up DynamoDB Streams by updating an existing table.

SampleDynamoDBTable:
  Type: AWS::DynamoDB::GlobalTable
  DeletionPolicy: Retain
  UpdateReplacePolicy: Retain
  Properties:
    TableName: Studenttable
    BillingMode: PAY_PER_REQUEST
    StreamSpecification:
      StreamViewType: NEW_AND_OLD_IMAGES
    Replicas:
      - Region: us-west-1
    AttributeDefinitions:
      - AttributeName: studentID
        AttributeType: S
    KeySchema:
      - AttributeName: studentID
        KeyType: HASH
SampleDynamoDBTable:
  Type: AWS::DynamoDB::GlobalTable
  DeletionPolicy: Retain
  UpdateReplacePolicy: Retain
  Properties:
    TableName: Studenttable
    BillingMode: PAY_PER_REQUEST
    StreamSpecification:
      StreamViewType: NEW_AND_OLD_IMAGES
    Replicas:
      - Region: us-west-1
    AttributeDefinitions:
      - AttributeName: studentID
        AttributeType: S
    KeySchema:
      - AttributeName: studentID
        KeyType: HASH

DynamoDB Backup Using CloudFormation

DynamoDB PITR (Point in Time Recovery) allows you to restore a DynamoDB table up to any point in the last 35 days. The below code shows how to enable PITR using a CloudFormation template.

studentTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: Student
    PointInTimeRecoverySpecification:
      PointInTimeRecoveryEnabled: true
    ProvisionedThroughput:
      ReadCapacityUnits: 10
      WriteCapacityUnits: 20
    KeySchema:
      - AttributeName: StudentId
        KeyType: HASH
studentTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: Student
    PointInTimeRecoverySpecification:
      PointInTimeRecoveryEnabled: true
    ProvisionedThroughput:
      ReadCapacityUnits: 10
      WriteCapacityUnits: 20
    KeySchema:
      - AttributeName: StudentId
        KeyType: HASH

Refer to this documentation for more details on AWS backup management with CloudFormation.

DynamoDB On-Demand Using CloudFormation

DynamoDB on-demand is one of the two capacity allocations available in DynamoDB. The below example shows how to create a DynamoDB table with on-demand capacity mode using a CloudFormation template.

MyTable:
  Type: AWS::DynamoDB::Table
  Description: Table created with DynamoDB on-demand
  Properties:
    TableName: studentTable
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - AttributeName: studentId
        AttributeType: S
    KeySchema:
      - AttributeName: studentId
        KeyType: HASH
MyTable:
  Type: AWS::DynamoDB::Table
  Description: Table created with DynamoDB on-demand
  Properties:
    TableName: studentTable
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - AttributeName: studentId
        AttributeType: S
    KeySchema:
      - AttributeName: studentId
        KeyType: HASH

The PAY_PER_REQUEST statement defines the on-demand capacity method.

DynamoDB TTL Using CloudFormation

DynamoDB TTL (Time to Live) allows automatically expiring items from a DynamoDB table. You can enable TTL in a DynamoDB table using the below CloudFormation template configuration.

TimeToLiveSpecification:
  AttributeName: TimeToLive
  Enabled: true
TimeToLiveSpecification:
  AttributeName: TimeToLive
  Enabled: true

DynamoDB Alarms Using CloudFormation

We can monitor our DynamoDB databases by using different metrics. These metrics enable the DynamoDB user to understand the workload evolvement with the database. So, we can create CloudWatch alarms to get alerts when the metrics fall out of levels. We can automate this by using CloudFormation.

DynamoDBAccountReadCapAlarm:
  Type: 'AWS::CloudWatch::Alarm'
  Properties:
    AlarmName: 'AccountReadCap'
    AlarmDescription: 'Alarm to detect when maximum read capacity units reached'
    AlarmActions:
      - !Ref DynamoDBMonitoringSNSTopic
    Namespace: 'AWS/DynamoDB'
    MetricName: 'AccountProvisionedReadCapacityUtilization'
    Statistic: 'Maximum'
    Threshold: 80
    ComparisonOperator: 'GreaterThanThreshold'
    Period: 300
    EvaluationPeriods: 1
DynamoDBAccountReadCapAlarm:
  Type: 'AWS::CloudWatch::Alarm'
  Properties:
    AlarmName: 'AccountReadCap'
    AlarmDescription: 'Alarm to detect when maximum read capacity units reached'
    AlarmActions:
      - !Ref DynamoDBMonitoringSNSTopic
    Namespace: 'AWS/DynamoDB'
    MetricName: 'AccountProvisionedReadCapacityUtilization'
    Statistic: 'Maximum'
    Threshold: 80
    ComparisonOperator: 'GreaterThanThreshold'
    Period: 300
    EvaluationPeriods: 1

Alternatives To Using CloudFormation for DynamoDB

1. Terraform
2. Serverless Framework
3. AWS CodeDeploy
4. AWS Config
5. AWS Elastic Beanstalk

Conclusion

In this article, I discussed how to use AWS CloudFormation templates with DynamoDB. The experience of working with CloudFormation will surely help you when dealing with multiple cloud resources. You can easily provision and manage them in multiple environments and drastically reduce the effort spent on infrastructure.

I hope you found this article insightful. Thank you for reading.