DynamoDB Transactions - The Ultimate Guide (w/ Examples)

DynamoDB supports classical ACID transactions. But let's start with the basics.

What is an ACID Transaction?

Wikipedia says:

A database transaction symbolizes a unit of work performed within a database management system (or similar system) against a database and treated coherently and reliably independent of other transactions. A transaction generally represents any change in a database.

Let me explain a "transaction" to you using more human terms:

Imagine it's Saturday. You're going to the bar with a strict entry policy with two of your friends. As you approach the bar, the bouncer looks at your first friend and lets him in. Then, he looks at you and lets you in. Finally, he looks at your second friend and says: "Sorry, it's not your day", and he is not letting him in. Because you are good friends and you always support each other, in the act of solidarity, you and your friend who managed to get in decide to leave the bar. One for all, all for one, like Three Musketeers. In transaction terms, you decided to "rollback".

Next Saturday, you're going to the same bar, but this time you got lucky, and all three of you managed to get in, and nobody needs to leave. Congrats! In the transactions world, this is a "committed" transaction.

Transactions in the database world have four basic properties:

• Atomicity: Guarantees that a transaction is either fully committed or not at all. Each transaction is a single unit of change to the database and it is either committed or rolled back.
• Consistency: Guarantees that the database is in a consistent state after a transaction is committed.
• Isolation: Guarantees that a transaction is executed in a way that does not affect other transactions.
• Durability: Guarantees that a transaction is committed even if the system fails.

Translating this story to more technical jargon

Transactions are groups of read or write operations performed atomically (multiple statements are treated as one), consistently (state after transaction will be valid), in isolation (ensures concurrent transactions are executed as if sequentially) and durably (once the transaction has been committed, you can be sure that it has been persisted to the table).

Why Transactions are Important

Transactions are super useful when dealing with mission-critical data like currency, item stocks, or, generally speaking - resources. Imagine implementing a banking system, sending money from account A to B. In your first naive implementation, you first subtract money from account A, and then add it to account B. It might sound totally correct.

However, there is a chance that your application will crash between the first and second operation. Or that the second operation was rejected. This will render your application state corrupted, and a portion of the money would be lost in-flight.

Transactions prevent problems like this one from happening. In your application, the operation of both subtracting money from account A, and adding it to account B, would be treated as one indivisible call. There's no possibility of your code crashing between calls, and if the second operation is rejected, the first calculation is rolled back too. It's always all-or-nothing.

Other use cases for transactions include:

• Maintaining uniqueness
• Checking idempotency
• Restricting actions depending on permissions

How to Do Transactions in DynamoDB Using Node.js

Since we already know why it is important, the chances are that you've identified such patterns in your application, and you would like to implement them. Here's how you can implement a simple transaction against DynamoDB using Node.js:

const AWS = require("aws-sdk");
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region:'us-east-1' });

await dynamoDB.transactWrite({
  TransactItems: [
    {
      Put: { // Write an item to the Cart table
        Item: { // Actual Item
          id: '777',
          count: '1',
          description: 'Lorem ipsum...'
        },
        TableName: "Cart",
      },
    },
    {
      Update: { // Decrement count of items available to buy only if the count is greater than zero
        ConditionExpression: "#count > 0",
        ExpressionAttributeNames: { "#count": "count" },
        ExpressionAttributeValues: {
         ":value": 1,
        },
        Key: {
         id: '777-000',
        },
        TableName: "Items",
        UpdateExpression: "SET #count = :count - :value",
        },
      },
    ],
  }).promise();
const AWS = require("aws-sdk");
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region:'us-east-1' });

await dynamoDB.transactWrite({
  TransactItems: [
    {
      Put: { // Write an item to the Cart table
        Item: { // Actual Item
          id: '777',
          count: '1',
          description: 'Lorem ipsum...'
        },
        TableName: "Cart",
      },
    },
    {
      Update: { // Decrement count of items available to buy only if the count is greater than zero
        ConditionExpression: "#count > 0",
        ExpressionAttributeNames: { "#count": "count" },
        ExpressionAttributeValues: {
         ":value": 1,
        },
        Key: {
         id: '777-000',
        },
        TableName: "Items",
        UpdateExpression: "SET #count = :count - :value",
        },
      },
    ],
  }).promise();

Let's Break It Down

To create a transaction in DynamoDB, you can use documentClient.transactWrite. It takes a parameter with the TransactItems property - an array of operations that should be performed.

• Each of these array items must have one of the top-level properties:
  • Put - Inserts an item into the table. Use the same syntax as in put operation.
  • Update - Updates an item from the table. Use the same syntax as in update operation. Inside it, you can write ConditionExpressions to update the item conditionally. If the condition is not met, the whole transaction is rolled back.
  • Delete - Deletes an item from the table. Use the same syntax as in delete operation.
  • ConditionCheck - A condition to an item that is not being modified by the transaction. Like in Update, if this one is not met, the whole transaction is rejected.
• To use modern ES syntax, the whole expression is ended with a .promise() call and awaited on.

This illustrates a write transaction. What if you wanted to read items using a transaction? Well, that's also possible.

Error Handling

Transactions can be cancelled for multiple reasons:

• ConditionalCheckFailed - the ConditionExpression in one of the items wasn't met so the whole transaction is rolled back.
• TransactionConflict - the transaction was rejected because of a conflict with another ongoing transaction.
• ValidationError - the transaction was rejected because of a validation error. Some parameters might be invalid, some updates might update keys beyond allowed limits, item size might exceed 400KB, there might be a mismatch between the operand in the update expression and the type, and so on.
• TransactionInProgressException - the transaction was rejected because another transaction with the same request token is already in progress.

Limits and Caveats

• Transaction operations have the same limitations as they have in a normal form. For example, the size of the item inserted must be smaller than 400KB.
• The total size of all the items used in the transaction must be smaller than 4MB.
• If there's another transaction in progress updating the same item, any other transactions will be rejected.
• TransactItems can only have up to 25 operations. Note that they can target multiple tables.
• Each of the TransactItems operations can use a different table! Yes, you can do multi-table transactions.
• Transactions do not incur any additional costs.

Best Practices for Using DynamoDB Transactions

When using DynamoDB transactions, it's important to follow best practices to ensure optimal performance and reliability. First, always use condition expressions to prevent overwriting existing data unintentionally. Second, minimize the size of the items involved in the transaction to stay within the 4MB limit. Third, avoid long-running transactions as they can lead to increased latency and potential conflicts. Fourth, use exponential backoff for retrying failed transactions to handle transient errors gracefully. Lastly, monitor your transactions using CloudWatch to gain insights into their performance and troubleshoot issues effectively.

Further Reading

• Brilliant article by Alex DeBrie on DynamoDB Transactions
• DynamoDB and Node.js Complete Guide
• Mastering Amazon DynamoDB ACID Transactions