DynamoDB Optimistic vs Pessimistic Locking [The Ultimate Guide]

Introduction to DynamoDB Locking

If you want to store application data in AWS, the go-to service is DynamoDB. It is easy to set up and auto-scales to serve thousands of requests as required. However, one common challenge that comes along the way is concurrency handling.

Without concurrency handling, it is likely to experience data loss as the changes from one user may override the other or affect the final calculations. Therefore, it is crucial to handle cases like this. Luckily, DynamoDB provides both Optimistic and Pessimistic Locking to solve this problem.

Optimistic Locking

What is Optimistic Locking?

Optimistic Locking is a locking strategy offered by DynamoDB to detect whether a record is modified prior to updating it. A typical example would be when saving a record you have queried into the frontend, where no one else has modified it before you save your changes.

It ensures that the data item present at the client side is the same in the DynamoDB table. It prevents your changes from being accidentally overridden by others and vice versa.

How does Optimistic Locking work?

With Optimistic Locking, there is always one attribute in your DynamoDB table that will act as a "version number." It can be a nano-id, an integer, or a timestamp. The version number associated with the record must also be sent when clients request data.

Whenever the client modifies the data item, the version number present on the client side must be the same as the item's version number present in the table item. If it is the same, it means that no other user has changed the record, allowing the write to go through. However, if the version numbers are different, it's likely that another user has already updated the record, causing DynamoDB to reject your write by throwing the exception - ConditionalCheckFailedException. You can retrieve the item again (with newly updated data) and retry your update when this happens.

How to use Optimistic Locking?

To use Optimistic Locking in DynamoDB, you need to implement Conditional Write in your code. Conditional Write is an expression used for put, delete, and update operations that test each record's version number. If the conditional check passes, the operation is allowed, else it fails.

To implement a Conditional Write, include the ConditionExpression to test the version number in your Put, Update and Delete methods.

For this walkthrough, I will use Node.JS and the AWS Node.JS SDK.

Let's first create a table in DynamoDB, to begin with.

const AWS = require("aws-sdk");
const dynamoDb = new AWS.DynamoDB({region: 'us-east-1'});

const createSampleTable = async () => {
 const table = await dynamoDb.createTable({
     TableName: 'notes',
     BillingMode: 'PAY_PER_REQUEST',
     KeySchema: [{
       AttributeName: 'note-id',
       KeyType: 'HASH'
       }],
     AttributeDefinitions:[{
       AttributeName: 'note-id',
       AttributeType: 'S'
     }]
  }).promise();
  console.log('TABLE CREATED');
};
          
createSampleTable();
const AWS = require("aws-sdk");
const dynamoDb = new AWS.DynamoDB({region: 'us-east-1'});

const createSampleTable = async () => {
 const table = await dynamoDb.createTable({
     TableName: 'notes',
     BillingMode: 'PAY_PER_REQUEST',
     KeySchema: [{
       AttributeName: 'note-id',
       KeyType: 'HASH'
       }],
     AttributeDefinitions:[{
       AttributeName: 'note-id',
       AttributeType: 'S'
     }]
  }).promise();
  console.log('TABLE CREATED');
};
          
createSampleTable();

Let's insert one record into the table. When inserting, include your attribute that acts as the version number. For this, I will use the attribute "updatedAt" as my version number for the record.

const documentClient = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

const insertRecordToNotesTable = async () => {
 const tableName = "notes";
 const randomUUID = Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15);
  
  const item = {
   "note-id": randomUUID,
   text: "This is a sample note",
   status: "active",
   updatedAt: Date.now(), // version number
  };

  await documentClient.put({ TableName: tableName, Item: item }).promise();
};
  
insertRecordToNotesTable();
const documentClient = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

const insertRecordToNotesTable = async () => {
 const tableName = "notes";
 const randomUUID = Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15);
  
  const item = {
   "note-id": randomUUID,
   text: "This is a sample note",
   status: "active",
   updatedAt: Date.now(), // version number
  };

  await documentClient.put({ TableName: tableName, Item: item }).promise();
};
  
insertRecordToNotesTable();

Next, let's include a conditional write and update the record we created above by intentionally passing an old value for updatedAt.

const documentClient = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

const triggerWriteError = async () => {
 try {
   const tableName = "notes";
   const noteId = "jkooi6r776ar7h5y3j45za";
   const { Item } = await documentClient
     .get({
     TableName: tableName,
     Key: { "note-id": noteId },
     })
     .promise();
    
   const { updatedAt } = Item;
     // intentionally make this item old by subtracting 5 minutes from the updatedAt.
     // this will help mimic the client containing an older version than DynamoDB.
     const oldUpdatedAt = updatedAt - 5 * 60 * 1000;
    
   await documentClient
     .update({
       TableName: tableName,
       Key: { "note-id": noteId },
       UpdateExpression: "set #updatedAt = :newChangeMadeAt, #status = :newStatus",
       // define the condition expression to implement optimistic concurrency
       
       ConditionExpression: "#updatedAt = :updatedAtHand",
       ExpressionAttributeNames: { "#updatedAt": "updatedAt", "#status": "status" },
       ExpressionAttributeValues: {
       ":newChangeMadeAt": Date.now(),
       ":updatedAtHand": oldUpdatedAt,
       ":newStatus": "inactive"
       },
      }).promise();
    console.log("Updated the item");
  } catch (err) {
      console.log(`Error updating the item because of error - ${err.code}`);
  }
};
      
triggerWriteError();
const documentClient = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

const triggerWriteError = async () => {
 try {
   const tableName = "notes";
   const noteId = "jkooi6r776ar7h5y3j45za";
   const { Item } = await documentClient
     .get({
     TableName: tableName,
     Key: { "note-id": noteId },
     })
     .promise();
    
   const { updatedAt } = Item;
     // intentionally make this item old by subtracting 5 minutes from the updatedAt.
     // this will help mimic the client containing an older version than DynamoDB.
     const oldUpdatedAt = updatedAt - 5 * 60 * 1000;
    
   await documentClient
     .update({
       TableName: tableName,
       Key: { "note-id": noteId },
       UpdateExpression: "set #updatedAt = :newChangeMadeAt, #status = :newStatus",
       // define the condition expression to implement optimistic concurrency
       
       ConditionExpression: "#updatedAt = :updatedAtHand",
       ExpressionAttributeNames: { "#updatedAt": "updatedAt", "#status": "status" },
       ExpressionAttributeValues: {
       ":newChangeMadeAt": Date.now(),
       ":updatedAtHand": oldUpdatedAt,
       ":newStatus": "inactive"
       },
      }).promise();
    console.log("Updated the item");
  } catch (err) {
      console.log(`Error updating the item because of error - ${err.code}`);
  }
};
      
triggerWriteError();

The snippet above utilizes Optimistic Concurrency with ConditionExpression. The expression checks if the value present in the attribute "updatedAt" in the table equals the value currently at hand. No other user has updated the record if these two are the same. But since we intentionally made the record look stale, we get the output shown below as the version number in the table is not the same as on hand.

When DynamoDB throws the error - ConditionalCheckFailedException, Optimistic Concurrency is in action.

To successfully perform an update, the version number in the client and database must be the same.

const { updatedAt } = Item; // record fetched from database
await documentClient
 .update({
   TableName: tableName,
   Key: { "note-id": noteId },
   UpdateExpression: "set #updatedAt = :newChangeMadeAt, #status = :newStatus",
   // define the condition expression to implement optimistic concurrency
   ConditionExpression: "#updatedAt = :updatedAtHand",
   ExpressionAttributeNames: { "#updatedAt": "updatedAt", "#status": "status" },

// when a change occurs, always update the version number so other clients can refresh
   ExpressionAttributeValues: {":newChangeMadeAt": Date.now(), ":updatedAtHand": updatedAt, ":newStatus": "inactive"},
}).promise();
console.log("Updated the item");
const { updatedAt } = Item; // record fetched from database
await documentClient
 .update({
   TableName: tableName,
   Key: { "note-id": noteId },
   UpdateExpression: "set #updatedAt = :newChangeMadeAt, #status = :newStatus",
   // define the condition expression to implement optimistic concurrency
   ConditionExpression: "#updatedAt = :updatedAtHand",
   ExpressionAttributeNames: { "#updatedAt": "updatedAt", "#status": "status" },

// when a change occurs, always update the version number so other clients can refresh
   ExpressionAttributeValues: {":newChangeMadeAt": Date.now(), ":updatedAtHand": updatedAt, ":newStatus": "inactive"},
}).promise();
console.log("Updated the item");

The snippet shown above modifies the initial ConditionalExpression by using the version number the same as the one in the database (ConditionExpression: "#updatedAt = :updatedAtHand").

When the updated code gets executed, it runs with no errors as the client's object is the same record present in the database.

Make sure to update the version number when a successful write occurs to the record in the database. The snippet above highlights this process as it updates the value ":newChangeMadeAt" with the current time. It ensures that clients' future requests containing the old version get rejected by DynamoDB.

Benefits of Optimistic Locking

Using Optimistic Locking has several benefits:

1. It ensures that stale data does not get edited in your application.
2. It notifies when DynamoDB detects that you have an old version of a record.
3. It does not require you to enforce a lock on the actual database resource, meaning faster operations.
4. It is suitable for scenarios where data contention is low.
5. It allows for higher throughput as it avoids locking resources.

Pessimistic Locking

What is Pessimistic Locking?

Pessimistic Locking is another strategy used by DynamoDB to prevent concurrent updates to a particular row. While Optimistic Locking sent version numbers to the client, Pessimistic Locking aims to avoid simultaneous updates from the database without maintaining version numbers.

How does Pessimistic Locking work?

Most database servers obtain a lock when a transaction starts and only release the lock after finishing the transaction. However, DynamoDB does this a bit differently. It does not lock the data under consideration. Instead, DynamoDB utilizes Transactions to identify changes that other threads made to the data under review, and if a change gets detected, DynamoDB cancels the transaction and throws an error. It helps perform atomic operations concurrently while maintaining data integrity. In addition, it ensures that the client always reads and updates the latest version.

How to use Pessimistic Locking?

DynamoDB offers a Transact Write API that helps implement DynamoDB's version of Pessimistic Locking. When used, DynamoDB does not lock the items that are getting processed, but it monitors the items, and if another thread modifies the data, the entire transaction fails due to the data change and rolls back the data.

To use this approach in DynamoDB, we use the method - transactWrite.

Let us continue with the sample Node.JS project used above to use the Transact Write API.

The TransactWrite method looks as shown below.

// asynchronous method
await documentClient.transactWrite({
 TransactItems: [
   {
     Update: {},
   },
 ],
}).promise();
// asynchronous method
await documentClient.transactWrite({
 TransactItems: [
   {
     Update: {},
   },
 ],
}).promise();

The method accepts an object consisting of TransactItems. In addition, it accepts an array of modification objects, including "Put," "Delete," and "Update (used above)." It is also important to note that DynamoDB supports a maximum of 25 items per transaction.

Inside the "Update" object, you can place the items to execute in the transaction. The snippet shown below highlights this.

const AWS = require("aws-sdk");
const documentClient = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

const transactionWriteDynamo = async () => {
   const noteId = "jkooi6r776ar7h5y3j45za";
   const tableName = "notes";
   await documentClient.transactWrite({
     TransactItems: [
       {
         Update: {
           TableName: tableName,
           Key: { "note-id": noteId },
           UpdateExpression:
           "set #updatedAt = :newChangeMadeAt, #status = :newStatus",
           ExpressionAttributeNames: {
           "#updatedAt": "updatedAt",
           "#status": "status",
           },
           ExpressionAttributeValues: {
           ":newChangeMadeAt": Date.now(),
           ":newStatus": "inactive",
         },
        },
       },
     ],
    }).promise();
  console.log("Updated Note - " + noteId);
};
            
transactionWriteDynamo();
const AWS = require("aws-sdk");
const documentClient = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

const transactionWriteDynamo = async () => {
   const noteId = "jkooi6r776ar7h5y3j45za";
   const tableName = "notes";
   await documentClient.transactWrite({
     TransactItems: [
       {
         Update: {
           TableName: tableName,
           Key: { "note-id": noteId },
           UpdateExpression:
           "set #updatedAt = :newChangeMadeAt, #status = :newStatus",
           ExpressionAttributeNames: {
           "#updatedAt": "updatedAt",
           "#status": "status",
           },
           ExpressionAttributeValues: {
           ":newChangeMadeAt": Date.now(),
           ":newStatus": "inactive",
         },
        },
       },
     ],
    }).promise();
  console.log("Updated Note - " + noteId);
};
            
transactionWriteDynamo();

The output for the snippet above is shown below.

In the code snippet above, the note gets updated in a transaction where its status and updatedAt are modified. While this occurs, if another thread modifies its data, the transaction gets canceled and is rolled back.

Another important note is that you can include many data items to modify across multiple tables in transactions. Therefore, if one item in the transaction gets modified by another thread, the entire transaction is canceled and rolled back. This approach is called the "All-or-nothing" approach, as all data made in the transaction get rolled back in case one fails.

Benefits of Pessimistic Locking

1. It allows you to perform updates across multiple tables, with rollbacks occurring when one item in the transaction gets modified.
2. All changes are atomic.
3. You prevent conflicts entirely from occurring, meaning that you do not have to deal with them.
4. It is suitable for high-contention scenarios where data is frequently accessed and modified.
5. It ensures data consistency across multiple operations and tables.

Optimistic vs. Pessimistic Locking - What should you use?

Optimistic Locking vs. Pessimistic Locking

First, let us look at the main difference between Optimistic and Pessimistic Locking in DynamoDB.

Optimistic Locking

1. DynamoDB allows the update only if the client has the latest version at hand.
2. It can provide fast concurrent access but occasionally throw an error when a version mismatch is detected.
3. It assumes that concurrent access to data items is rare.

Pessimistic Locking

1. DynamoDB allows multiple threads to update the item. However, when a transaction detects that another thread has already modified the data, it aborts the transaction and rolls the data back.
2. It monitors other threads making changes to the data. Therefore, it has a higher performance overhead than Optimistic Locking.
3. It assumes that data received is accessed and modified concurrently at a high frequency.

What should you use?

There is no right or wrong concurrency control approach. It depends on the use case. But, here are some guidelines that may help you determine the right concurrency control approach for you.

You may use Optimistic Locking if:

1. Your data is updated concurrently at a low frequency.
2. Your data is mainly read and not updated frequently.

You may use Pessimistic Locking if:

1. Your data is updated concurrently at a high frequency.
2. Your data gets updated across multiple tables. This way, even if another thread modifies one data item in your transaction, all items get rolled back.
3. Your data gets read and updated frequently by many users.

Conclusion

This article provided a comparison of Optimistic and Pessimistic Locking in DynamoDB. It also discussed when to use either of these.

I hope this article helps you select the appropriate concurrency control for your use case so that your application data maintains its integrity.

The code used in the article is accessible in my GitHub repository.

Thank you for reading.

FAQs

Does DynamoDB have Locking by default?

No. DynamoDB does not use any locking by default. Therefore, when you start using DynamoDB, implement locking to maintain data integrity.