DynamoDB Indexes Explained [Global & Local Secondary]

What makes DynamoDB so much more than just a simple Key-Value store is the secondary indexes. They allow you to quickly query and lookup items based on not only the primary index attributes, but also attributes of your choice. Secondary Indexes, unlike primary keys, are not required, and they don't have to be unique. Generally speaking, they allow much more flexible query access patterns.

Different DynamoDB Indexes

• Local Secondary Index
• Global Secondary Index

There are also a few flavors of "indexes":

• Sparse index
• Inverted index

DynamoDB Local Secondary Index (LSI)

Local Secondary Indexes use the same hash key as the primary index but allow you to use a different sort key. That also means that they can be created only on tables with composite primary key.

Additionally, LSIs:

• Limit you to only 10GB of data per Hash/Partition Key.
• Unlike GSIs, they share throughput with the base table - if you query for data using LSI, the usage will be calculated against the capacity of the underlying table and its base index.
• They have to be specified at table creation - you can't add or remove them after provisioning the table.

DynamoDB Global Secondary Index (GSI)

Global Secondary Indexes are much more flexible and don't have LSI limitations. They don't have to share the same hash key, and they can also be created on a table with a simple key schema. Most of the time, you'll find yourself using GSIs over LSIs because they enable much more flexible query access patterns.

Moreover, GSIs:

• Don't have that 10GB of data per Hash/Partition Key limit like LSIs have.
• Global Secondary Indexes don't have to be unique! Two items can have the same partition and sort key pair on a GSI.
• They don't share throughput with the base table. Each of the GSIs is billed independently, and throttling is also separated as a consequence.
• GSIs can be altered after the table has been created. You can add and delete them whenever you want. Moreover, if you create a GSI on an attribute already defined in a collection of items, the index will also be backfilled with that data.
• They are eventually consistent - if you're writing an item to a table, it is asynchronously propagated to the rest of GSIs. It means that query results might sometimes not be consistent and you should be aware of that when creating your application. However, if you want to get the most up-to-date data, you can use "Strongly Consistent Reads". They cost x2 more, they might have a little bit bigger latency and you cannot use them on GSIs but they reflect the most accurate state of the database.

Sparse index

Sparse Index is a special type of GSI that allows you to index only a subset of the collection by indexing an attribute that is not present on all the items. This technique is useful to quickly query for a set of items that have a specific attribute value, e.g. only rows that have an attribute deletedAt defined. To create a sparse index, make sure that only items that should be in that index have a sort key value of that index present.

Sparse Indexes are a great replacement for queries with FilterExpressions that include a contains operator.

Inverted index

Inverted Index is a GSI that is basically a primary key but inversed - table's hash key becomes inverted index's sort key and table's sort key becomes inverted index's hash key.

Inverted Indexes are helpful in bidirectional many-to-many relationships, e.g. defining connections between cities or friendships between users.

Best Practices for Using DynamoDB Indexes

When designing your DynamoDB tables and indexes, consider the following best practices:

• Plan your access patterns: Before creating your table, think about how you will query your data. This will help you decide which attributes to index.
• Use sparse indexes wisely: Sparse indexes can save storage and improve query performance by indexing only a subset of items.
• Monitor and adjust throughput: Regularly monitor your table and index usage to ensure you have provisioned the appropriate throughput. Adjust as necessary to avoid throttling.
• Leverage GSIs for flexibility: Use GSIs to enable flexible query patterns that your primary index does not support.
• Understand eventual consistency: Be aware that GSIs are eventually consistent, which means there might be a delay before changes are reflected in the index. Use strongly consistent reads if immediate consistency is required, but note that they are not available for GSIs.