dynobase-icon
Dynobase

DynamoDB Python Boto3 Query Cheat Sheet [14 Examples]

Rafal Wilinski

Written by Rafal Wilinski

Published on February 27th, 2020

Still using AWS console to work with DynamoDB? 🙈

Time to 10x your DynamoDB productivity with Dynobase [learn more]

What is Boto3?

Boto3 is a Python library for AWS (Amazon Web Services), which helps interacting with their services including DynamoDB - you can think of it as DynamoDB Python SDK. It empowers developers to manage and create AWS resources and DynamoDB Tables and Items.

If you're looking for similar guide but for Node.js, you can find it here, for Java, for Rust, and for Golang / Go here.

List of DynamoDB Boto3 Query Examples

Dynobase Codegen Feature

Connecting Boto3 to DynamoDB

Connecting to DynamoDB with boto3 is simple if you want to do that using Access and Secret Key combination:

Keep in mind that using access and secret keys is against best security practices, and you should instead use IAM roles/policies to interact with DynamoDB. Boto3, if ran on Lamba function or EC2 instance, will automatically consume IAM Role attached to it.

Create Table with Boto3

DynamoDB structures data in tables, so if you want to save some data to DynamoDB, first you need to create a table. You can do that using AWS Console, AWS CLI or using boto3, like this:

Keep in mind that provisioning a table takes some before it's active. If you want to know when it's ready to be used, you can use waiter function.

Delete table

If you changed your mind and need to remove DynamoDB table, don't worry, it's simple:

List Tables

If you want to check what tables are available at your disposal in current region, use list-tables call. Keep in mind that if selected region has more than 100 tables you'll have to paginate through them to fetch a complete list.

Boto3 Get All Items aka Scan

To get all items from DynamoDB table, you can use Scan operation. The problem is that Scan has 1 MB limit on the amount of data it will return in a request, so we need to paginate through the results in a loop.

You can apply FilterExpression attribute in order to filter the results like this:

Boto3 Get Item

To get a single item from DynamoDB using Partition Key (and Sort Key if using composite key), you can use GetItem operation.

Keep in mind to replace primaryKeyName and sortKeyName with actual keys from your table.

Boto3 Batch Get Item

If you want to retrieve multiple items identified by a key(s) in one call, use batch_get_item call with the following syntax:

Keep in mind that batch_get_item is limited to 100 items and 16 MB of data.

Boto3 Put Item

To write a single item into the DynamoDB Table, use PutItem operation:

Boto3 Query for a set of items

Alternative way to get a collection of items is the Query method. Query is much faster than Scan because it uses Indexes. It should be your preferred way to get a collection of items with the same partition key.

Keep in mind that Query can return up to 1MB of data and you can also use FilterExpressions here to narrow the results on non-key attributes.

If you don't know how to construct your Query, use Dynobase with Query Code Generation feature which will automatically generate it for you.

Dynobase Codegen Feature


Query an Index

DynamoDB allows querying not only on the main table index, but also on LSIs (Local Secondary Indexes) and GSIs (Global Secondary Indexes).

To do that in SDK, you need to change two things:

  • Specify the index you want to query using IndexName parameter
  • Provide correct KeyConditionExpression with corresponding ExpressionAttributeValues and ExpressionAttributeNames`

So, to give you an example. Imagine your table is having a Global Secondary Index called GSI1 with key attribute gsi1pk. If you'd like to query that index in Node.js, it will look like this:

Boto3 Update Item

DynamoDB update_item operation consists of three primary attributes:

  • Key - which object should be updated
  • ExpressionAttributeValues - map with new values
  • UpdateExpression - how these new values should be applied to the object in the table

They can be used like this:

Boto3 Conditionally Update Item

Moreover, you can also add a ConditionExpression parameter, which restricts the update logic only if the evaluated expression equals true.

Boto3 Increment Item Attribute

Incrementing a Number value in DynamoDB item can be achieved in two ways:

  1. Fetch item, update the value with code and send a Put request overwriting item
  2. Using update_item operation.

While it might be tempting to use first method because Update syntax is unfriendly, I strongly recommend using second one because of the fact it's much faster (requires only one request) and atomic (imagine value updated by other client after you fetched item).

To do that using single update_item operation, use following syntax:

Boto3 Delete Item

Deleting a single item from DynamoDB table is similar to GetItem operation. Key argument accepts primary key and sort/range key if table has composite key.

Boto3 Delete All Items

Unfortunately, there's no easy way to delete all items from DynamoDB just like in SQL-based databases by using DELETE FROM my-table;. To achieve the same result in DynamoDB, you need to query/scan to get all the items in a table using pagination until all items are scanned and then perform delete operation one-by-one on each record.

Fortunately, this is possible just with 3 clicks using Dynobase.

Boto3 Query with Sorting

Unfortunately, DynamoDB offers only one way of sorting the results on the database side - using the sort key. If your table does not have one, your sorting capabilities are limited to sorting items in application code after fetching the results. However, if you need to sort DynamoDB results on sort key descending or ascending, you can use following syntax:

Boto3 Query Pagination

Similar to Scan operation, Query returns results up to 1MB of items. If you need to fetch more records, you need to issue a second call to fetch the next page of results. If LastEvaluatedKey was present in response object, this table has more items like requested and another call with ExclusiveStartKey should be sent to fetch more of them:

Using Boto3 with DynamoDB Local/Offline

If you need to use DynamoDB offline locally, you can use DynamoDB local distributed by AWS or DynamoDB from Localstack. Connecting to it is as easy as changing the endpoint parameter in boto3.resource call.

Learn more about running and using DynamoDB locally.

Spend less time in the AWS console, use Dynobase.

First 7 days are on us. No strings attached.

Product Features

© 2022 Dynobase