dynobase-icon
Dynobase

How to run DynamoDB Local and Offline - Complete Guide

Written by Rafal Wilinski

Published on 2020-04-17

Imagine you're traveling by plane to a different city, and the deadline for your DynamoDB-powered project is really tight. Obviously, during flight, there's no internet, but you just had one of these "Aha!" moments. You want to rush into the implementation, but there's one problem:

Since DynamoDB is fully managed NoSQL database, you cannot access it when you're offline. Bummer.

This didn't happened to me just once. Until recently, when I discovered I can run DynamoDB offline and locally on my computer, without AWS Cloud. In some scenarios, it makes developing and testing applications without accessing DynamoDB easier. In this guide, I'll show you how you can do that too. There are a few ways to do this; let's run through them below:

Different ways to setup DynamoDB locally

Using Offline DynamoDB Local

Once you started DynamoDB offline, you might also be wondering - How do I connect to it?

Troubleshooting

Using Docker

This is my favorite way because it does not require Java installed. In this setup, DynamoDB is running in an isolated container on the port of your choice. Another good thing about this solution is that it works the same across all the operating systems and can be easily added to your Docker Compose setup.

  1. Make sure you have Docker installed. If you don't have Docker yet, you can get it here.

  2. Open terminal and type:

And that's pretty much it.

Your DynamoDB local instance is now running on port 8000. If you want to connect to this container using SDK or CLI, don't forget to change the endpoint parameter in the configuration. Otherwise, you'll keep trying to connect to the AWS network.

Using Docker Compose

A bit better way to run DynamoDB locally is to use Docker Compose. Using this approach, you can configure persistent volumes, network and resources much easier without adding parameters to the docker run command. Moreover, a YAML file is easier to maintain and share as a file inside repo.

  1. Create a file called docker-compose.yml and paste following code:
  1. Run docker-compose up -d. You can skip -d flag if you don't want to run in in the "detached" mode.

Using Java

In this setup, we're running Java binary in our system without any containerization. It's a bit more complicated

  1. First, make sure you have Java Runtime Environment (JRE) version 6.x or newer already installed. It's required to run DynamoDB locally.

  2. Second, download DynamoDB

  3. Then, extract the archive and copy its contents to a location of your choice.

  4. Lastly, open terminal in the location where you've extracted files in step 3 and enter the following command:

DynamoDB is now running on port 8000. If you want to change it, use -port flag.

Similar to the Docker setup, you need to change the endpoint parameter in the configuration.

Running out of memory

If you're going to use DynamoDB really heavily, it's possible that the allocated amount of memory for your JVM might not be enough. In a case like that, use a combination of Xms and Xmx flags to adjust the amount of RAM. For example:

This will start your local DynamoDB instance with 256 MB of memory and allow the process to use up to 2048 MB of memory.

With Serverless Framework

Using DynamoDB with Serverless Framework is a pretty popular combination. DDB in a case like this one is perfect for running offline integration tests or just running local replica of the AWS environment for development purposes. Here's how:

  1. Run following command:
  1. Open serverless.yml file and add the following entry to the plugins section:
  1. Now, run the following command to let the plugin setup everything for us, including downloading DynamoDB Local:
  1. If you don't have DynamoDB tables defined in your serverless.yml file, do it now.

  2. Finally, start DynamoDB Local and migrate by running this command:

For more information, checkout the plugin docs.

Using LocalStack

LocalStack is a project aiming to mock most of the AWS resources locally. Once again, in my opinion, using Localstack is much easier with Docker. Running DynamoDB local with it look like this:

This will run a container exposing three ports:

  • 4572 running S3
  • 4569 running DynamoDB
  • 8080 running Localstack's status dashboard

Similar to the previous approaches, if we want to use our offline services, we need to set the endpoint parameter of SDK or CLI to http://localhost:4569. If you don't want to set the endpoint setting explicitly when using CLI with each command, alternatively, you can use awscli-local.

With AWS Amplify mock

This method is only useful when you're working with Amplify-powered projects.

One of the AWS Amplify toolchain options is to mock some of the cloud functionality locally. In our case, when working with an Amplify-powered API, you can run command amplify mock api and it will not only mock your API, but also the DynamoDB which is used as a persistence layer for this interface. Under the hood, it will simply spin a Java process with a copy of Java-based local DynamoDB.

The database will be ran with following parameters:

  • Port: 62224
  • Region: us-fake-1
  • AWS Access Key ID: fake

Connecting to DynamoDB Local using SDK

All you need to do, is to tell the DynamoDB or DocumentClient to use the local endpoint in the constructor.

In Node.js

In Python

In Golang

Connecting to DynamoDB Local using CLI

Similarly to the SDK, in order to use DynamoDB Local in the CLI you need to end all your commands with --endpoint-url=<endpoint>:<port>.

Example:

Connecting to DynamoDB Local in Dynobase

Dynobase UI with DynamoDB offline

Even though DynamoDB offline comes with an interactive shell that allows you to query and modify data inside the database using a web-based console, I found it a bit clunky. Fortunately, Dynobase works with DynamoDB local just like with the AWS managed one.

Dynobase is a Professional GUI Client for DynamoDB

Start your 7-day free trial today



Troubleshooting

How can I check if my DynamoDB is already running?

On Mac and Linux, you can enter following command in your Terminal:

If you can see an entry looking something like this:

Then it means that your Local DynamoDB started successfully and you can start using it right away.

Alternatively, if you know on which port it should be running, you can use following command to verify if it's running correctly (replace 62224 with your port):

I know that my DynamoDB Local has start? How can I find its port?

Java-based distribution of DynamoDB Local uses port 8000 by default. If it's not available, DynamoDB Local shouldn't even start. If it was started with custom port (like in Amplify's case), you can run following command:

And find a number supplied as -port. For example, in following result:

You can see number 8080 after -port which indicates that DynamoDB local has been started on port 8080.

Error - You must specify a region

What? I'm trying to connect to my local DynamoDB, there are no regions! This error is a not an DynamoDB specific error, it can happen with any AWS API/CLI/SDK command if you're not having a default AWS region setup in your environment variables or in credentials configuration.

To solve it, you can either set the AWS_DEFAULT_REGION environment variable to something like us-east-1:

or end your command with --region=us-east-1:

Error - Unable to locate credentials.

Similarly to the previous one, SDK or CLI is failing to find AWS Access and Secret Keys. Don't worry, they don't have to be real. DynamoDB offline is not validating them, you just need to provide something, it cannot be blank. You have two options here. Either provide them as environment variables:

or add them as a "profile" using aws configure command and reference using --profile flag.

NetworkError - EPROTO

This one might look scary but it is very likely that you're trying to connect to the local DynamoDB instance using https protocol. Change that to http and you should be fine:

I can't connect to my DynamoDB Offline in Dynobase

First, get your DynamoDB Local settings and verify that you can connect to it from the CLI or SDK. You can use following command to verify that parameters you're having are correct:

If following command responds with data, it means that parameters you're using are correct and you can use them inside Dynobase for Offline Connection setting. You can change them inside View -> Settings -> Offline Settings section.

Dynobase Offline Settings

Dynobase is a Professional GUI Client for DynamoDB

Start your 7-day free trial today

Product Features

© 2021 Dynobase

+
Have you tried Dynobase yet?
Try Dynobase to accelerate your DynamoDB workflow. Start your 7-day free trial today.