Today I will look at the NoSQL concept and specifically Amazon’s DynamoDB.
NoSQL stands for “No SQL”, which means you don’t use SQL (Structured Query Language) to manage the database. Data is not arranged into relational tables (with relational indices that point to other tables), but is arranged in a variety of data models, including document, graph, key-value, and columnar.
NoSQL was born because of the need for scalability and performance. In the early 2000s it was discovered that relational databases do not scale well at a reasonable cost.
DynamoDB is an Amazon NoSQL database that runs in Amazon’s cloud or locally on a personal computer.
AWS DynamoDB
I created a new DynamoDB table through the AWS Console and populated it with some data in about 5 minutes:
Local DynamoDB
Then, I downloaded and installed the local version of DynamoDB as well. I got it up and running with the terminal command:
$ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb
Interacting with DynamoDB from Java
First, I needed an AWS access key. To get one, I went to the AWS IAM Management console and created a new group, named DynamoDB_Users and gave the group AmazonDynamoDBFullAccess permissions. Then, I created a new user, named lowell_list, and added this user to the new group. On successful user creation, I was given an AWS access key and password. I saved these credentials in a new ~/.aws/credentials file.
Then, I needed to create a Java project that includes the AWS SDK for Java. I used Maven to create a quickstart template that includes the SDK:
$ mvn -B archetype:generate \ -DarchetypeGroupId=org.apache.maven.archetypes \ -DgroupId=com.lowelllist.test.dynamodb \ -DartifactId=dynamodb_test_app
Then, I added the latest version of the AWS SDK as a dependency to the Maven pom.xml file:
<dependencyManagement>
<dependencies>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bom -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>1.11.234</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Now the project is buildable:
$ mvn package
and runnable:
$ mvn exec:java
Next, some Java code to create a table and insert some movie data:
package com.lowelllist.test.dynamodb;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.model.*;
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.io.File;
import java.util.Arrays;
import java.util.Iterator;
/**
* Interact with a local DynamoDB database!
*/
public class App {
public static void main(String[] args) throws Exception {
// connect to the DB
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
.withEndpointConfiguration(
new AwsClientBuilder.EndpointConfiguration(
"http://localhost:8000",
"us-west-2"
))
.build();
DynamoDB dynamoDB = new DynamoDB(client);
// attempt to create table
try {
System.out.println("Attempting to create table; please wait...");
String tableName = "Movies";
Table table = dynamoDB.createTable(tableName,
Arrays.asList(new KeySchemaElement("year", KeyType.HASH), // partition key
new KeySchemaElement("title", KeyType.RANGE)), // sort key
Arrays.asList(new AttributeDefinition("year", ScalarAttributeType.N),
new AttributeDefinition("title", ScalarAttributeType.S)),
new ProvisionedThroughput(10L, 10L));
table.waitForActive();
System.out.println("Success. Table status: " + table.getDescription().getTableStatus());
}
catch (Exception e) {
System.err.println("Unable to create table: ");
System.err.println(e.getMessage());
}
// add some records
Table table = dynamoDB.getTable("Movies");
JsonParser parser = new JsonFactory().createParser(new File("moviedata.json"));
JsonNode rootNode = new ObjectMapper().readTree(parser);
Iterator iter = rootNode.iterator();
ObjectNode currentNode;
while (iter.hasNext()) {
currentNode = (ObjectNode) iter.next();
int year = currentNode.path("year").asInt();
String title = currentNode.path("title").asText();
try {
table.putItem(new Item().withPrimaryKey("year", year, "title", title).withJSON("info",
currentNode.path("info").toString()));
System.out.println("PutItem succeeded: " + year + " " + title);
}
catch (Exception e) {
System.err.println("Unable to add movie: " + year + " " + title);
System.err.println(e.getMessage());
break;
}
}
parser.close();
}
}
It is also useful to interact with the database using the AWS command line tools. This command queries the local database and prints all movies from the year 1999:
$ aws dynamodb query --table-name Movies \ --key-condition-expression "#yr = :yyyy" \ --expression-attribute-values file://expression-attributes.json \ --expression-attribute-names file://expression-names.json \ --endpoint-url http://localhost:8000
where expression-attributes.json is:
{
":yyyy": {"N": "1999"}
}
and expression-names.json is:
{"#yr": "year"}