Moving data from DocumentDB

In this article we’re going to look at extracting data from Microsoft Azure’s DocumentDB service and then importing the data into IBM Cloudant.

Getting the data out🔗

Once again, I’ve written a script to do this for you: documentdbexport.

First, install the tool:

$ npm install -g documentdbexport

Define a couple of environment variables with your Azure credentials:

$ export AZURE_ENDPOINT="https://mydocumentdb.documents.azure.com:443/"
$ export AZURE_KEY="GeIZysnonvgpk2"

Then, simply run documentdbexport, supplying the name of the database and collection to export:

$ documentdbexport --database iot --collection temperaturereadings
{"temperature":30730,"time":"2017-03-09T02:21:48+0000","id":"1489026108"}
{"temperature":17072,"time":"2017-03-09T02:15:22+0000","id":"1489025722"}
{"temperature":18177,"time":"2017-03-08T21:27:23+0000","id":"1489008443"}
Export complete { records: 3, time: 0.145 }

The tool makes as many API calls as it needs to extract the data, converting the JSON to a more compact form as it goes.

Importing into CouchDB/Cloudant🔗

We can use couchimport to do the import stage for us. Install it with:

$ npm install -g couchimport

Set an environment variable with your target Cloudant/CouchDB service’s URL:

$ export COUCH_URL="https://MYUSERNAME:MYPASSWORD@MYHOST.cloudant.com"

Then, run both the documentdbexport and couchimport commands together, piping the output of the former into the latter:

$ documentdbexport --database iot --collection temperaturereadings | couchimport --db iot --type jsonl

The --type jsonl parameter tells couchimport that it is to expect one JSON document per line and --db iot defines the name of the target database.

It’s that simple! You’ll find more details on command-line usage and programmatic access for documentdbexport on npm.