Command-line tools

Some developers spend their days dragging and dropping in a graphical user interface, others are more comfortable typing green letters into a black background on a command-line terminal. If you are the latter type of developer, then this blog post is for you. We introduce a range of command-line tools that you can use to interface with IBM Cloudant (or plain Apache CouchDB).

Cloudant and CouchDB share an RESTful HTTP API allowing access from any programming language or from the command-line using the curl utility. The packages featured in this blog post are all free to download and open-source, allowing you to fork and modify them for your own purposes.

ccurl🔗

Most Cloudant & CouchDB developers use curl to access the RESTful HTTP API. The trouble with curl is that the commands can get overly long, with lots of repetition between commands, for example:

curl -X POST -H 'Content-type: application/json' -g "https://myusername:mypassword@myhost.cloudant.com/mydb"
  -d '{"val": "json"}'

The utility ccurl is a wrapper around curl that removes some of the repetition. It adds the following features:

• The protocol, username, password and hostname are not required; instead they are taken from an environment variable.
• The content-type header
• The "-g" fix

Configuring ccurl is a one-off task: simply set your Cloudant or CouchDB URL as the COUCH_URL environment variable:

export COUCH_URL="https://myusername:mypassword@myhost.cloudant.com"

or

export COUCH_URL="http://localhost:5984"

ccurl makes command-line API requests much less verbose, as these examples show:

# fetch stats about database ‘mydb’
ccurl /mydb

# fetch single document
ccurl /mydb/12345

# add a document
ccurl -X POST -d'{"a":1,"b":2}' /mydb

jq🔗

Simplify dealing with JSON on the command line by installing the jq utility, which parses and filters JSON strings. In this example, jq takes the stream of data coming from ccurl, and turns it into nicely formatted, coloured terminal output:

ccurl /mydb/12345 | jq .

You can supply arguments to the jq command. For example:

ccurl /mydb/12345 | jq .geometry.coordinates

fetches the coordinates from the geometry object included within the JSON object returned by the ccurl command. jq has its own syntax that allows JSON objects to be filtered, manipulated and queried. See the jq website for further details.

couchshell🔗

If you’re familiar with the file and directory commands of a Unix shell, then you should find couchshell intuitive to use. It presents a Cloudant account or CouchDB installation as a directory tree, with top-level directories being databases, and their contents being documents. It uses the same environment variable as ccurl and can be invoked by typing couchshell. The result is you find yourself in a shell-like environment, with a >> prompt. The environment is optimized for working with Couch and Cloudant commands with “tab autocomplete” of database names and document ids:

The above sequence of commands creates a database, creates two documents, and deletes one of them.

A full list of the couchshell commands is provided on the tool’s website.

couchimport🔗

If you have CSV files containing data which you need to upload to Cloudant or CouchDB, then couchimport can import the files. The following sequence of shell commands download a dataset containing US crime data, unzip it, creates a new CouchDB database, and finally imports the CSV data by piping the file to the couchimport:

Note: The crime data for the curl command below has been removed

curl 'https://data.octo.dc.gov/feeds/crime_incidents/archive/crime_incidents_2013_CSV.zip' > crime.zip
unzip crime.zip
ccurl -X PUT /crime
cat crime_incidents_2013_CSV.csv | couchimport --db crime --delimiter ","

Once again, couchimport uses the same COUCH_URL environment variable to determine the database to write to. couchimport can also be used programmatically to allow your Node.js applications to import CSV files or streams into CouchDB or Cloudant.

couchbackup🔗

If you need to backup and restore CouchDB data, then couchbackup and couchrestore utilities can help. Backup is as simple as running the couchbackup; in this case taking a copy of the animals database and saving it to the file animals.txt:

couchbackup --db animals > animals.txt

Restoring a backup is the reverse operation - pipe the file to couchrestore:

cat animals.txt | couchrestore --db animals

We can compress the data using standard compression utilities:

couchbackup --db animals | gzip > animals.txt.gz
cat animals.txt.gz | gunzip | couchrestore --db animals

couchbackup takes a shallow copy of a Cloudant database with only the winning revisions being backed up.

couchmigrate🔗

When changing a database’s design documents, you need to take care that users of the database don’t suffer performance issues as the new index rebuilds. The couchmigrate utility creates a new design document, waits for the index to build, and finally makes the index live.

For example, if our new design document is in a file dd.json, we could run the following command:

couchmigrate --dd dd.json --db movies

This command blocks use until the views defined in dd.json are ready to use.