February 22, 2018 | Glynn Bird | Replication CLI

One of Apache CouchDBโ„ขโ€™s killer features is replication. JSON data is easily replicated between data centers, from a mobile device to the cloud, or vice versa. The CouchDB replication protocol is shared by Apache CouchDB itself, the IBM Cloudant database-as-a-service, Cloudant Sync libraries for iOS and Android and the PouchDB in-browser database.

Setting up a single replication is as easy as filling in a form in the Replication tab of your Cloudant or CouchDB dashboard:


But what if you have hundreds of databases to replicate? What if you want to move lots of small databases, but only want to replicate 5 at any one time? This is where couchreplicate is here to help.

couchreplicate is a command-line tool that allows you to kick off and monitor replications with minimal effort.

Installing couchreplicate ๐Ÿ”—

Assuming you already have Node.js and npm installed on your machine, couchreplicate is installed with a single line:

$ npm install -g couchreplicate

You should now have the couchreplicate tool installed on your machine. Check with:

$ couchreplicate --help

Replicating a single database ๐Ÿ”—

Migrating one database is as easy as running couchreplicate, supplying your source URL and target URL.

$ couchreplicate -s http://u:p@localhost:5984/mysource -t
cities [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 32% 21.1s triggered

Your replication will start and you should see a progress bar start to fill up. You can also supply the database name as a separate -d parameter:

$ couchreplicate -d food -s http://u:p@localhost:5984-t

And that database name will be used at both the source and target ends.

Replicating multiple databases ๐Ÿ”—

The tool really comes into its own when replicating more than one database. Simply supply a comma-separated list of database names in your -d parameter:

$ couchreplicate -d food,drink,hardware,software -s http://u:p@localhost:5984-t
food  [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡]100% 42.5s completed
drink [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 32% 21.1s triggered

As one replication completes, another will be scheduled, until the entire list is exhausted.

You can control the number of simultaneous replications with the -c parameter:

$ couchreplicate -c 3 -d food,drink,hardware,software -s http://u:p@localhost:5984-t
food     [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”] 82% 34.5s triggered
drink    [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 63% 34.1s triggered
hardware [โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 10% 34.1s triggered

Replicating all of your databases ๐Ÿ”—

Instead of supplying a list of database names, the -a option can be used to indicate that all of the databases in the source cluster are to be migrated. This is handy if you are moving from local CouchDB to Cloudant, or from one Cloudant service to another.

$ couchreplicate -c 3 -a -s http://u:p@localhost:5984-t

Itโ€™s all about the open source ๐Ÿ”—

The couchreplicate tool is free to use and is open-sourced under the Apache 2.0 license, so you can fork the code yourself and modify as you see fit. If you find a bug or have a contribution, weโ€™d love to hear from you on the projectโ€™s Github page.

Further reading ๐Ÿ”—

If you like using command-line tools, then there more CouchDB- and Cloudant-compatible tools from the same stable:

Developers can use Node.js and Cloudant using the following libraries:

Apacheยฎ, Apache CouchDBโ„ข, CouchDBโ„ข, and the red couch logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.