Cloudant blog Home Search


One of Apache CouchDBโ„ข’s killer features is replication. JSON data is easily replicated between data centers, from a mobile device to the cloud, or vice versa. The CouchDB replication protocol is shared by Apache CouchDB itself, the IBM Cloudant database-as-a-service, Cloudant Sync libraries for iOS and Android and the PouchDB in-browser database.

Setting up a single replication is as easy as filling in a form in the Replication tab of your Cloudant or CouchDB dashboard:


But what if you have hundreds of databases to replicate? What if you want to move lots of small databases, but only want to replicate 5 at any one time? This is where couchreplicate is here to help.

couchreplicate is a command-line tool that allows you to kick off and monitor replications with minimal effort.

Installing couchreplicate๐Ÿ”—

Assuming you already have Node.js and npm installed on your machine, couchreplicate is installed with a single line:

$ npm install -g couchreplicate

You should now have the couchreplicate tool installed on your machine. Check with:

$ couchreplicate --help

Replicating a single database๐Ÿ”—

Migrating one database is as easy as running couchreplicate, supplying your source URL and target URL.

$ couchreplicate -s http://u:p@localhost:5984/mysource -t
cities [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 32% 21.1s triggered

Your replication will start and you should see a progress bar start to fill up. You can also supply the database name as a separate -d parameter:

$ couchreplicate -d food -s http://u:p@localhost:5984-t

And that database name will be used at both the source and target ends.

Replicating multiple databases๐Ÿ”—

The tool really comes into its own when replicating more than one database. Simply supply a comma-separated list of database names in your -d parameter:

$ couchreplicate -d food,drink,hardware,software -s http://u:p@localhost:5984-t
food  [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡]100% 42.5s completed
drink [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 32% 21.1s triggered

As one replication completes, another will be scheduled, until the entire list is exhausted.

You can control the number of simultaneous replications with the -c parameter:

$ couchreplicate -c 3 -d food,drink,hardware,software -s http://u:p@localhost:5984-t
food     [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”] 82% 34.5s triggered
drink    [โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 63% 34.1s triggered
hardware [โ–‡โ–‡โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”โ€”] 10% 34.1s triggered

Replicating all of your databases๐Ÿ”—

Instead of supplying a list of database names, the -a option can be used to indicate that all of the databases in the source cluster are to be migrated. This is handy if you are moving from local CouchDB to Cloudant, or from one Cloudant service to another.

$ couchreplicate -c 3 -a -s http://u:p@localhost:5984-t

It’s all about the open source๐Ÿ”—

The couchreplicate tool is free to use and is open-sourced under the Apache 2.0 license, so you can fork the code yourself and modify as you see fit. If you find a bug or have a contribution, we’d love to hear from you on the project’s Github page.

Further reading๐Ÿ”—

If you like using command-line tools, then there more CouchDB- and Cloudant-compatible tools from the same stable:

  • couchimport - import your CSV data into CouchDB/Cloudant and export your JSON documents to CSV files
  • couchbackup - backup a single database to a file
  • couchmigrate - migrate design documents into production without loss of service
  • couchdiff - find the difference between two databases
  • couchshell - interact with your database cluster as if it were a file system

Developers can use Node.js and Cloudant using the following libraries:

Apacheยฎ, Apache CouchDBโ„ข, CouchDBโ„ข, and the red couch logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.

Syndicated from: