Cloudant

We're very excited to be joinin up with our friends at 10gen to sponsor the NoSQL Live conference right here in Boston, our backyard.  The official press release can be found here.

The event will take place on March 11th at the John Hancock hotel.  A number of us from Cloudant will be at the conference in both official (panel discussions, lightning talks) and unofficial (milling about, drinking) capacities.  If you are in the New England area or plan on coming to the New England area, you should register and attend the conference.  You can register here.

From the 10gen events site

The Cloudant team is growing! 

We're excited to welcome Dr. Dave Hardtke who has joined Cloudant as Director of Search.  Like many of us here at Cloudant, Dave began his career as an experimental particle physicist.  In over a decade of experience Dave illuminated the secrets of the universe at various experiments such as STAR (Brookhaven National Lab), ICECUBE (Antarctica), and NA44 (CERN).  Dave's seminal work lead to the discovery of the Quark Gluon Plasma, the 2005 physics story of the year.  More recently Dave has turned his talents in algorithms and analyses to search, where he has built his own startup stinkyteddy.com, a general purpose search engine combining multiple keyword-driven search feeds and intelligent semantic analysis to deliver the most timely search results regardless of source and content types.  Prior to that Dave served as Chief Scientist at Surf Canyon, a startup focusing on personalization of real-time search.

Dave is accelerating Cloudant Search.  As we add customers it has become clear that they are hungry not just for a flexible way to store thier data, but a powerful way to "discover" their data. Search is a critical component in nearly all data driven applications.  We are therefore expanding our efforts to integrate a robust search platform into our CouchDB-based, cloud database service.  Dave brings his expertise in internet and enterpise search technology to the project.

Please join me in welcoming Dave to the team.

After looking into the problems from last night, we believe we've found the cause. In the database cluster, we tried a new version of dynamic code loading in a distributed erlang environment.  We learned that this introduced a single point of failure in the database and we've since reverted the system to eliminate this problem.  We were running the test to improve performance for customers and with today's update we have resolved the issue.

Again, we're sorry to all those affected.  We appreciate your patience as we work to improve Cloudant's performance and service.

Hi to everyone out there in Cloudant land.  In the wee morning hours (eastern standard time) we had some technical difficulties with our database cluster.  We're really sorry to those of you who were affected.  We think we have found the issue and resolved it.  We want to do a full post-mortem before declaring vicotry, but operations should be back to normal now.

Again, we're very sorry to those people who saw problems.  Please let us know if you were affected.  We're working hard to prevent things like this from happening in the future.

Our new front page just went live on Cloudant.com. The main goal of this redesign is to give more information about the service we are currently rolling out. While we are still in private beta, we are starting to accept more people onto our platform and therefore we are ready to tell you more about what we are doing. We are currently aiming at existing CouchDB users, and to them we are offering professional hosting of CouchDB in the cloud. 

The new design follows the AIDA principle, according to which a succesful marketing flow involves four main phases: attention (A), interest (I), desire (D), and action (A).

Since launching the beta we’ve received a bunch of feedback along the lines of “Uh... I can’t change my password, wtf?” Yes, wtf indeed. We knew we would be building this feature eventually but we didn’t want to hold up the release for it. Well, now we have it on our new Account Options page.

You can access it by selecting ‘account options’ from the user account pulldown on the top right of the page. Thanks for all the feedback and keep it coming.

Since Monday we’ve been sending out invites to the private beta (if you haven’t gotten yours yet, be patient, it’s coming). And we’ve gotten a lot of great feedback on how to make the service better. One suggestion that came from a few users was to allow database access to non-Cloudant users through API keys that can be shared with anyone. Well, your requests did not fall on deaf ears. For any of your databases you can now navigate to the permissions tab and create a database-specific API key and password. You can then set permissions for that key as if it were a Cloudant user. The screenshot below shows what that should look like. Thanks to everyone for the great suggestion. You keep the feature requests coming and we’ll get to them as fast as we can.

Baracus is a new project built specifically for benchmarking CouchDB. Using trusty httperf, Baracus makes it easy to create and perform a battery of tests based on simple configuration files. In this post I'll go into the details of a Baracus run and show you how to run your own benchmarks.

First, let's install Baracus. It depends on httperf so install it first either from source (check the README for compile options for httperf) or from your distribution. Note that Baracus uses gemcutter for gem hosting.

Next, let's examine a simple Baracus config file.

This configuration file specifies where the httperf executable lives, the host, port, user, password, and database to use for the test. Baracus puts all of the results and wsesslog files back into CouchDB for later consumption, the report_url is where you define this database.

As I mentioned, Baracus uses httperf to perform benchmarks. Specifically it uses httperf's wsesslog feature. wsesslog is a simple file format that describes what httperf should do each session. Here's an example of an entry generated with Baracus that does an HTTP GET:

Baracus creates two wsesslogs for each run, one for writes (POST) and one for reads (GET). Baracus generates these files based on a few items in your configuration. Specifically, the number of sessions, document size and how many reads or writes to do per session. Additionally in the configuration you can specify at what rate httperf should perform the operations within each wsesslog file. Baracus creates the wsesslog for writing with random data and creates the read file with random docs from _all_docs. Since Baracus uses _all_docs, it performs all the writes first. The above config will create 100 sessions with 10 writes and reads each at a rate of 100 per second. Each document will be roughly 100 bytes.

The info section at the end of the config contains any additional information you may want to have tag along with your results. You can put anything you want in there; I generally use this to describe the system I just ran the tests on.

Once you have a configuration you are happy with it's very simple to run your tests.

This will print the standard httperf output as well as a tab delimited version. If you want to checkout the wsesslog files they will be written to the directory you ran Baracus in as well as to the results document in the reporting database. Here's what the results look like in Couch.

Baracus is meant to make repeatable benchmarking easy. If you have any questions, bugs or would like to contribute check out Baracus on github.

Cloudant’s own Brad Anderson is helping to organize No:SQL (east), a conference centered around the latest nonrelational database systems. The conference will be held at the Georgia Tech Research Institute in Atlanta on Oct 28-30. The description from the website reads:

NoSQL East, a conference of non-relational data stores, aims to present the experiences of different companies using so-called “nosql” solutions in production.

The reasons for using these new systems are varied, but often involve scale that relational solutions cannot achieve. We have a new tool in our toolbelts, and as you will see, some people are using them quite effectively.

Join us in Atlanta as we discuss “Big Data” and the systems that are completely transforming how people look at data.

The organizers have a great venue picked out (courtesy of Georgia Tech) and an exciting group of speakers. A number of nonrelational systems will be covered, and Cloudant co-founder Mike Miller will be giving a talk on CouchDB. Anyone interested in the NoSQL space should really try to make it down to Atlanta to attend.

Cloudant is one of two primary sponsors of the event, the other being GTRI. Many, if not all, of us will be attending the conference. We’re always happy to talk about NoSQL, CouchDB, JQuery, The Bears winning the Superbowl, etc. You should come say hi.

More information about No:SQL (east) can be found at https://nosqleast.com/2009/. Check it out.

Update (10/22/09):  Since posting the above, a couple other great companies, including Basho and The Rackspace Cloud, have also joined on as sponsors for the event.