It's not just for testing. We found that loading/updating keys was somewhat slow, even with a low replication count. So we're exploring routes of taking a node down in an existing cluster, loading a large batch of keys, and then bringing it up. It's ok if it takes the data a while to rebalance after we rejoin the node.
It sounds like this isn't as explored as I first thought, so we'll do our own analysis. -Nate On Oct 10, 2011, at 9:18 AM, Ryan Zezeski wrote: > Nate, > > You could do this but if you're just doing it for testing then it seems like > more work than necessary. If you insist on doing it this way you'll want to > write it in Erlang so you can make use of the preflist calculation, > riak_object, and eleveldb driver. > > You could do the following: > > 1. stand up a fresh cluster > > 2. use one of the clients to load the data > > 3. archive the data dir on all nodes > > Next time you want to stand up a cluster with the same data simply unarchive > the data dirs before starting the cluster. > > -Ryan > > > On Fri, Oct 7, 2011 at 9:29 PM, Nate Lawson <[email protected]> wrote: > We're working on a test where we want to add batches of keys to Riak. We're > using the LevelDB backend with 1.0. One suggestion we heard was to take down > a node, batch insert directly into the backing store, and then bring the node > up again. > > Could someone give us some more details on this? How would Riak know which > keys in LevelDB had not yet been redistributed to the rest of the cluster? > > Thanks, > Nate > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
