Hi Gui, I recently pushed 70 million records of size 1K each into a 5-node Riak cluster (which was replicating to another 5-node cluster) at around 1000 writes/second using basho_bench and the REST interface. I probably could have pushed it further, but I wanted to confirm that it could maintain the load for the entire data set, which it did.
My point being that your speed-limit of 200 writes/second is likely specific to your configuration. I wonder: 1) what's your average write latency? 2) how big is your connection pool? Because it's possible that you don't have enough connections available to handle your desired load. -matt On Thu, Apr 7, 2011 at 6:01 PM, Gui Pinto <[email protected]> wrote: > Hey guys, > I'm attempting to importing 300M+ objects into a Riak cluster, but have > quickly reached the REST API's speed-limit at 200-store()'s per second.. > At the rate of 200/s, I'm looking at 20-days to import this data set! That > can't be the fastest method to do this.. > > Any recommendations? > > Thanks! > Gui Pinto > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
