When running throughput tests, I've usually found that the bottleneck is in the loading/testing script itself. That number (400-600 writes/sec) is usually the limit that a simple load script (consisting of a while loop that does PUTs to Riak) reaches.
Meaning, a single while loop issuing writes and waiting back for responses can't realistically write much faster than that. The solution is - you have to bulk up your testing script, and introduce parallelism. Fire up several loading scripts in parallel. If you have multiple machines to write to the riak cluster, use those. Make sure you're load-balancing your writes to all 3 nodes in the cluster, and not just hitting one. Use threads or worker pools to issue your writes, if that's available in your language. General performance test bottleneck checklist (to get you past that 600 ops/sec): 1. Are you load balancing which nodes you're writing to, or always hitting the same node/IP? Either use an external balancer like HA proxy, and write to that, or use the round-robin balancer that's built into some of the Riak clients (like the Java client). 2. Is your test script single threaded? (Are you doing your writes linearly, in a while loop?) Spread the work to several different load scripts working in parallel. Spin up different threads to issue writes, if your language does multithreading well. If you're in Ruby, consider using worker pool managers like Celluloid, or something like Sidekiq. There's other fine-tuning performance knobs having to do with backend settings, but those two above will get you the biggest gains. On Wed, Feb 20, 2013 at 3:10 PM, catchme <[email protected]> wrote: > I have configured Riak using > 3 nodes > n_val=3 > dw=1 > w=1 > r=1 > Using memory backend > > The throughput is very low. Results after running the script for 10 mins > #samples 90% Line Throughput > 408679 17 683/sec > > I am doing reads and writes to all the nodes. > > How do I improve the throughput? > > Thanks. > > > > -- > View this message in context: > http://riak-users.197444.n3.nabble.com/Riak-low-throughput-tp4026917.html > Sent from the Riak Users mailing list archive at Nabble.com. > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
