I ran into TCP Reset issues with both the python and nodejs clients. I ended up solving them by putting an HAProxy server on the client machine configured to route to all available riak servers. In addition, I also configured the HAProxy frontend with:
option http-server-close option http-pretend-keepalive Those two options increased the stability of connections to the riak nodes substantially, and I was able to go from 5 simultaneous http connections to 2000 simultaneous connections and around 12000 rps with 0 resets. In short: I think if the clients made efficient use of keepalive (riakjs explicitly disables it! and I didn't dig into the specifics on the python client), they would be able to achieve much higher concurrency rates with greater stability. - Alex Thompson iDigBio Project (www.idigbio.org) ACIS Laboratory (www.acis.ufl.edu) P.S. Our Riak backed metadata store is now live at portal.idigbio.org / api.idigbio.org ----- Original Message ----- From: "Jeremy Ong" <[email protected]> To: "Riak Users Mailing List" <[email protected]> Sent: Sunday, July 15, 2012 2:00:02 AM Subject: Setting up Riak for highly concurrent load testing Hi Basho and Riak Users, I am in the process of evaluating Riak for commercial usage here at Playmesh. Riak seems great and has a lot of features we like but I am running into some difficulties in my initial testing. I am attempting to hit the database at 100 rps on a single machine (aka one node). In doing so, I run into a number of problems (tcp sockets closing being the primary one). I am using the Erlang client and have tried talking to Riak through a single client process and also by spawning a new concurrent process for every request. I suspect that Riak is throttling the number of allowable concurrent connections. Also, I suspect that my understanding about how to properly utilize the Erlang client is underwhelming at best. Any suggestions on how to properly set up a local test bed for Riak and communicating to Riak? Thanks, Jeremy -- Jeremy Ong Playmesh, Inc. _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
