http://wiki.apache.org/cassandra/FAQ#slows_down_after_lotso_inserts
Aaron On 11 Mar 2011, at 05:08, sridhar basam wrote: > > Sounds like GC from your description of fast->slow->fast. Collect GC times > from both the client and server side and plot against your application timing. > > If you uncomment the verbose GC entries in the cassandra-env.sh file you > should get timing for the server side, pass in the same arguments for your > client. Align time across the 3 files and plot to see if GC is the cause. > > Sridhar > > > > On Thu, Mar 10, 2011 at 9:30 AM, Roshan Dawrani <roshandawr...@gmail.com> > wrote: > Hi, > > I am in the middle of some load testing on a 1-node Cassandra setup. We are > not on very high loads yet. We have recorded the timings taken up by > mutator.execute() calls and we see this kind of variation during the test run: > > So, 25% of the times, execute() calls come back in 25 milli-seconds, but the > longer calls go upto 4 seconds. > > Can someone please provide some pointers on what and where to focus on in my > Hector / Cassandra setup? We are mostly on the default Cassandra > configuration at this time - only change is the max connection pool size > (CassandraHostConfigurator.maxActive) is changed to 300 from a default of 50. > > I would also like to add that the time increase is not linear - it starts > fast, goes, slow, very slow, and becomes faster again. > > ------------------------ > 25% 29 > 50% 105 > 66% 185 > 70% 208 > 75% 240 > 80% 297 > 90% 510 > 95% 854 > 98% 1075 > 99% 1215 > 100% 4442 > ------------------------ > > -- > Roshan > Blog: http://roshandawrani.wordpress.com/ > Twitter: @roshandawrani > Skype: roshandawrani > >
