Looking at http://blog.mikiobraun.de/2010/08/cassandra-gc-tuning.html and server logs, I think my situation is this
"The default cassandra settings has the highest peak heap usage. The problem with this is that it raises the possibility that during the CMS cycle, a collection of the young generation runs out of memory to migrate objects to the old generation (a so-called concurrent mode failure), leading to stop-the-world full garbage collection. However, with a slightly lower setting of the CMS threshold, we get a bit more headroom, and more stable overall performance." I see concurrentMarkSweep system.log Entries trying to gc 2-4 collections. Any suggestions for preemptive measure for this would be welcome.