Sanjeeth, Looks like the error is being populated from the hintedhandoff, what is the size of your hints cf?
Thanks Rahul On Wed, Dec 25, 2013 at 8:54 PM, Sanjeeth Kumar <sanje...@exotel.in> wrote: > Hi all, > One of my cassandra nodes crashes with the following exception > periodically - > ERROR [HintedHandoff:33] 2013-12-25 20:29:22,276 SliceQueryFilter.java > (line 200) Scanned over 100000 tombstones; query aborted (see > tombstone_fail_thr > eshold) > ERROR [HintedHandoff:33] 2013-12-25 20:29:22,278 CassandraDaemon.java > (line 187) Exception in thread Thread[HintedHandoff:33,1,main] > org.apache.cassandra.db.filter.TombstoneOverwhelmingException > at > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122) > at > org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80) > at > org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306) > at > org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351) > at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309) > at > org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:92) > at > org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > Why does this happen? Does this relate to any incorrect config value? > > The Cassandra Version I'm running is > ReleaseVersion: 2.0.3 > > - Sanjeeth > >
