Re: Exceptions whenever compaction happens
Hi Ben, Thanks for your help. We have created an issue here: https://issues.apache.org/jira/browse/CASSANDRA-12706 Let's see if we can get some comments there. Regards, Nikhil Sharma On Mon, Sep 26, 2016 at 1:12 PM, Ben Slaterwrote: > Hi Nikhil, > > If you haven’t already done so I would recommend logging Cassandra project > JIRA for this issue - it looks like a defect to me and occurring during > compaction will probably be hard to work around. If you can work out what > table is being compacted when the issue occurs and include the schema of > the table that might help. > > Beyond that the only thing I can think of is running scrub if you haven’t > already done so. > > Cheers > Ben > > On Mon, 26 Sep 2016 at 16:36 Nikhil Sharma wrote: > >> Hi, >> >> We are not exactly sure what is causing this problem. But after >> compaction happens (after 1 week ttl). We start getting this exception: >> >> WARN [SharedPool-Worker-1] 2016-09-26 04:07:19,849 >> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on >> thread Thread[SharedPool-Worker-1,5,main]: {} >> java.lang.NullPointerException: null >> at org.apache.cassandra.db.Slices$ArrayBackedSlices$ >> ComponentOfSlice.isEQ(Slices.java:748) ~[apache-cassandra-3.0.9.jar: >> 3.0.9] >> at >> org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter. >> toCQLString(ClusteringIndexSliceFilter.java:150) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.SinglePartitionReadCommand. >> appendCQLWhereClause(SinglePartitionReadCommand.java:911) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.transform.BasePartitions. >> runOnClose(BasePartitions.java:70) ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:76) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb( >> ReadCommandVerbHandler.java:48) ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> ~[na:1.8.0_102] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$FutureTask.run(AbstractLocalAwareExecutorService.java:164) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] >> WARN [SharedPool-Worker-2] 2016-09-26 04:07:20,639 >> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on >> thread Thread[SharedPool-Worker-2,5,main]: {} >> java.lang.RuntimeException: java.lang.NullPointerException >> at org.apache.cassandra.service.StorageProxy$ >> DroppableRunnable.run(StorageProxy.java:2470) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> ~[na:1.8.0_102] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$FutureTask.run(AbstractLocalAwareExecutorService.java:164) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] >> Caused by: java.lang.NullPointerException: null >> at org.apache.cassandra.db.Slices$ArrayBackedSlices$ >> ComponentOfSlice.isEQ(Slices.java:748) ~[apache-cassandra-3.0.9.jar: >> 3.0.9] >> at >> org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter. >> toCQLString(ClusteringIndexSliceFilter.java:150) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.SinglePartitionReadCommand. >> appendCQLWhereClause(SinglePartitionReadCommand.java:911) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >>
Re: Exceptions whenever compaction happens
Hi Nikhil, If you haven’t already done so I would recommend logging Cassandra project JIRA for this issue - it looks like a defect to me and occurring during compaction will probably be hard to work around. If you can work out what table is being compacted when the issue occurs and include the schema of the table that might help. Beyond that the only thing I can think of is running scrub if you haven’t already done so. Cheers Ben On Mon, 26 Sep 2016 at 16:36 Nikhil Sharmawrote: > Hi, > > We are not exactly sure what is causing this problem. But after compaction > happens (after 1 week ttl). We start getting this exception: > > WARN [SharedPool-Worker-1] 2016-09-26 04:07:19,849 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-1,5,main]: {} > java.lang.NullPointerException: null > at > org.apache.cassandra.db.Slices$ArrayBackedSlices$ComponentOfSlice.isEQ(Slices.java:748) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.filter.ClusteringIndexSliceFilter.toCQLString(ClusteringIndexSliceFilter.java:150) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.SinglePartitionReadCommand.appendCQLWhereClause(SinglePartitionReadCommand.java:911) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.transform.BasePartitions.runOnClose(BasePartitions.java:70) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:76) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:48) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_102] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) > [apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) > [apache-cassandra-3.0.9.jar:3.0.9] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] > WARN [SharedPool-Worker-2] 2016-09-26 04:07:20,639 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-2,5,main]: {} > java.lang.RuntimeException: java.lang.NullPointerException > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_102] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) > [apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) > [apache-cassandra-3.0.9.jar:3.0.9] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] > Caused by: java.lang.NullPointerException: null > at > org.apache.cassandra.db.Slices$ArrayBackedSlices$ComponentOfSlice.isEQ(Slices.java:748) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.filter.ClusteringIndexSliceFilter.toCQLString(ClusteringIndexSliceFilter.java:150) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.SinglePartitionReadCommand.appendCQLWhereClause(SinglePartitionReadCommand.java:911) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at > org.apache.cassandra.db.transform.BasePartitions.runOnClose(BasePartitions.java:70) > ~[apache-cassandra-3.0.9.jar:3.0.9] > at