Hi Ben, Thanks for your help. We have created an issue here: https://issues.apache.org/jira/browse/CASSANDRA-12706
Let's see if we can get some comments there. Regards, Nikhil Sharma On Mon, Sep 26, 2016 at 1:12 PM, Ben Slater <ben.sla...@instaclustr.com> wrote: > Hi Nikhil, > > If you haven’t already done so I would recommend logging Cassandra project > JIRA for this issue - it looks like a defect to me and occurring during > compaction will probably be hard to work around. If you can work out what > table is being compacted when the issue occurs and include the schema of > the table that might help. > > Beyond that the only thing I can think of is running scrub if you haven’t > already done so. > > Cheers > Ben > > On Mon, 26 Sep 2016 at 16:36 Nikhil Sharma <nikhils...@gmail.com> wrote: > >> Hi, >> >> We are not exactly sure what is causing this problem. But after >> compaction happens (after 1 week ttl). We start getting this exception: >> >> WARN [SharedPool-Worker-1] 2016-09-26 04:07:19,849 >> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on >> thread Thread[SharedPool-Worker-1,5,main]: {} >> java.lang.NullPointerException: null >> at org.apache.cassandra.db.Slices$ArrayBackedSlices$ >> ComponentOfSlice.isEQ(Slices.java:748) ~[apache-cassandra-3.0.9.jar: >> 3.0.9] >> at >> org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter. >> toCQLString(ClusteringIndexSliceFilter.java:150) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.SinglePartitionReadCommand. >> appendCQLWhereClause(SinglePartitionReadCommand.java:911) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.transform.BasePartitions. >> runOnClose(BasePartitions.java:70) ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:76) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb( >> ReadCommandVerbHandler.java:48) ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> ~[na:1.8.0_102] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$FutureTask.run(AbstractLocalAwareExecutorService.java:164) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] >> WARN [SharedPool-Worker-2] 2016-09-26 04:07:20,639 >> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on >> thread Thread[SharedPool-Worker-2,5,main]: {} >> java.lang.RuntimeException: java.lang.NullPointerException >> at org.apache.cassandra.service.StorageProxy$ >> DroppableRunnable.run(StorageProxy.java:2470) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> ~[na:1.8.0_102] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$FutureTask.run(AbstractLocalAwareExecutorService.java:164) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ >> ice$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) >> [apache-cassandra-3.0.9.jar:3.0.9] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] >> Caused by: java.lang.NullPointerException: null >> at org.apache.cassandra.db.Slices$ArrayBackedSlices$ >> ComponentOfSlice.isEQ(Slices.java:748) ~[apache-cassandra-3.0.9.jar: >> 3.0.9] >> at >> org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter. >> toCQLString(ClusteringIndexSliceFilter.java:150) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.SinglePartitionReadCommand. >> appendCQLWhereClause(SinglePartitionReadCommand.java:911) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.db.transform.BasePartitions. >> runOnClose(BasePartitions.java:70) ~[apache-cassandra-3.0.9.jar:3.0.9] >> at >> org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:76) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.service.StorageProxy$ >> LocalReadRunnable.runMayThrow(StorageProxy.java:1797) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> at org.apache.cassandra.service.StorageProxy$ >> DroppableRunnable.run(StorageProxy.java:2466) >> ~[apache-cassandra-3.0.9.jar:3.0.9] >> ... 5 common frames omitted >> >> >> We have no idea how to solve this. I am losing sleep over this, please >> help! >> >> Cassandra: 3.0.9 (also happened in 3.0.8) >> Java: Oracle jdk "1.8.0_102" >> jemalloc enabled >> >> >> Regards, >> >> Nikhil Sharma >> > -- > ———————— > Ben Slater > Chief Product Officer > Instaclustr: Cassandra + Spark - Managed | Consulting | Support > +61 437 929 798 >