[jira] [Commented] (CASSANDRA-12259) Weird select issue on counters
[ https://issues.apache.org/jira/browse/CASSANDRA-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505382#comment-15505382 ] Aleksey Yeschenko commented on CASSANDRA-12259: --- Pretty sure this is a duplicate of CASSANDRA-11726. If you can still see the issue there, feel free to reopen. Otherwise closing as a Duplicate. Thanks. > Weird select issue on counters > -- > > Key: CASSANDRA-12259 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12259 > Project: Cassandra > Issue Type: Bug > Environment: datastax-3.6.0, CentOS 7, x86_64 >Reporter: Gábor Auth > > Tried to select value from a counter table: > {code} > cassandra@cqlsh:keyspace> select id,science,treasury,tutorial from counters > where id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > id | science | treasury | tutorial > --+-+--+-- > 38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 | 100 |5 > (1 rows) > cassandra@cqlsh:keyspace> select * from counters where > id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > id | science | treasury | tutorial > --+-+--+-- > 38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 | 100 |5 > (1 rows) > cassandra@cqlsh:keyspace> select tutorial from counters where > id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > ReadTimeout: code=1200 [Coordinator node timed out waiting for replica nodes' > responses] message="Operation timed out - received only 0 responses." > info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'} > {code} > Meanwhile in the log of the node: > {code} > WARN [SharedPool-Worker-2] 2016-07-21 11:21:09,055 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-2,5,main]: {} > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_91] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) > [apache-cassandra-3.6.0.jar:3.6.0] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) > [apache-cassandra-3.6.0.jar:3.6.0] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: java.lang.IndexOutOfBoundsException: null > at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91] > at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) > ~[na:1.8.0_91] > at > org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:272) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:591) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > >
[jira] [Commented] (CASSANDRA-12259) Weird select issue on counters
[ https://issues.apache.org/jira/browse/CASSANDRA-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387870#comment-15387870 ] Gábor Auth commented on CASSANDRA-12259: I've tried this 'toolset' before submit this issue: {code} /home/cassandra/datastax-ddc-3.6.0/bin/nodetool upgradesstables /home/cassandra/datastax-ddc-3.6.0/bin/nodetool compact /home/cassandra/datastax-ddc-3.6.0/bin/nodetool scrub /home/cassandra/datastax-ddc-3.6.0/bin/nodetool cleanup /home/cassandra/datastax-ddc-3.6.0/bin/nodetool clearsnapshot /home/cassandra/datastax-ddc-3.6.0/bin/nodetool truncatehints /home/cassandra/datastax-ddc-3.6.0/bin/nodetool repair -full -pr {code} > Weird select issue on counters > -- > > Key: CASSANDRA-12259 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12259 > Project: Cassandra > Issue Type: Bug > Environment: datastax-3.6.0, CentOS 7, x86_64 >Reporter: Gábor Auth > > Tried to select value from a counter table: > {code} > cassandra@cqlsh:keyspace> select id,science,treasury,tutorial from counters > where id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > id | science | treasury | tutorial > --+-+--+-- > 38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 | 100 |5 > (1 rows) > cassandra@cqlsh:keyspace> select * from counters where > id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > id | science | treasury | tutorial > --+-+--+-- > 38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 | 100 |5 > (1 rows) > cassandra@cqlsh:keyspace> select tutorial from counters where > id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > ReadTimeout: code=1200 [Coordinator node timed out waiting for replica nodes' > responses] message="Operation timed out - received only 0 responses." > info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'} > {code} > Meanwhile in the log of the node: > {code} > WARN [SharedPool-Worker-2] 2016-07-21 11:21:09,055 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-2,5,main]: {} > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_91] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) > [apache-cassandra-3.6.0.jar:3.6.0] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) > [apache-cassandra-3.6.0.jar:3.6.0] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: java.lang.IndexOutOfBoundsException: null > at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91] > at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) > ~[na:1.8.0_91] > at > org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:272) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) > ~[apache-cassandra-3.6.0.jar:3.6.0] > >
[jira] [Commented] (CASSANDRA-12259) Weird select issue on counters
[ https://issues.apache.org/jira/browse/CASSANDRA-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387727#comment-15387727 ] Aleksey Yeschenko commented on CASSANDRA-12259: --- Assuming that this is indeed corruption, you should try running scrub first on the affected table. That may or may not help. Can you do that and report back, please? > Weird select issue on counters > -- > > Key: CASSANDRA-12259 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12259 > Project: Cassandra > Issue Type: Bug > Environment: datastax-3.6.0, CentOS 7, x86_64 >Reporter: Gábor Auth > > Tried to select value from a counter table: > {code} > cassandra@cqlsh:keyspace> select id,science,treasury,tutorial from counters > where id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > id | science | treasury | tutorial > --+-+--+-- > 38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 | 100 |5 > (1 rows) > cassandra@cqlsh:keyspace> select * from counters where > id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > id | science | treasury | tutorial > --+-+--+-- > 38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 | 100 |5 > (1 rows) > cassandra@cqlsh:keyspace> select tutorial from counters where > id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d'; > ReadTimeout: code=1200 [Coordinator node timed out waiting for replica nodes' > responses] message="Operation timed out - received only 0 responses." > info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'} > {code} > Meanwhile in the log of the node: > {code} > WARN [SharedPool-Worker-2] 2016-07-21 11:21:09,055 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-2,5,main]: {} > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_91] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) > [apache-cassandra-3.6.0.jar:3.6.0] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) > [apache-cassandra-3.6.0.jar:3.6.0] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: java.lang.IndexOutOfBoundsException: null > at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91] > at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) > ~[na:1.8.0_91] > at > org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758) > ~[apache-cassandra-3.6.0.jar:3.6.0] > at > org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:272) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) > ~[apache-cassandra-3.6.0.jar:3.6.0] > > > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:591) > ~[apache-cassandra-3.6.0.jar:3.6.0] > >