[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov edited comment on CASSANDRA-6525 at 5/20/14 8:18 PM:
----------------------------------------------------------------------
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like
{code}
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
{code}
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
was (Author: vkuptcov):
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like "
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
"
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
> Cannot select data which using "WHERE"
> --------------------------------------
>
> Key: CASSANDRA-6525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux RHEL5
> RAM: 1GB
> Cassandra 2.0.3
> CQL spec 3.1.1
> Thrift protocol 19.38.0
> Reporter: Silence Chow
> Assignee: Tyler Hobbs
> Fix For: 2.0.8
>
> Attachments: 6525-2.0.txt, 6981_test.py
>
>
> I am developing a system on my single machine using VMware Player with 1GB
> Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
> I using "WHERE" and it has just below 10 records. I have got this error in
> system log:
> {noformat}
> ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
> Exception in thread Thread[ReadStage:41,5,main]
> java.io.IOError: java.io.EOFException
> at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
> at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
> at
> org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
> at
> org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:87)
> at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
> at
> org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:120)
> at
> org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
> at
> org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
> at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
> at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:332)
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1401)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1936)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.EOFException
> at java.io.RandomAccessFile.readFully(Unknown Source)
> at java.io.RandomAccessFile.readFully(Unknown Source)
> at
> org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:348)
> at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371)
> at
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:74)
> at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
> ... 27 more
> {noformat}
> E.g.
> {{SELECT * FROM table;}}
> Its fine.
> {{SELECT * FROM table WHERE field = 'N';}}
> field is the partition key.
> Its said "Request did not complete within rpc_timeout." in cqlsh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
