[
https://issues.apache.org/jira/browse/CASSANDRA-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037199#comment-15037199
]
wateray commented on CASSANDRA-10303:
-------------------------------------
The node which execute rebuild, when it read stream throws java.io.IOException:
Connection timed out.,
It will hang up, because of this exception does not been caughted. The
StreamSession didn't know the exception, it cann't end up.
The latest release version 2.1.11,2.2.3 3.0 don't fix this bug. The develop
trunk has fixed this bug.
CompressedInputStream.java
*with bug version*
{code:java}
while (bufferRead < readLength)
{
int r = source.read(compressedWithCRC, bufferRead,
readLength - bufferRead);
if (r < 0)
{
dataBuffer.put(POISON_PILL);
return; // throw exception where we consume dataBuffer
}
bufferRead += r;
}
{code}
*fixed version*
{code:java}
protected void runMayThrow() throws Exception
{
..............
while (bufferRead < readLength)
{
int r;
try
{
r = source.read(compressedWithCRC, bufferRead,
readLength - bufferRead);
if (r < 0)
{
dataBuffer.put(POISON_PILL);
return; // throw exception where we consume
dataBuffer
}
}
catch (IOException e)
{
dataBuffer.put(POISON_PILL);
throw e;
}
bufferRead += r;
}
dataBuffer.put(compressedWithCRC);
}
.......}
{code}
> streaming for 'nodetool rebuild' fails after adding a datacenter
> -----------------------------------------------------------------
>
> Key: CASSANDRA-10303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10303
> Project: Cassandra
> Issue Type: Bug
> Environment: jdk1.7
> cassandra 2.1.8
> Reporter: zhaoyan
> Fix For: 2.1.x
>
>
> we add another datacenter.
> use nodetool rebuild DC1
> stream from some node of old datacenter always hang up with these exception:
> {code}
> ERROR [Thread-1472] 2015-09-10 19:24:53,091 CassandraDaemon.java:223 -
> Exception in thread Thread[Thread-1472,5,RMI Runtime]
> java.lang.RuntimeException: java.io.IOException: Connection timed out
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_60]
> Caused by: java.io.IOException: Connection timed out
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_60]
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> ~[na:1.7.0_60]
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.7.0_60]
> at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.7.0_60]
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> ~[na:1.7.0_60]
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:59)
> ~[na:1.7.0_60]
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
> ~[na:1.7.0_60]
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> ~[na:1.7.0_60]
> at
> org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStream.java:172)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.8.jar:2.1.8]
> ... 1 common frames omitted
> {code}
> i must restart node to stop current rebuild, and rebuild agagin and again to
> success....
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
