you need to check 192.168.0.36/10.234.86.36 for streaming ERRORS 2016-08-05 12:08 GMT-03:00 Jean Carlo <jean.jeancar...@gmail.com>:
> Hi Paulo > > I found the lines, we got an exception "Outgoing stream handler has been > closed" these are > > ERROR [STREAM-IN-/10.234.86.36] 2016-08-04 16:55:53,772 > StreamSession.java:621 - [Stream #c4e79260-5a46-11e6-9993-e11d93fd5b40] > Remote peer 192.168.0.36 failed stream session. > INFO [STREAM-IN-/10.234.86.36] 2016-08-04 16:55:53,772 > StreamResultFuture.java:180 - [Stream #c4e79260-5a46-11e6-9993-e11d93fd5b40] > Session with /192.168.0.36 is complete > WARN [STREAM-IN-/10.234.86.36] 2016-08-04 16:55:53,773 > StreamResultFuture.java:207 - [Stream #c4e79260-5a46-11e6-9993-e11d93fd5b40] > Stream failed > ERROR [StreamReceiveTask:107995] 2016-08-04 16:55:53,782 > StreamReceiveTask.java:183 - Error applying streamed data: > java.lang.RuntimeException: Outgoing stream handler has been closed > at org.apache.cassandra.streaming.ConnectionHandler. > sendMessage(ConnectionHandler.java:138) ~[apache-cassandra-2.1.14.jar: > 2.1.14] > at org.apache.cassandra.streaming.StreamSession. > maybeCompleted(StreamSession.java:697) ~[apache-cassandra-2.1.14.jar: > 2.1.14] > at org.apache.cassandra.streaming.StreamSession. > taskCompleted(StreamSession.java:653) ~[apache-cassandra-2.1.14.jar: > 2.1.14] > at org.apache.cassandra.streaming.StreamReceiveTask$ > OnCompletionRunnable.run(StreamReceiveTask.java:179) > ~[apache-cassandra-2.1.14.jar:2.1.14] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_60] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_60] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_60] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_60] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] > > ERROR [StreamReceiveTask:107995] 2016-08-04 16:55:53,782 > StreamSession.java:505 - [Stream #c4e79260-5a46-11e6-9993-e11d93fd5b40] > Streaming error occurred > java.lang.RuntimeException: Outgoing stream handler has been closed > at org.apache.cassandra.streaming.ConnectionHandler. > sendMessage(ConnectionHandler.java:138) ~[apache-cassandra-2.1.14.jar: > 2.1.14] > at org.apache.cassandra.streaming.StreamSession. > maybeCompleted(StreamSession.java:697) ~[apache-cassandra-2.1.14.jar: > 2.1.14] > at org.apache.cassandra.streaming.StreamSession. > taskCompleted(StreamSession.java:653) ~[apache-cassandra-2.1.14.jar: > 2.1.14] > at org.apache.cassandra.streaming.StreamReceiveTask$ > OnCompletionRunnable.run(StreamReceiveTask.java:179) > ~[apache-cassandra-2.1.14.jar:2.1.14] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_60] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_60] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_60] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_60] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] > > I al looking through the changes files to see if it is a bug fixed in the > 2.1.15 > > > > Saludos > > Jean Carlo > > "The best way to predict the future is to invent it" Alan Kay > > On Fri, Aug 5, 2016 at 4:24 PM, Paulo Motta <pauloricard...@gmail.com> > wrote: > >> It seems you have a streaming error, look for ERROR statement in the >> streaming classes before that which may give you a more specific root >> cause. In any case, I'd suggest you to upgrade to 2.1.15 as there were a >> couple of streaming fixes on this version that might help. >> >> 2016-08-05 11:15 GMT-03:00 Jean Carlo <jean.jeancar...@gmail.com>: >> >>> >>> Hi guys, Doing a repair I got this error for 2 tokenranges. >>> >>> ERROR [Thread-2499244] 2016-08-04 20:05:24,288 StorageService.java:3068 >>> - Repair session 41e4bab0-5a63-11e6-9993-e11d93fd5b40 for range >>> (487410372471205090,492009442088088379] failed with error >>> org.apache.cassandra.exceptions.RepairExcep >>> tion: [repair #41e4bab0-5a63-11e6-9993-e11d93fd5b40 on ks/cf_adv, >>> (487410372471205090,492009442088088379]] Sync failed between / >>> 192.168.0.144 and /192.168.0.37 >>> java.util.concurrent.ExecutionException: java.lang.RuntimeException: >>> org.apache.cassandra.exceptions.RepairException: [repair >>> #41e4bab0-5a63-11e6-9993-e11d93fd5b40 on ks/cf_adv, (487410372471205090, >>> 492009442088088379]] Sync failed between /192.168.0.144 and / >>> 192.168.0.37 >>> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >>> [na:1.8.0_60] >>> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >>> [na:1.8.0_60] >>> at >>> org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:3059) >>> ~[apache-cassandra-2.1.14.jar:2.1.14] >>> at >>> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) >>> [apache-cassandra-2.1.14.jar:2.1.14] >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> [na:1.8.0_60] >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> [na:1.8.0_60] >>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] >>> Caused by: java.lang.RuntimeException: >>> org.apache.cassandra.exceptions.RepairException: >>> [repair #41e4bab0-5a63-11e6-9993-e11d93fd5b40 on ks/cf_adv, >>> (487410372471205090,492009442088088379]] Sync failed between / >>> 10.234.72.144 and >>> /10.234.86.37 >>> at com.google.common.base.Throwables.propagate(Throwables.java:160) >>> ~[guava-16.0.jar:na] >>> at >>> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) >>> [apache-cassandra-2.1.14.jar:2.1.14] >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> [na:1.8.0_60] >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> [na:1.8.0_60] >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> ~[na:1.8.0_60] >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> ~[na:1.8.0_60] >>> ... 1 common frames omitted >>> Caused by: org.apache.cassandra.exceptions.RepairException: [repair >>> #41e4bab0-5a63-11e6-9993-e11d93fd5b40 on ks/cf_adv, (487410372471205090, >>> 492009442088088379]] Sync failed between /192.168.0.144 and / >>> 192.168.0.37 >>> at >>> org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:223) >>> ~[apache-cassandra-2.1.14.jar:2.1.14] >>> at org.apache.cassandra.service.ActiveRepairService.handleMessa >>> ge(ActiveRepairService.java:422) ~[apache-cassandra-2.1.14.jar:2.1.14] >>> at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb( >>> RepairMessageVerbHandler.java:134) ~[apache-cassandra-2.1.14.jar:2.1.14] >>> at >>> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) >>> ~[apache-cassandra-2.1.14.jar:2.1.14] >>> ... 3 common frames omitted >>> >>> Is it a network problem or we are having a bug ? There is nothing else >>> in the log of cassandra more than >>> Sync failed between. >>> >>> We are having cassandra 2.1.14 >>> >>> Best regards >>> >>> Jean Carlo >>> >>> "The best way to predict the future is to invent it" Alan Kay >>> >> >> >
