Hello,
We have a 6 node cluster which was recently upgraded from 2.2.5 to 3.0.10.
Since then, we have been facing a lot of these exceptions. What could be the
possible cause for this exception?
INFO [SharedPool-Worker-2] 2017-02-23 20:14:04,984 Message.java:611 -
Unexpected exception during request; channel = [id: 0x034e0914,
/xx.xx.117.34:19836 :> /xx.xx.9.119:9042]
java.io.IOException: Error while read(...): Connection reset by peer
at io.netty.channel.epoll.Native.readAddress(Native Method)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.doReadBytes(EpollSocketChannel.java:675)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:714)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,330 StatusLogger.java:52 - Pool
Name Active Pending Completed Blocked All Time
Blocked
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,340 StatusLogger.java:56 -
MutationStage 0 0 88103 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,340 StatusLogger.java:56 -
ViewMutationStage 0 0 0 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,340 StatusLogger.java:56 -
ReadStage 0 0 155903 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,340 StatusLogger.java:56 -
RequestResponseStage 0 0 188450 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,341 StatusLogger.java:56 -
ReadRepairStage 0 0 4118 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,341 StatusLogger.java:56 -
CounterMutationStage 0 0 0 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,341 StatusLogger.java:56 -
MiscStage 0 0 0 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,341 StatusLogger.java:56 -
CompactionExecutor 0 0 20443 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,342 StatusLogger.java:56 -
MemtableReclaimMemory 0 0 52 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,342 StatusLogger.java:56 -
PendingRangeCalculator 0 0 8 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,342 StatusLogger.java:56 -
GossipStage 0 0 21308 0
0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,345 StatusLogger.java:56 -
Native-Transport-Requests 0 0 155489 0
4
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,346 StatusLogger.java:103 - Table
Memtable ops,data
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,346 StatusLogger.java:106 - look
300,21900
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,346 StatusLogger.java:106 -
segment 8,336
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,347 StatusLogger.java:106 -
sys_spend 13,5678
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,347 StatusLogger.java:106 - list
0,0
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,347 StatusLogger.java:106 -
a_count 14,392
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,347 StatusLogger.java:106 -
r_count 618,79787
INFO [ScheduledTasks:1] 2017-02-23 20:13:12,347 StatusLogger.java:106 -
sys_prd 642,1393965
Thank you in advance,
Meghana Mara
