[ https://issues.apache.org/jira/browse/IGNITE-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492243#comment-14492243 ]
Artem Shutak commented on IGNITE-172: ------------------------------------- TC build: http://94.72.60.102/viewLog.html?buildId=420545&buildTypeId=Ignite_Spi&tab=buildResultsDiv Stacktrace: {code} org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote node: 4a578303-fb73-402d-bd0e-e42a174be80b at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1595) at org.apache.ignite.spi.communication.tcp.GridTcpCommunicationSpiRecoveryAckSelfTest.checkOverflow(GridTcpCommunicationSpiRecoveryAckSelfTest.java:258) at org.apache.ignite.spi.communication.tcp.GridTcpCommunicationSpiRecoveryAckSelfTest.testQueueOverflow(GridTcpCommunicationSpiRecoveryAckSelfTest.java:210) Caused by: org.apache.ignite.IgniteCheckedException: Failed to connect to node (is node still alive?). Make sure that each GridComputeTask and GridCacheTransaction has a timeout set in order to prevent parties from waiting forever in case of network issues [nodeId=4a578303-fb73-402d-bd0e-e42a174be80b, addrs=[gg-teamcity-5/192.168.2.15:45081, /192.168.2.15:45081]] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:1863) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:1692) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:1633) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$4000(TcpCommunicationSpi.java:141) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$RecoveryWorker.body(TcpCommunicationSpi.java:2450) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) ------- Stdout: ------- [08:20:57,736][INFO ][main][root] >>> Starting test: testQueueOverflow <<< [08:20:57,742][INFO ][test-runner][TcpCommunicationSpi] Successfully bound to TCP port [port=45080, locHost=/192.168.2.15] [08:20:57,751][INFO ][test-runner][TcpCommunicationSpi] Successfully bound to TCP port [port=45081, locHost=/192.168.2.15] [08:20:57,761][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=1, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=2, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=3, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=4, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=5, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=6, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=7, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=8, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=9, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=10, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=11, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=12, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=13, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=14, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=15, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=16, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=17, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=18, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=19, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=20, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=21, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=22, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=23, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=24, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=25, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=26, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=27, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=28, resId=0] [08:20:57,762][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=29, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=30, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=31, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=32, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=33, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=34, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=35, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=36, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=37, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=38, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=39, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=40, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=41, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=42, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=43, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=44, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=45, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=46, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=47, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=48, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=49, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=50, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=51, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=52, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=53, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=54, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=55, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=56, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=57, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=58, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=59, resId=0] [08:20:57,763][INFO ][grid-nio-worker-0-#9013%null%][root] Test listener received message: GridTestMessage [srcNodeId=aa6167bd-8c9b-4ad3-b39c-bf375d3c2200, msgId=60, resId=0] [08:21:21,822][INFO ][main][root] >>> Stopping test: testQueueOverflow in 24086 ms <<< [08:21:21,822][INFO ][main][root] >>> Stopping test class: GridTcpCommunicationSpiRecoveryAckSelfTest <<< ------- Stderr: ------- [08:20:57,761][WARN ][grid-nio-worker-0-#9008%null%][TcpCommunicationSpi] Unacknowledged messages queue size overflow, will attempt to reconnect [remoteAddr=gg-teamcity-5/192.168.2.15:45081, queueLimit=50] [08:21:09,785][WARN ][grid-nio-worker-0-#9013%null%][TcpCommunicationSpi] Communication SPI Session write timed out (consider increasing 'socketWriteTimeout' configuration property) [remoteAddr=/192.168.2.15:33900, writeTimeout=5000] [08:21:21,815][WARN ][grid-nio-worker-0-#9013%null%][TcpCommunicationSpi] Communication SPI Session write timed out (consider increasing 'socketWriteTimeout' configuration property) [remoteAddr=/192.168.2.15:33904, writeTimeout=5000] [08:21:21,817][WARN ][grid-nio-worker-1-#9014%null%][TcpCommunicationSpi] Closing NIO session because of unhandled exception [cls=class o.a.i.i.util.nio.GridNioException, msg=Thread has been interrupted.] [08:21:21,817][ERROR][main][root] Test failed. class org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote node: 4a578303-fb73-402d-bd0e-e42a174be80b at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1595) at org.apache.ignite.spi.communication.tcp.GridTcpCommunicationSpiRecoveryAckSelfTest.checkOverflow(GridTcpCommunicationSpiRecoveryAckSelfTest.java:258) at org.apache.ignite.spi.communication.tcp.GridTcpCommunicationSpiRecoveryAckSelfTest.testQueueOverflow(GridTcpCommunicationSpiRecoveryAckSelfTest.java:210) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1346) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:67) at org.apache.ignite.testframework.junits.GridAbstractTest$2.run(GridAbstractTest.java:1289) Caused by: class org.apache.ignite.IgniteCheckedException: Failed to connect to node (is node still alive?). Make sure that each GridComputeTask and GridCacheTransaction has a timeout set in order to prevent parties from waiting forever in case of network issues [nodeId=4a578303-fb73-402d-bd0e-e42a174be80b, addrs=[gg-teamcity-5/192.168.2.15:45081, /192.168.2.15:45081]] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:1863) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:1692) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:1633) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$4000(TcpCommunicationSpi.java:141) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$RecoveryWorker.body(TcpCommunicationSpi.java:2450) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address: gg-teamcity-5/192.168.2.15:45081 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:1868) ... 5 more Caused by: class org.apache.ignite.IgniteCheckedException: Failed to read remote node ID (connection closed). at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:1942) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:1772) ... 5 more Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address: /192.168.2.15:45081 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:1868) ... 5 more Caused by: class org.apache.ignite.IgniteCheckedException: Failed to read remote node ID (connection closed). at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:1942) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:1772) ... 5 more {code} > GridTcpCommunicationSpiRecoveryAckSelfTest > ------------------------------------------ > > Key: IGNITE-172 > URL: https://issues.apache.org/jira/browse/IGNITE-172 > Project: Ignite > Issue Type: Bug > Components: general > Reporter: Irina Vasilinets > > GridTcpCommunicationSpiRecoveryAckSelfTest.testQueueOverflow and > GridTcpCommunicationSpiTcpNoDelayOffSelfTest.testSendToManyNodes > fail sometimes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)