[jira] [Comment Edited] (IGNITE-5965) Ignite Basic: Flaky failure of GridServiceProcessorMultiNodeConfigSelfTest.testDeployOnEachNodeUpdateTopology
[
https://issues.apache.org/jira/browse/IGNITE-5965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473107#comment-16473107
]
Andrew Medvedev edited comment on IGNITE-5965 at 5/12/18 2:39 PM:
--
Failing tests are related to messaging layer. All failing tests have "connect
timed out" and "failed to send event notification" messages, so we get larger
set of ServiceDescriptors for nodes, than there are actual nodes and topology
does not get updated for all "Node left topology" events
{quote}[2018-05-12
16:55:54,072][ERROR][utility-#20468%service.GridServiceProcessorMultiNodeConfigSelfTest2%|#20468%service.GridServiceProcessorMultiNodeConfigSelfTest2%][query]
Failed to send event notification to node: cd8be305-ed5c-4f38-baad-64817296
class org.apache.ignite.IgniteCheckedException: Failed to send message (node
may have left the grid or TCP connection cannot be established due to firewall
issues) [node=TcpDiscoveryNode [id=cd8be305-ed5c-4f38-baad-64817296,
addrs=[127.0.0.1, 192.168.3.235, 192.168.43.79], sockAddrs=[/127.0.0.1:0,
/192.168.3.235:0, /192.168.43.79:0], discPort=0, order=31, intOrder=19,
lastExchangeTime=1526133351915, loc=false, ver=2.4.0#19700101-sha1:,
isClient=true], topic=T4 [topic=TOPIC_CACHE,
id1=5dc785b4-dd3d-3c3b-b270-c5fe2d7ed9a2,
id2=cd8be305-ed5c-4f38-baad-64817296, id3=1], msg=GridContinuousMessage
[type=MSG_EVT_NOTIFICATION, routineId=a1034d59-1476-4e44-8811-d498b65f450c,
data=null, futId=null], policy=2]
at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1650)
{quote}
was (Author: andmed):
Failing tests are related to messaging layer. All failing tests have connect
timed out and failed to send event notification messages, so we get larger set
of ServiceDescriptors for nodes, than there are actual nodes.
{quote}[2018-05-12
16:55:54,072][ERROR][utility-#20468%service.GridServiceProcessorMultiNodeConfigSelfTest2%][query]
Failed to send event notification to node: cd8be305-ed5c-4f38-baad-64817296
class org.apache.ignite.IgniteCheckedException: Failed to send message (node
may have left the grid or TCP connection cannot be established due to firewall
issues) [node=TcpDiscoveryNode [id=cd8be305-ed5c-4f38-baad-64817296,
addrs=[127.0.0.1, 192.168.3.235, 192.168.43.79], sockAddrs=[/127.0.0.1:0,
/192.168.3.235:0, /192.168.43.79:0], discPort=0, order=31, intOrder=19,
lastExchangeTime=1526133351915, loc=false, ver=2.4.0#19700101-sha1:,
isClient=true], topic=T4 [topic=TOPIC_CACHE,
id1=5dc785b4-dd3d-3c3b-b270-c5fe2d7ed9a2,
id2=cd8be305-ed5c-4f38-baad-64817296, id3=1], msg=GridContinuousMessage
[type=MSG_EVT_NOTIFICATION, routineId=a1034d59-1476-4e44-8811-d498b65f450c,
data=null, futId=null], policy=2]
at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1650)
{quote}
> Ignite Basic: Flaky failure of
> GridServiceProcessorMultiNodeConfigSelfTest.testDeployOnEachNodeUpdateTopology
> --
>
> Key: IGNITE-5965
> URL: https://issues.apache.org/jira/browse/IGNITE-5965
> Project: Ignite
> Issue Type: Test
>Affects Versions: 2.1
>Reporter: Ivan Rakov
>Priority: Major
> Labels: MakeTeamcityGreenAgain
> Attachments: log.txt
>
>
> Test has 85% success rate in master:
> http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=-2642357454043293898=testDetails_Ignite20Tests=%3Cdefault%3E
> Flaky failure is reproduced locally with similar success rate (24/30, Win 10).
> {noformat}
> junit.framework.AssertionFailedError:
> Expected :4
> Actual :5
>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessorAbstractSelfTest.checkCount(GridServiceProcessorAbstractSelfTest.java:765)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest.checkDeployOnEachNodeUpdateTopology(GridServiceProcessorMultiNodeConfigSelfTest.java:287)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest.testDeployOnEachNodeUpdateTopology(GridServiceProcessorMultiNodeConfigSelfTest.java:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
[jira] [Comment Edited] (IGNITE-5965) Ignite Basic: Flaky failure of GridServiceProcessorMultiNodeConfigSelfTest.testDeployOnEachNodeUpdateTopology
[
https://issues.apache.org/jira/browse/IGNITE-5965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464568#comment-16464568
]
Andrew Medvedev edited comment on IGNITE-5965 at 5/5/18 2:08 AM:
-
Is this platform specific?
I get 10/10 success on Linux, current master:
{quote}for run in \{1..10}; do mvn -DfailIfNoTests=false
-Dtest=GridServiceProcessorMultiNodeConfigSelfTest#testDeployOnEachNodeUpdateTopology
test; done > /tmp/out 2> /dev/null && grep "Tests run: 1" /tmp/out|grep
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.849 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.649 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.028 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.828 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.657 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.861 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.796 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.97 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.903 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.106 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest
{quote}
was (Author: andmed):
Is this platform specific?
I get 10/10 success on Linux, current master:
??for run in \{1..10}; do mvn -DfailIfNoTests=false
-Dtest=GridServiceProcessorMultiNodeConfigSelfTest#testDeployOnEachNodeUpdateTopology
test; done > /tmp/out 2> /dev/null && grep "Tests run: 1" /tmp/out|grep
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.849
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.649
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.028
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.828
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.657
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.861
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.796
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.97 s
- in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.903
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
??[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.106
s - in
org.apache.ignite.internal.processors.service.GridServiceProcessorMultiNodeConfigSelfTest??
> Ignite Basic: Flaky failure of
> GridServiceProcessorMultiNodeConfigSelfTest.testDeployOnEachNodeUpdateTopology
> --
>
> Key: IGNITE-5965
> URL:
