[
https://issues.apache.org/jira/browse/IGNITE-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16666210#comment-16666210
]
Pavel Pereslegin edited comment on IGNITE-9858 at 10/29/18 10:24 AM:
---------------------------------------------------------------------
[~agoncharuk], check for client node is performed in
{{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}} (I updated ticket
description).
We have one instance of IpFinder and following execution order:
1. Server injects ignite instance resource into IpFinder.
{noformat}
at
org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:283)
at
org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:252)
at
org.apache.ignite.internal.processors.resource.GridResourceProcessor.injectGeneric(GridResourceProcessor.java:233)
at
org.apache.ignite.internal.managers.GridManagerAdapter.inject(GridManagerAdapter.java:173)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:257)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
2. Client injects ignite instance resource into IpFinder (same stacktrace).
3. Client initializes SPI, checks for client discovery:
{noformat}
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:298)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
4. Server initializes SPI:
{noformat}
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:374)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
As a result both nodes assume that the local node is a client and don't
register local address.
Usually in tests we start client node after starting the server, so this
problem is not reproduced.
So, I see the following options:
1. Just replace ipfinder with non-shared version in this test (if this is not
production case I think it's better solution).
2. Remove check for client discovery in
{{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}}, but we still have
{{MulticastIpFinder}} that relies on this check.
3. Change API {{TcpDiscoveryIpFinder#initializeLocalAddresses}} add explicit
"client discovery" flag.
4. Add some static synchronization for {{startSpi}} in {{GridManagerAdapter}}
(may affect multithread startup time).
Other thoughts?
was (Author: xtern):
[~agoncharuk], check for client node is performed in
{{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}} (I updated ticket
description).
We have one instance of IpFinder and following execution order:
1. Server injects ignite instance resource into IpFinder.
{noformat}
at
org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:283)
at
org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:252)
at
org.apache.ignite.internal.processors.resource.GridResourceProcessor.injectGeneric(GridResourceProcessor.java:233)
at
org.apache.ignite.internal.managers.GridManagerAdapter.inject(GridManagerAdapter.java:173)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:257)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
2. Client injects ignite instance resource into IpFinder (same stacktrace).
3. Client initializes SPI, checks for client discovery:
{noformat}
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:298)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
4. Server initializes SPI:
{noformat}
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:374)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
As a result both nodes assume that the local node is a client and don't
register local address.
Usually in tests we start client node after starting the server, so this
problem is not reproduced.
> [Test Failed] SystemCacheNotConfiguredTest#test flaky fails on TC (timeout).
> ----------------------------------------------------------------------------
>
> Key: IGNITE-9858
> URL: https://issues.apache.org/jira/browse/IGNITE-9858
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.6
> Reporter: Pavel Pereslegin
> Assignee: Pavel Pereslegin
> Priority: Major
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.8
>
>
> SystemCacheNotConfiguredTest hangs sometimes on TeamCity (timeout).
> Example of such failures on master branch:
> [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&buildTypeId=&tab=testDetails&testNameId=-2762467041583095183&order=TEST_STATUS_DESC&itemsCount=50&branch_IgniteTests24Java8=%3Cdefault%3E]
> When we using ip finder in shared mode each node should register self address
> (except clients, obviously).
> Check that the node is a client uses installed (via DI
> @IgniteInstanceResource) Ignite instance (see
> {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}}).
> So when client and server starts simultaneously, the following scenario is
> possible - Ignite server injected at first, then the Ignite client injected
> when the SPI is initialized ({{spiStart}}) both nodes assume that the local
> node is a client and don't register local address.
> {noformat}
> [2018-10-11 18:03:49,794][WARN
> ][tcp-client-disco-msg-worker-#57%client%][TcpDiscoverySpi] IP finder
> returned empty addresses list. Please check IP finder configuration. Will
> retry every 2000 ms. Change 'reconnectDelay' to configure the frequency of
> retries.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
