[
https://issues.apache.org/jira/browse/IGNITE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342570#comment-17342570
]
Alexey Scherbakov commented on IGNITE-14583:
--------------------------------------------
[~ibelyakov]
Overall the changes look good, but I've noticed the test [1] has become flaky
due to triggered FH with following stack trace [2]
Can you fix it ?
[1]
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryOperationP2PTest#testTxReplicatedClient
[2]
{noformat}
[2021-05-11
16:31:44,988][ERROR][sys-#283%continuous.CacheContinuousQueryOperationP2PTest4%][IgniteTestResources]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]],
failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
o.a.i.IgniteCheckedException: Failed to send custom message: client is
starting.]][2021-05-11
16:31:44,988][ERROR][sys-#283%continuous.CacheContinuousQueryOperationP2PTest4%][IgniteTestResources]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]],
failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
o.a.i.IgniteCheckedException: Failed to send custom message: client is
starting.]]class org.apache.ignite.IgniteCheckedException: Failed to send
custom message: client is starting. at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.sendCustomEvent(GridDiscoveryManager.java:2251)
at
org.apache.ignite.internal.processors.metastorage.persistence.DistributedMetaStorageImpl.startWrite(DistributedMetaStorageImpl.java:1071)
at
org.apache.ignite.internal.processors.metastorage.persistence.DistributedMetaStorageImpl.writeAsync(DistributedMetaStorageImpl.java:482)
at
org.apache.ignite.internal.processors.cluster.ClusterProcessor.lambda$onReadyForWrite$3(ClusterProcessor.java:289)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7181)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:829)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)Caused by: class
org.apache.ignite.IgniteException: Failed to send custom message: client is
starting. at
org.apache.ignite.spi.discovery.tcp.ClientImpl.sendCustomEvent(ClientImpl.java:514)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.sendCustomEvent(TcpDiscoverySpi.java:546)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.sendCustomEvent(GridDiscoveryManager.java:2243)
... 9 more {noformat}
> Server node fails on remote filter deployment after client reconnect with new
> node id
> -------------------------------------------------------------------------------------
>
> Key: IGNITE-14583
> URL: https://issues.apache.org/jira/browse/IGNITE-14583
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.10
> Reporter: Igor Belyakov
> Assignee: Igor Belyakov
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Cluster contains 1 server and 1 client node. PeerClassLoading turned on.
> *1.* Client node deploys CQ with remote filter, server node doesn't have
> classes required for remote filter in classpath.
> *2.* Remote filter successfully deployed to server node.
> *3.* Client node disconnects and reconnects by using new node id:
> [15:44:43] Client node was reconnected after it was already considered failed
> by the server topology (this could happen after all servers restarted or due
> to a long network outage between the client and servers). All continuous
> queries and remote event listeners created by this client will be
> unsubscribed, consider listening to EVT_CLIENT_NODE_RECONNECTED event to
> restore them.
> *4.* On connect, client node tries to deploy remote filter to server node,
> but server node fails due to wrong ClassLoaderId, which is sent in
> GridDeploymentInfo (used ClassLoaderId for old client node):
> {code:java}
> [15:44:42] (err) Failed to notify listener:
> o.a.i.i.util.future.GridFutureChainListener@778d0fc6class
> org.apache.ignite.IgniteException: Failed to initialize a remote filter.
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.lambda$register$2bf956f5$1(CacheContinuousQueryHandler.java:353)
> at
> org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:77)
> at
> org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:69)
> at
> org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:29)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:477)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1311)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: class
> org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain
> deployment for class: Client1$$Lambda$513/0x00000008003b8840
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
> ... 8 more
> авг. 07, 2020 3:44:42 PM org.apache.ignite.logger.java.JavaLogger error
> SEVERE: Failed to unmarshal continuous routine handler
> [routineId=8f0137dc-cb32-40ff-83f4-867f272d338e,
> srcNodeId=ebcf4d5b-a476-43d5-b057-00b5694104e6]
> class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to
> obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
