[
https://issues.apache.org/jira/browse/IGNITE-11235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev updated IGNITE-11235:
-------------------------------------
Description:
*In 2.7*, when you create a Continuous Query on client with class which is not
present in server classpath, server node will be terminated immediately.
{code}
[2019-02-05 18:20:22,508][ERROR][tcp-disco-msg-worker-#2][root] Critical system
error detected. Will be handled accordingly to configured handler
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]],
failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class
o.a.i.IgniteException: Failed to unmarshal discovery custom message:
TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage
[sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0,
id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0,
verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0,
failedNodes=null, isClient=false]]]]
class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom
message: TcpDiscoveryCustomEventMessage [msg=null,
super=TcpDiscoveryAbstractMessage
[sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0,
id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0,
verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0,
failedNodes=null, isClient=false]]
{code}
This means that one rogue client (e.g. with mismatched code base version) can
knock whole cluster offline.
I think we should disconnect client from topology instead when it sends
incorrect discovery message.
*In 2.6*, if you tried to create a Continuous Query on client with class which
is not present in server classpath, you will get the following erorr on server:
{code}
[15:32:57,844][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
unmarshal discovery custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given
class loader for unmarshalling (make sure same versions of all classes are
available on all nodes or enable peer-class-loading)
[clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
at
org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5476)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5305)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
... 12 more
[15:32:57,846][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
unmarshal discovery custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given
class loader for unmarshalling (make sure same versions of all classes are
available on all nodes or enable peer-class-loading)
[clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
at
org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5323)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
... 11 more
{code}
and client thread will wait forever:
{code}
"main" #1 prio=5 os_prio=0 tid=0x00007f403800d000 nid=0x1fa2 waiting on
condition [0x00007f403e74a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery0(CacheContinuousQueryManager.java:694)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery(CacheContinuousQueryManager.java:517)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.queryContinuous(IgniteCacheProxyImpl.java:569)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:658)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:344)
{code}
But otherwise both client and server are intact and no hanging PME occurs.
was:
*In 2.7*, when you create a Continuous Query on client with class which is not
present in server classpath, server node will be terminated immediately.
{code}
[2019-02-05 18:20:22,508][ERROR][tcp-disco-msg-worker-#2][root] Critical system
error detected. Will be handled accordingly to configured handler
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]],
failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class
o.a.i.IgniteException: Failed to unmarshal discovery custom message:
TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage
[sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0,
id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0,
verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0,
failedNodes=null, isClient=false]]]]
class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom
message: TcpDiscoveryCustomEventMessage [msg=null,
super=TcpDiscoveryAbstractMessage
[sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0,
id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0,
verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0,
failedNodes=null, isClient=false]]
{code}
This means that one rogue client (e.g. with mismatched code base version) can
knock whole cluster offline.
I think we should disconnect client from discovery instead when it sends
incorrect discovery message.
*In 2.6*, if you tried to create a Continuous Query on client with class which
is not present in server classpath, you will get the following erorr on server:
{code}
[15:32:57,844][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
unmarshal discovery custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given
class loader for unmarshalling (make sure same versions of all classes are
available on all nodes or enable peer-class-loading)
[clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
at
org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5476)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5305)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
... 12 more
[15:32:57,846][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
unmarshal discovery custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given
class loader for unmarshalling (make sure same versions of all classes are
available on all nodes or enable peer-class-loading)
[clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
at
org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5323)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
... 11 more
{code}
and client thread will wait forever:
{code}
"main" #1 prio=5 os_prio=0 tid=0x00007f403800d000 nid=0x1fa2 waiting on
condition [0x00007f403e74a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery0(CacheContinuousQueryManager.java:694)
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery(CacheContinuousQueryManager.java:517)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.queryContinuous(IgniteCacheProxyImpl.java:569)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:658)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:344)
{code}
But otherwise both client and server are intact and no hanging PME occurs.
> Sending Continuous Query with absent class on client knocks out server node
> ---------------------------------------------------------------------------
>
> Key: IGNITE-11235
> URL: https://issues.apache.org/jira/browse/IGNITE-11235
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.7
> Reporter: Ilya Kasnacheev
> Priority: Critical
> Labels: discovery
>
> *In 2.7*, when you create a Continuous Query on client with class which is
> not present in server classpath, server node will be terminated immediately.
> {code}
> [2019-02-05 18:20:22,508][ERROR][tcp-disco-msg-worker-#2][root] Critical
> system error detected. Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]],
> failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class
> o.a.i.IgniteException: Failed to unmarshal discovery custom message:
> TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage
> [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0,
> id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0,
> verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0,
> failedNodes=null, isClient=false]]]]
> class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom
> message: TcpDiscoveryCustomEventMessage [msg=null,
> super=TcpDiscoveryAbstractMessage
> [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0,
> id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0,
> verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0,
> failedNodes=null, isClient=false]]
> {code}
> This means that one rogue client (e.g. with mismatched code base version) can
> knock whole cluster offline.
> I think we should disconnect client from topology instead when it sends
> incorrect discovery message.
> *In 2.6*, if you tried to create a Continuous Query on client with class
> which is not present in server classpath, you will get the following erorr on
> server:
> {code}
> [15:32:57,844][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
> unmarshal discovery custom message.
> class org.apache.ignite.IgniteCheckedException: Failed to find class with
> given class loader for unmarshalling (make sure same versions of all classes
> are available on all nodes or enable peer-class-loading)
> [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
> at
> org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5476)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5305)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
> at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
> at
> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
> at
> java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at
> org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
> at
> java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
> ... 12 more
> [15:32:57,846][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to
> unmarshal discovery custom message.
> class org.apache.ignite.IgniteCheckedException: Failed to find class with
> given class loader for unmarshalling (make sure same versions of all classes
> are available on all nodes or enable peer-class-loading)
> [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
> at
> org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5323)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
> at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
> at
> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
> at
> java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at
> org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
> at
> java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
> ... 11 more
> {code}
> and client thread will wait forever:
> {code}
> "main" #1 prio=5 os_prio=0 tid=0x00007f403800d000 nid=0x1fa2 waiting on
> condition [0x00007f403e74a000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery0(CacheContinuousQueryManager.java:694)
> at
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery(CacheContinuousQueryManager.java:517)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.queryContinuous(IgniteCacheProxyImpl.java:569)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:658)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:344)
> {code}
> But otherwise both client and server are intact and no hanging PME occurs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
