Dan Smith created GEODE-1915: -------------------------------- Summary: Registering instantiators can cause a deadlock with gateways Key: GEODE-1915 URL: https://issues.apache.org/jira/browse/GEODE-1915 Project: Geode Issue Type: Bug Components: wan Reporter: Dan Smith
If two WAN sites are connected bidirectionally, registering an instantiator in one of the sites can cause a deadlock. The issue is that when an instantiator is registered, a message is sent synchronously from one site to the other, while holding a static lock on the InternalInstantiator class. Unfortunately, when the second site receives the registration, it tries to send it back to the first site. In the first site, the registeration message then is stuck trying to get the same lock. {code} "ServerConnection on port 28517 Thread 4" #80 daemon prio=5 os_prio=0 tid=0x00007fce78007000 nid=0xc48a runnable [0x00007fce377f7000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at org.apache.geode.internal.cache.tier.sockets.Message.fetchHeader(Message.java:693) at org.apache.geode.internal.cache.tier.sockets.Message.readHeaderAndPayload(Message.java:710) at org.apache.geode.internal.cache.tier.sockets.Message.read(Message.java:661) at org.apache.geode.internal.cache.tier.sockets.Message.recv(Message.java:1103) - locked <0x00000000fb3aec38> (a java.nio.HeapByteBuffer) at org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:171) at org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:388) at org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:272) - locked <0x00000000fb3ad430> (a org.apache.geode.cache.client.internal.ConnectionImpl) at org.apache.geode.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:328) at org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:937) at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:155) at org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:711) at org.apache.geode.cache.client.internal.RegisterInstantiatorsOp.execute(RegisterInstantiatorsOp.java:49) at org.apache.geode.internal.cache.PoolManagerImpl.allPoolsRegisterInstantiator(PoolManagerImpl.java:227) at org.apache.geode.internal.InternalInstantiator.sendRegistrationMessageToServers(InternalInstantiator.java:219) at org.apache.geode.internal.InternalInstantiator._register(InternalInstantiator.java:174) - locked <0x00000000fb3ad678> (a java.lang.Class for org.apache.geode.internal.InternalInstantiator) at org.apache.geode.internal.InternalInstantiator.register(InternalInstantiator.java:310) at org.apache.geode.internal.cache.tier.sockets.command.RegisterInstantiators.cmdExecute(RegisterInstantiators.java:100) at org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:145) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1180) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:546) at java.lang.Thread.run(Thread.java:745) "ServerConnection on port 28517 Thread 3" #78 daemon prio=5 os_prio=0 tid=0x00007fce78005000 nid=0xc487 waiting for monitor entry [0x00007fce379fa000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.geode.internal.InternalInstantiator._register(InternalInstantiator.java:117) - waiting to lock <0x00000000fb3ad678> (a java.lang.Class for org.apache.geode.internal.InternalInstantiator) at org.apache.geode.internal.InternalInstantiator.register(InternalInstantiator.java:310) at org.apache.geode.internal.cache.tier.sockets.command.RegisterInstantiators.cmdExecute(RegisterInstantiators.java:100) at org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:145) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913) at org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1180) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:546) at java.lang.Thread.run(Thread.java:745) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)