[ 
https://issues.apache.org/jira/browse/GEODE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514234#comment-15514234
 ] 

ASF subversion and git services commented on GEODE-1915:
--------------------------------------------------------

Commit 4f2e27749b0f9e94d0bfe7399fddbe1e9041ecdb in incubator-geode's branch 
refs/heads/develop from [~upthewaterspout]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=4f2e277 ]

GEODE-1915: Prevent deadlock registering instantiators with gateways

Don't hold a lock while distributing instantiators. This prevents the
deadlock because incoming registrations won't wait for registrations
that are being distributed.

This change might cause instantiators to be distributed in a different
order that they were registered in, but that's ok because the order in
which different instantiators are registered is not important.


> Registering instantiators can cause a deadlock with gateways
> ------------------------------------------------------------
>
>                 Key: GEODE-1915
>                 URL: https://issues.apache.org/jira/browse/GEODE-1915
>             Project: Geode
>          Issue Type: Bug
>          Components: wan
>            Reporter: Dan Smith
>            Assignee: Dan Smith
>
> If two WAN sites are connected bidirectionally, registering an instantiator 
> in one of the sites can cause a deadlock. 
> The issue is that when an instantiator is registered, a message is sent 
> synchronously from one site to the other, while holding a static lock on the 
> InternalInstantiator class. Unfortunately, when the second site receives the 
> registration, it tries to send it back to the first site. In the first site, 
> the registeration message then is stuck trying to get the same lock.
> {code}
> "ServerConnection on port 28517 Thread 4" #80 daemon prio=5 os_prio=0 
> tid=0x00007fce78007000 nid=0xc48a runnable [0x00007fce377f7000]
>    java.lang.Thread.State: RUNNABLE
>       at java.net.SocketInputStream.socketRead0(Native Method)
>       at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
>       at java.net.SocketInputStream.read(SocketInputStream.java:170)
>       at java.net.SocketInputStream.read(SocketInputStream.java:141)
>       at 
> org.apache.geode.internal.cache.tier.sockets.Message.fetchHeader(Message.java:693)
>       at 
> org.apache.geode.internal.cache.tier.sockets.Message.readHeaderAndPayload(Message.java:710)
>       at 
> org.apache.geode.internal.cache.tier.sockets.Message.read(Message.java:661)
>       at 
> org.apache.geode.internal.cache.tier.sockets.Message.recv(Message.java:1103)
>       - locked <0x00000000fb3aec38> (a java.nio.HeapByteBuffer)
>       at 
> org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:171)
>       at 
> org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:388)
>       at 
> org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:272)
>       - locked <0x00000000fb3ad430> (a 
> org.apache.geode.cache.client.internal.ConnectionImpl)
>       at 
> org.apache.geode.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:328)
>       at 
> org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:937)
>       at 
> org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:155)
>       at 
> org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:711)
>       at 
> org.apache.geode.cache.client.internal.RegisterInstantiatorsOp.execute(RegisterInstantiatorsOp.java:49)
>       at 
> org.apache.geode.internal.cache.PoolManagerImpl.allPoolsRegisterInstantiator(PoolManagerImpl.java:227)
>       at 
> org.apache.geode.internal.InternalInstantiator.sendRegistrationMessageToServers(InternalInstantiator.java:219)
>       at 
> org.apache.geode.internal.InternalInstantiator._register(InternalInstantiator.java:174)
>       - locked <0x00000000fb3ad678> (a java.lang.Class for 
> org.apache.geode.internal.InternalInstantiator)
>       at 
> org.apache.geode.internal.InternalInstantiator.register(InternalInstantiator.java:310)
>       at 
> org.apache.geode.internal.cache.tier.sockets.command.RegisterInstantiators.cmdExecute(RegisterInstantiators.java:100)
>       at 
> org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:145)
>       at 
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783)
>       at 
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913)
>       at 
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1180)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>       at 
> org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:546)
>       at java.lang.Thread.run(Thread.java:745)
> "ServerConnection on port 28517 Thread 3" #78 daemon prio=5 os_prio=0 
> tid=0x00007fce78005000 nid=0xc487 waiting for monitor entry 
> [0x00007fce379fa000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
>       at 
> org.apache.geode.internal.InternalInstantiator._register(InternalInstantiator.java:117)
>       - waiting to lock <0x00000000fb3ad678> (a java.lang.Class for 
> org.apache.geode.internal.InternalInstantiator)
>       at 
> org.apache.geode.internal.InternalInstantiator.register(InternalInstantiator.java:310)
>       at 
> org.apache.geode.internal.cache.tier.sockets.command.RegisterInstantiators.cmdExecute(RegisterInstantiators.java:100)
>       at 
> org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:145)
>       at 
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783)
>       at 
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913)
>       at 
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1180)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>       at 
> org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:546)
>       at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to