[ 
https://issues.apache.org/jira/browse/IGNITE-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745447#comment-14745447
 ] 

Semen Boikov commented on IGNITE-1400:
--------------------------------------

Found on TC issue when exchange worker hangs on node stop:

Exchange-worker:
{noformat}
        at 
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
        at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.updateTopologyVersion(GridDhtPartitionTopologyImpl.java:209)
        at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:714)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1159)
        at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

Some system thread:
{noformat}
        at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:115)
        at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.awaitTopologyVersion(GridAffinityAssignmentCache.java:463)
        at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.cachedAffinity(GridAffinityAssignmentCache.java:421)
        at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:386)
        at 
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodes(GridCacheAffinityManager.java:225)
        at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.checkEvictions(GridDhtPartitionTopologyImpl.java:1037)
        at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.update(GridDhtPartitionTopologyImpl.java:1006)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(GridCachePartitionExchangeManager.java:945)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1100(GridCachePartitionExchangeManager.java:95)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:262)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:260)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:1397)
        at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:1379)
        at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:573)
        at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:268)
        at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:191)
        at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:77)
        at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:160)
        at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:811)
        at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:106)
        at 
org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:774)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

Before this fix GridAffinityAssignmentCache.onKernalStop was called before 
exchange manager onKernalStop and all affinity futures were already cancelled.

To avoid this hang I changed GridDhtPartitionTopology methods called by 
exchange worker to allow interruption on topology lock. 

> Race during concurrent node stop and cache creation
> ---------------------------------------------------
>
>                 Key: IGNITE-1400
>                 URL: https://issues.apache.org/jira/browse/IGNITE-1400
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>    Affects Versions: ignite-1.4
>            Reporter: Alexey Goncharuk
>             Fix For: ignite-1.4
>
>
> When node attempts to create a cache and is being stopped, both processes 
> hang. The hang happens because gateway is blocked and cache create future is 
> not cancelled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to