[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943541#comment-16943541
]
Ilya Kasnacheev commented on IGNITE-9068:
-
[~mmuzaf]I remember this issue was mostly fixed elsewhere and no longer
presents. Maybe we should close this ticket or at least deprioritize.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.8
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942818#comment-16942818
]
Maxim Muzafarov commented on IGNITE-9068:
-
[~ilantukh], [~ilyak], [~agoncharuk]
Folks, any updates? Can we move it to the next release since there is no
attention to this issue for more that one year?
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.8
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631413#comment-16631413
]
Nikolay Izhikov commented on IGNITE-9068:
-
Hello, [~ilantukh], [~ilyak], [~agoncharuk]
Can we move this ticket to the 2.8 release?
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.prepareServiceConfigurations(GridServiceProcessor.java:570)
> at
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602803#comment-16602803
]
Ilya Kasnacheev commented on IGNITE-9068:
-
[~ilantukh] I have passed full thread dump to you privately since it may
contain sensitive information. Please take a look.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.prepareServiceConfigurations(GridServiceProcessor.java:570)
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602213#comment-16602213
]
Ilya Lantukh commented on IGNITE-9068:
--
We need full thread dump from all nodes if such situation happens again.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.prepareServiceConfigurations(GridServiceProcessor.java:570)
> at
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597301#comment-16597301
]
Alexey Goncharuk commented on IGNITE-9068:
--
[~ilantukh] Let's add diagnostic information about the threads that are holding
read locks if we fail to acquire the write lock for a long time.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590264#comment-16590264
]
Ilya Kasnacheev commented on IGNITE-9068:
-
I don't really have ideas how to make it more correct. This test attains same
thread dump that was observed on a problematic node.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.prepareServiceConfigurations(GridServiceProcessor.java:570)
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580912#comment-16580912
]
Alexey Goncharuk commented on IGNITE-9068:
--
[~ilantukh] the fix does not fix anything because we already drop the attempt
to acquire the lock after timeout elapses.
[~ilyak], the reproducer is incorrect because it blocks forever in methods
which are designed to timeout. We need more information to investigate.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Blocker
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568159#comment-16568159
]
Ilya Lantukh commented on IGNITE-9068:
--
https://ci.ignite.apache.org/viewLog.html?buildId=1584926=buildResultsDiv=IgniteTests24Java8_RunAll
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Major
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:157)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:144)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
> at
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10069)
> at
> org.apache.ignite.internal.processors.service.GridServiceProcessor.prepareServiceConfigurations(GridServiceProcessor.java:570)
> at
>
[jira] [Commented] (IGNITE-9068) Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed inside guard()/unguard()
[
https://issues.apache.org/jira/browse/IGNITE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568155#comment-16568155
]
ASF GitHub Bot commented on IGNITE-9068:
GitHub user ilantukh opened a pull request:
https://github.com/apache/ignite/pull/4482
IGNITE-9068 : Avoided potential deadlock when service is deployed while
node is stopping.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9068
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4482.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4482
commit 44cff17b4f2c444b2cbb71d70b735066e96f1bb8
Author: Ilya Lantukh
Date: 2018-08-03T12:31:31Z
IGNITE-9068 : Avoided potential deadlock when service is deployed while
node is stopping.
> Node fails to stop when CacheObjectBinaryProcessor.addMeta() is executed
> inside guard()/unguard()
> -
>
> Key: IGNITE-9068
> URL: https://issues.apache.org/jira/browse/IGNITE-9068
> Project: Ignite
> Issue Type: Bug
> Components: binary, managed services
>Affects Versions: 2.5
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Lantukh
>Priority: Major
> Labels: test
> Fix For: 2.7
>
> Attachments: GridServiceDeadlockTest.java, MyService.java
>
>
> When addMeta is called in e.g. service deployment it us executed inside
> guard()/unguard()
> If node will be stopped at this point, Ignite.stop() will hang.
> Consider the following thread dump:
> {code}
> "Thread-1" #57 prio=5 os_prio=0 tid=0x7f7780005000 nid=0x7f26 runnable
> [0x7f766cbef000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0005cb7b0468> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
> at
> org.apache.ignite.internal.util.StripedCompositeReadWriteLock$WriteLock.tryLock(StripedCompositeReadWriteLock.java:220)
> at
> org.apache.ignite.internal.GridKernalGatewayImpl.tryWriteLock(GridKernalGatewayImpl.java:143)
> // Waiting for lock to cancel futures of BinaryMetadataTransport
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2171)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2094)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2545)
> - locked <0x0005cb423f00> (a
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2508)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:2033)
> "test-runner-#1%service.GridServiceDeadlockTest%" #13 prio=5 os_prio=0
> tid=0x7f77b87d5800 nid=0x7eb8 waiting on condition [0x7f778cdfc000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> // May never return if there's discovery problems
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:463)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:188)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerUserClassDescriptor(BinaryContext.java:802)
> at
> org.apache.ignite.internal.binary.BinaryContext.registerClassDescriptor(BinaryContext.java:761)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:627)
> at
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:174)
> at
>
