[
https://issues.apache.org/jira/browse/GEODE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982984#comment-16982984
]
Kirk Lund edited comment on GEODE-7503 at 4/15/20, 6:02 PM:
------------------------------------------------------------
One known cause in dunit tests is when creation of a persistent region fails
and DiskStoreImpl.lambda$handleDiskAccessException forks a new Thread to close
the Cache which succeeds in closing the Cache before the main test thread
(which is invoking Cache.close()) closes it. The main test thread then
early-outs because the DiskStore thread is already closing the Cache. The main
test thread then tries to create a Cache which collides with the DiskStore
thread which is still closing the Cache and is about to disconnect the
DistributedSystem.
{noformat}
java.lang.Throwable: KIRK GemFireCacheImpl closed 632230948
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:2365)
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:1917)
at
org.apache.geode.internal.cache.DiskStoreImpl.lambda$handleDiskAccessException$2(DiskStoreImpl.java:3380)
at java.lang.Thread.run(Thread.java:748)
{noformat}
{noformat}
java.lang.Throwable: KIRK InternalDistributedSystem closed 306674056
at
org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:1637)
at
org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:1225)
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:2351)
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:1917)
at
org.apache.geode.internal.cache.DiskStoreImpl.lambda$handleDiskAccessException$2(DiskStoreImpl.java:3380)
at java.lang.Thread.run(Thread.java:748)
{noformat}
{noformat}
java.lang.Throwable: KIRK DiskStoreImpl closing cache 1555793073
at
org.apache.geode.internal.cache.DiskStoreImpl.handleDiskAccessException(DiskStoreImpl.java:3376)
at
org.apache.geode.internal.cache.PartitionedRegion.createAndValidatePersistentConfig(PartitionedRegion.java:956)
at
org.apache.geode.internal.cache.PartitionedRegion.initPRInternals(PartitionedRegion.java:999)
at
org.apache.geode.internal.cache.PartitionedRegion.initialize(PartitionedRegion.java:1179)
at
org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3043)
at
org.apache.geode.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2931)
at
org.apache.geode.internal.cache.GemFireCacheImpl.createRegion(GemFireCacheImpl.java:2918)
at org.apache.geode.cache.RegionFactory.create(RegionFactory.java:755)
at
org.apache.geode.cache.CacheFactoryRecreateRegressionTest.createCacheAndColocatedPRs(CacheFactoryRecreateRegressionTest.java:109)
at
org.apache.geode.cache.CacheFactoryRecreateRegressionTest.lambda$recreateDoesNotThrowDistributedSystemDisconnectedException$0(CacheFactoryRecreateRegressionTest.java:62)
at
org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:62)
at
org.assertj.core.api.AssertionsForClassTypes.catchThrowable(AssertionsForClassTypes.java:786)
at org.assertj.core.api.Assertions.catchThrowable(Assertions.java:1200)
at
org.apache.geode.cache.CacheFactoryRecreateRegressionTest.recreateDoesNotThrowDistributedSystemDisconnectedException(CacheFactoryRecreateRegressionTest.java:62)
{noformat}
was (Author: klund):
This bug appears to be caused when creation of a persistent region fails and
DiskStoreImpl.lambda$handleDiskAccessException forks a new Thread to close the
Cache which succeeds in closing the Cache before the main test thread closes
it. The main test thread then early outs because the DiskStore thread is
already closing the Cache. The main test thread then tries to create a Cache
which collides with the DiskStore thread which is still closing the Cache and
DistributedSystem.
{noformat}
java.lang.Throwable: KIRK GemFireCacheImpl closed 632230948
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:2365)
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:1917)
at
org.apache.geode.internal.cache.DiskStoreImpl.lambda$handleDiskAccessException$2(DiskStoreImpl.java:3380)
at java.lang.Thread.run(Thread.java:748)
{noformat}
{noformat}
java.lang.Throwable: KIRK InternalDistributedSystem closed 306674056
at
org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:1637)
at
org.apache.geode.distributed.internal.InternalDistributedSystem.disconnect(InternalDistributedSystem.java:1225)
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:2351)
at
org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:1917)
at
org.apache.geode.internal.cache.DiskStoreImpl.lambda$handleDiskAccessException$2(DiskStoreImpl.java:3380)
at java.lang.Thread.run(Thread.java:748)
{noformat}
{noformat}
java.lang.Throwable: KIRK DiskStoreImpl closing cache 1555793073
at
org.apache.geode.internal.cache.DiskStoreImpl.handleDiskAccessException(DiskStoreImpl.java:3376)
at
org.apache.geode.internal.cache.PartitionedRegion.createAndValidatePersistentConfig(PartitionedRegion.java:956)
at
org.apache.geode.internal.cache.PartitionedRegion.initPRInternals(PartitionedRegion.java:999)
at
org.apache.geode.internal.cache.PartitionedRegion.initialize(PartitionedRegion.java:1179)
at
org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3043)
at
org.apache.geode.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2931)
at
org.apache.geode.internal.cache.GemFireCacheImpl.createRegion(GemFireCacheImpl.java:2918)
at org.apache.geode.cache.RegionFactory.create(RegionFactory.java:755)
at
org.apache.geode.cache.CacheFactoryRecreateRegressionTest.createCacheAndColocatedPRs(CacheFactoryRecreateRegressionTest.java:109)
at
org.apache.geode.cache.CacheFactoryRecreateRegressionTest.lambda$recreateDoesNotThrowDistributedSystemDisconnectedException$0(CacheFactoryRecreateRegressionTest.java:62)
at
org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:62)
at
org.assertj.core.api.AssertionsForClassTypes.catchThrowable(AssertionsForClassTypes.java:786)
at org.assertj.core.api.Assertions.catchThrowable(Assertions.java:1200)
at
org.apache.geode.cache.CacheFactoryRecreateRegressionTest.recreateDoesNotThrowDistributedSystemDisconnectedException(CacheFactoryRecreateRegressionTest.java:62)
{noformat}
> Creating cache after closing cache may throw
> DistributedSystemDisconnectedException
> -----------------------------------------------------------------------------------
>
> Key: GEODE-7503
> URL: https://issues.apache.org/jira/browse/GEODE-7503
> Project: Geode
> Issue Type: Bug
> Components: core
> Reporter: Kirk Lund
> Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> This problem is caused by a race condition and may reproduce if you attempt
> to create a Cache using the CacheFactory immediately after more than one
> thread invokes Cache.close().
> {noformat}
> org.apache.geode.distributed.DistributedSystemDisconnectedException: This
> connection to a distributed system has been disconnected.
> at
> org.apache.geode.distributed.internal.InternalDistributedSystem.checkConnected(InternalDistributedSystem.java:945)
> at
> org.apache.geode.distributed.internal.InternalDistributedSystem.getDistributionManager(InternalDistributedSystem.java:1665)
> at
> org.apache.geode.internal.cache.GemFireCacheImpl.<init>(GemFireCacheImpl.java:791)
> at
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:187)
> at
> org.apache.geode.internal.cache.InternalCacheBuilder.create(InternalCacheBuilder.java:158)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:142)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
