[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786953#comment-16786953 ] Ignite TC Bot commented on IGNITE-10261: {panel:title=-- Run :: All: No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} [TeamCity *-- Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=3194790buildTypeId=IgniteTests24Java8_RunAll] > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786956#comment-16786956 ] Roman Kondakov commented on IGNITE-10261: - [~amashenkov], I've fixed the issues we've discussed. Please, take a look. Tests are fine. > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780182#comment-16780182 ] Roman Kondakov commented on IGNITE-10261: - [~amashenkov], I've fixed this test. Please, take a look. > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780176#comment-16780176 ] Ignite TC Bot commented on IGNITE-10261: {panel:title=-- Run :: All: No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} [TeamCity *-- Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=3194790buildTypeId=IgniteTests24Java8_RunAll] > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760907#comment-16760907 ] Andrew Mashenkov commented on IGNITE-10261: --- Please check "Muted Failures". GridCacheColocatedPreloadRestartSelfTest.testDisabledPreloadRestart has failed on TC. > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760005#comment-16760005 ] Ignite TC Bot commented on IGNITE-10261: {panel:title=-- Run :: All: No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} [TeamCity *-- Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=2964160buildTypeId=IgniteTests24Java8_RunAll] > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760007#comment-16760007 ] Roman Kondakov commented on IGNITE-10261: - [~amashenkov], [~gvvinblade], patch is ready for review. TC looks good. > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-10261) MVCC: cache operation may hang during late affinity assignment.
[ https://issues.apache.org/jira/browse/IGNITE-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757438#comment-16757438 ] Roman Kondakov commented on IGNITE-10261: - The root cause of the bug was the case when get request is mapped to the node with partition which is being preloaded (partition is not in the {{OWNING}} state), this causes a {{ForceKeyRequest}} to the supplier. This request is not supported by MVCC caches, even more, there are intentions to get rid of it for non-MVCC caches too (IGNITE-10251). So the decision is to prohibit {{ForceKeyRequest}} for MVCC caches and send error back to the near node if partition is not in the {{OWNING}} state. In this case near node resends request to the next affinity node to obtain a value, until the {{OWNING}} partition is found. > MVCC: cache operation may hang during late affinity assignment. > --- > > Key: IGNITE-10261 > URL: https://issues.apache.org/jira/browse/IGNITE-10261 > Project: Ignite > Issue Type: Bug > Components: mvcc >Affects Versions: 2.7 >Reporter: Andrew Mashenkov >Assignee: Roman Kondakov >Priority: Critical > Labels: failover, mvcc_stabilization_stage_1 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > ForceKey response processing fails with ClassCastException with Mvcc mode > that causes test hanging. > Issue can be easily reproduced with backups > 0 and disabled rebalance. See > GridCacheDhtPreloadPutGetSelfTest.testPutGetNone1(). > Also CacheLateAffinityAssignmentTest.testRandomOperations() hangs sometimes > due to same reason. > > {noformat} > java.lang.ClassCastException: > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry > cannot be cast to > org.apache.ignite.internal.processors.cache.mvcc.MvccVersionAware > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:545) > at > org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:202) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:180) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:208) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$11.onMessage(GridDhtTransactionalCacheAdapter.java:206) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1434) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1416) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)