[
https://issues.apache.org/jira/browse/IGNITE-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652035#comment-16652035
]
Ignite TC Bot commented on IGNITE-9739:
---------------------------------------
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Hadoop{color} [[tests 0 TIMEOUT , Out Of Memory Error , Exit
Code |https://ci.ignite.apache.org/viewLog.html?buildId=2089514]]
* HadoopMapReduceErrorResilienceTest.testRecoveryAfterAnError7_Runtime (last
started)
{color:#d04437}_Javadoc_{color} [[tests 0 Exit Code , BuildFailureOnMessage
|https://ci.ignite.apache.org/viewLog.html?buildId=2089563]]
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=2089597&buildTypeId=IgniteTests24Java8_RunAll]
> Critical exception in transaction processing in case we have nodes out of
> baseline and non-persisted cache
> ----------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-9739
> URL: https://issues.apache.org/jira/browse/IGNITE-9739
> Project: Ignite
> Issue Type: Bug
> Components: persistence
> Reporter: Sergey Kosarev
> Assignee: Sergey Kosarev
> Priority: Major
>
> Activation finished
> {code:java}
> 2018-09-20 20:47:05.169 [INFO
> ][sys-#307%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
> Successfully performed final activation steps
> [nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557, client=false,
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=1]]
> {code}
> but we have nodes not in base line
> {code:java}
> 2018-09-20 20:45:36.116 [INFO
> ][sys-#305%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
> Local node is not included in Baseline Topology and will not be used for
> persistent data storage. Use control.(sh|bat) script or IgniteCluster
> interface to include the node to Baseline Topology.
> {code}
> And we have cache (869481129) in the data region with persistanceEnabled=false
> {code:java}
> 2018-09-20 20:49:01.825 [INFO
> ][exchange-worker-#154%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheProcessor]
> Started cache [name=DPL_PUBLISHED_CACHES_REGISTRY$, *id=869481129*, group=SY
> STEM_CACHEGROUP_PUBLISHED_REGISTRY, memoryPolicyName=not-persisted,
> mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=3]
> {code}
> Transaction on this cache(869481129)
> {code:java}
> 869481129{code}
> leads to critical error causing nodes by faulure handler:
> {code:java}
> 2018-09-20 20:50:24.275
> [ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed processing message [senderId=62e986f0-62b5-4ec8-8cc7-27b74d345235,
> msg=GridDhtTxPrepareRequest [nearNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774,
> futId=520e308f561-255fdea5-a996-4102-a120-afa380c54570, miniId=1,
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2],
> invalidateNearEntries={}, nearWrites=null, owned=null,
> nearXidVer=GridCacheVersion [topVer=148944365, order=1537511036821,
> nodeOrder=132], subjId=814af7c4-2de5-4511-b1ea-065b91eaa774, taskNameHash=0,
> preloadKeys=null, skipCompletedVers=false,
> super=GridDistributedTxPrepareRequest [threadId=58, concurrency=PESSIMISTIC,
> isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=148944365,
> order=1537511036824, nodeOrder=7], timeout=299970, reads=null,
> writes=ArrayList [
> IgniteTxEntry [key=KeyCacheObjectImpl [part=27254,
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
> *cacheId=869481129*,
> txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=27254,
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
> *cacheId=869481129*], val=[op=CREATE,
> val=com.sbt.dpl.gridgain.PublishedRegistry$PublishedCacheTuple
> [idHash=811765531, hash=1522508040,
> cacheName=com.sbt.gbk.entities.DocType_DPL_union-module,indexes=ArrayList
> {com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=1583970836, hash=363194492, isSoftReference=false,
> unselectiveBuckets=4096, fieldNames=ArrayList
> \{isDeleted},moduleName=union-module
> , cachedUnselectives=10000, selectors=ArrayList {isDeleted},
> exceptUnselectives=false, primitiveCollection=false, isVersioned=false,
> isComposite=false, isSystemTypeBelongs=false,
> name=com.sbt.gbk.entities.DocType_DPL_isDeleted, isIndexedCollection=false,
> isGlobal=false, maxSelective=1000],
> com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=2060926101, hash=1983794578, isSoftReference=false,
> unselectiveBuckets=4096, fieldNames=ArrayList ,moduleName=union-module,
> cachedUnselectives=10000, selectors=ArrayList, exceptUnselectives=false,
> primitiveCollection=false, isVersioned=false, isComposite=false,
> isSystemTypeBelongs=false, name=com.sbt.gbk.entities.DocType_DPL_code,
> isIndexedCollection=false, isGlobal=true, maxSelective=1000]
> , com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=1821682714, hash=-1245813786, isSoftReference=false,
> unselectiveBuckets=4096, fieldNames=ArrayList {globalId},
> moduleName=union-module, cachedUnselectives=10000, selectors=ArrayList
> {globalId}, exceptUnselectives=false, primitiveCollection=false,
> isVersioned=false, isComposite=false, isSystemTypeBelongs=false,
> name=com.sbt.gbk.entities.DocType_DPL_globalId, isIndexedCollection=false,
> isGlobal=false, maxSelective=1000]
> }, partitionDependencyClassName=null, moduleName=union-module,
> cacheModuleName=union-module]
> ], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null],
> entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null,
> explicitVer=null, dhtVer=null, filters=CacheEntryPredicate[] [],
> filtersPassed=false, filtersSet=false, entry=GridCacheMapEntry
> [key=KeyCacheObjectImpl [part=27254,
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true], val=null,
> startVer=1537511036806, ver=GridCacheVersion [topVer=148944365,
> order=1537511036806, nodeOrder=86], hash=-1897857526,
> extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=null, rmts=Link
> edList [GridCacheMvccCandidate [nodeId=62e986f0-62b5-4ec8-8cc7-27b74d345235,
> ver=GridCacheVersion [topVer=148944365, order=1537511036822, nodeOrder=7],
> threadId=302, id=2, topVer=AffinityTopologyVersion [topVer=-1,
> minorTopVer=0], reentry=null,
> otherNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774, otherVer=null,
> mappedDhtNodes=null, mappedNearNodes=null, ownerVer=null, serOrder=null,
> key=KeyCacheObjectImpl [part=27254,
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
> masks=local=0|owner=0|ready=0|reentry=0|used=0|tx=1|single_implicit=0|dht_local=0|near_loc
> al=0|removed=0|read=0, prevVer=null, nextVer=null]]]],
> flags=2]GridDistributedCacheEntry [super=]GridDhtCacheEntry [rdrs=ReaderId[]
> [], part=27254, super=], prepared=1, locked=false, nodeId=null,
> locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=0,
> partUpdateCntr=0, serReadVer=null, xidVer=null]], dhtVers=null, txSize=0,
> plc=2, txState=null, flags=last, super=GridDistributedBaseMessage
> [ver=GridCacheVersion [topVer=148944365, order=1537511036822, nodeOrder=7],
> committedVers=null, rolledbackVers=null, cnt=0, super=GridCacheIdMessage
> [cacheId=0]]]]]
> java.lang.IllegalStateException: Unable to find consistentId by UUID
> [*nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557*,
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2]]
> at
> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactId(ConsistentIdMapper.java:62)
> at
> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactIds(ConsistentIdMapper.java:123)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1108)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:959)
> at
> org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.prepareRemoteTx(GridDistributedTxRemoteAdapter.java:409)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.startRemoteTx(IgniteTxHandler.java:1758)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareRequest(IgniteTxHandler.java:1125)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$400(IgniteTxHandler.java:101)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:205)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:203)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
> at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:496)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> 2018-09-20 20:50:24.277
> [ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][org.apache.ignite.Ignite]
> Critical system error detected. Will be handled accordingly to configured
> handler
> {code}
> nodeId (382437eb-fd8a-4f92-acd5-d9ea562c8557) causing the error belongs to
> the node out of baseline.
> I found this issue only happens if system property IGNITE_WAL_LOG_TX_RECORDS
> = true
> when we try write to WAL information about transaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
