[
https://issues.apache.org/jira/browse/IGNITE-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603580#comment-17603580
]
Nikita Amelchev commented on IGNITE-16926:
------------------------------------------
[~ibessonov], [[email protected]], Hi guys.
Could you please explain why the fileIO is not being closed now?
This leads to unable to delete these files (freeing space) until the process is
complete on unix.
> Interrupted compute job may fail a node
> ---------------------------------------
>
> Key: IGNITE-16926
> URL: https://issues.apache.org/jira/browse/IGNITE-16926
> Project: Ignite
> Issue Type: Bug
> Components: persistence
> Reporter: Ivan Bessonov
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ise.lts
> Fix For: 2.14
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is
> corrupted [groupId=1234619879, pageIds=[72902014673333513],
> cacheId=645096946, cacheName=*, indexName=*, msg=Runtime failure on row:
> Row@79570772[ key: 1168930235, val: Data hidden due to
> IGNITE_SENSITIVE_DATA_LOGGING flag. ][ data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden
> ]]]]","logger_name":"ROOT","thread_name":"pub-#1278%x%","level":"ERROR","level_value":40000,"stack_trace":"org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> B+Tree is corrupted [groupId=1234619879, pageIds=[72902014673333513],
> cacheId=645096946, cacheName=*, indexName=*, msg=Runtime failure on row:
> Row@79570772[ key: 1168930235, val: Data hidden due to
> IGNITE_SENSITIVE_DATA_LOGGING flag. ][ data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden, data hidden, data hidden, data hidden, data hidden, data hidden,
> data hidden ]] at
> org.apache.ignite.internal.processors.query.h2.database.H2Tree.corruptedTreeException(H2Tree.java:1003)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2492)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putx(BPlusTree.java:2432)
> at
> org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:500)
> at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.addToIndex(GridH2Table.java:880)
> at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:794)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:411)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:2546)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:418)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:2815)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1871)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1842)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:3067)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:471)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2262)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2539)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1999)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1830)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1685)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:319)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture.map(GridNearAtomicUpdateFuture.java:825)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture.mapOnTopology(GridNearAtomicUpdateFuture.java:673)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:267)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAll0(GridDhtAtomicCache.java:1118)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.putAll0(GridDhtAtomicCache.java:669)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAll(GridCacheAdapter.java:2564)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.putAll(IgniteCacheProxyImpl.java:1382)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.putAll(GatewayProtectedCacheProxy.java:865)
> at x.Job.execute(...) at
> org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:609)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7435)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:603)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:528)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)\n Caused by:
> org.apache.ignite.IgniteException: Failed to close WAL write handle
> [idx=13554] at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1713)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:525)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:517)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writeUnlock(PageHandler.java:401)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:376)
> at
> org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:347)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$11400(BPlusTree.java:100)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.tryReplace(BPlusTree.java:3932)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2863)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2843)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2843)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2843)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2452)\n
> \t... 35 common frames omitted\n Caused by:
> org.apache.ignite.internal.processors.cache.persistence.StorageException:
> Failed to close WAL write handle [idx=13554] at
> org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl.close(FileWriteHandleImpl.java:554)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.rollOver(FileWriteAheadLogManager.java:1300)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:906)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:827)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.beforeReleaseWrite(PageMemoryImpl.java:1879)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1710)\n
> \t... 47 common frames omitted\n Caused by:
> java.nio.channels.ClosedByInterruptException: null at
> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
> at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:392) at
> org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:122)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:147)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FileIODecorator.force(FileIODecorator.java:100)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl.close(FileWriteHandleImpl.java:533)\n
> \t... 52 common frames omitted\n
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
