Hi, Can you attach logs in normal format? It's really hard to read it. Also, please attach full logs from nodes, not only the stacktrace.
Thanks, Evgenii вт, 25 авг. 2020 г. в 19:27, Steven Zheng <[email protected]>: > Hi community, > Currently I have 25 nodes in my ignite cluster and all of them were added > into the baseline, and I was trying to add another 5 nodes into it. The > data in my cluster is about 8TB and the persistence is enabled. > At first I start all the 5 nodes; then execute in the command line: > ``` > bin/control.sh --baseline add ${my_node_id} > ``` > Meanwhile, there is still read/write workloads on the cluster. > > After a few minutes, a critical error detected , several nodes crashed and > emits the logs like this(json format): > ``` > { > "message": "Critical system error detected. Will be handled accordingly > to configured handler [hnd=StopNodeFailureHandler > [super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > failureCtx=FailureContext [type=CRITICAL_ERROR, err=class > o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is > corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=-1526563570, > val2=844420635165881]], cacheId=-2021712086, cacheName=MY_SQL_TABLE, > indexName=AFFINITY_KEY, msg=Runtime failure on search row: Row@d00bc1[ > key: SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf_KEY > [idHash=2015873411, hash=-2113698004, CARDNUMBER=25323111322, IP=], val: > SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf > [idHash=1729397221, hash=2090272279, MY_BIZ_UID=0, ETL_TIME=1596023962011] > ][ 25323111322, , 0, 1596023962011 ]]]]", > "loggerFqcn": "org.apache.logging.log4j.spi.AbstractLogger", > "thrown": { > "localizedMessage": "B+Tree is corrupted [pages(groupId, > pageId)=[IgniteBiTuple [val1=-1526563570, val2=844420635165881]], > cacheId=-2021712086, cacheName=MY_SQL_TABLE, indexName=AFFINITY_KEY, > msg=Runtime failure on search row: Row@d00bc1[ key: > SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf_KEY > [idHash=2015873411, hash=-2113698004, CARDNUMBER=25323111322, IP=], val: > SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf > [idHash=1729397221, hash=2090272279, MY_BIZ_UID=0, ETL_TIME=1596023962011] > ][ 25323111322, , 0, 1596023962011 ]]", > "message": "B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple > [val1=-1526563570, val2=844420635165881]], cacheId=-2021712086, > cacheName=MY_SQL_TABLE, indexName=AFFINITY_KEY, msg=Runtime failure on > search row: Row@d00bc1[ key: > SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf_KEY > [idHash=2015873411, hash=-2113698004, CARDNUMBER=25323111322, IP=], val: > SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf > [idHash=1729397221, hash=2090272279, MY_BIZ_UID=0, ETL_TIME=1596023962011] > ][ 25323111322, , 0, 1596023962011 ]]", > "commonElementCount": 0, > "cause": { > "localizedMessage": "java.lang.IllegalStateException: Item not > found: 27", > "message": "java.lang.IllegalStateException: Item not found: 27", > "commonElementCount": 21, > "cause": { > "localizedMessage": "Item not found: 27", > "message": "Item not found: 27", > "commonElementCount": 21, > "name": "java.lang.IllegalStateException", > "extendedStackTrace": [{ > "line": 351, > "method": "findIndirectItemIndex", > "exact": false, > "file": "AbstractDataPageIO.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 459, > "method": "getDataOffset", > "exact": false, > "file": "AbstractDataPageIO.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 501, > "method": "readPayload", > "exact": false, > "file": "AbstractDataPageIO.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 325, > "method": "readIncomplete", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 261, > "method": "doInitFromLink", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 160, > "method": "initFromLink", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 131, > "method": "initFromLink", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 277, > "method": "createRow0", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 262, > "method": "createRow", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 154, > "method": "getLookupRow", > "exact": false, > "file": "AbstractH2ExtrasLeafIO.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 36, > "method": "getLookupRow", > "exact": false, > "file": "AbstractH2ExtrasLeafIO.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 339, > "method": "getRow", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 63, > "method": "getRow", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5218, > "method": "getRow", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 418, > "method": "compare", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 63, > "method": "compare", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5202, > "method": "compare", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5121, > "method": "findInsertionPoint", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 95, > "method": "access$1100", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 299, > "method": "run0", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5714, > "method": "run", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 279, > "method": "run", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5700, > "method": "run", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 169, > "method": "readPage", > "exact": false, > "file": "PageHandler.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 364, > "method": "read", > "exact": false, > "file": "DataStructure.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.DataStructure", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5901, > "method": "read", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2064, > "method": "removeDown", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2083, > "method": "removeDown", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2083, > "method": "removeDown", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 1994, > "method": "doRemove", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > } > ] > }, > "name": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTreeRuntimeException", > "extendedStackTrace": [{ > "line": 290, > "method": "doInitFromLink", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 160, > "method": "initFromLink", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 131, > "method": "initFromLink", > "exact": false, > "file": "CacheDataRowAdapter.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 277, > "method": "createRow0", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 262, > "method": "createRow", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 154, > "method": "getLookupRow", > "exact": false, > "file": "AbstractH2ExtrasLeafIO.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 36, > "method": "getLookupRow", > "exact": false, > "file": "AbstractH2ExtrasLeafIO.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 339, > "method": "getRow", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 63, > "method": "getRow", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5218, > "method": "getRow", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 418, > "method": "compare", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 63, > "method": "compare", > "exact": false, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5202, > "method": "compare", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5121, > "method": "findInsertionPoint", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 95, > "method": "access$1100", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 299, > "method": "run0", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5714, > "method": "run", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 279, > "method": "run", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5700, > "method": "run", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 169, > "method": "readPage", > "exact": false, > "file": "PageHandler.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 364, > "method": "read", > "exact": false, > "file": "DataStructure.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.DataStructure", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 5901, > "method": "read", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2064, > "method": "removeDown", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2083, > "method": "removeDown", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2083, > "method": "removeDown", > "exact": false, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 1994, > "method": "doRemove", > "exact": true, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > } > ] > }, > "name": > "org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException", > "extendedStackTrace": [{ > "line": 678, > "method": "corruptedTreeException", > "exact": true, > "file": "H2Tree.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2Tree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2029, > "method": "doRemove", > "exact": true, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 1825, > "method": "removex", > "exact": true, > "file": "BPlusTree.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 453, > "method": "removex", > "exact": true, > "file": "H2TreeIndex.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 805, > "method": "remove", > "exact": true, > "file": "GridH2Table.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.opt.GridH2Table", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 404, > "method": "remove", > "exact": true, > "file": "IgniteH2Indexing.java", > "location": "ignite-indexing-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2828, > "method": "remove", > "exact": true, > "file": "GridQueryProcessor.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.query.GridQueryProcessor", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 440, > "method": "remove", > "exact": true, > "file": "GridCacheQueryManager.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2695, > "method": "finishRemove", > "exact": true, > "file": "IgniteCacheOffheapManagerImpl.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2672, > "method": "remove", > "exact": true, > "file": "IgniteCacheOffheapManagerImpl.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 2439, > "method": "remove", > "exact": true, > "file": "GridCacheOffheapManager.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 615, > "method": "remove", > "exact": true, > "file": "IgniteCacheOffheapManagerImpl.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 4403, > "method": "removeValue", > "exact": true, > "file": "GridCacheMapEntry.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.GridCacheMapEntry", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 678, > "method": "clearInternal", > "exact": true, > "file": "GridDhtCacheEntry.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 1175, > "method": "clearAll", > "exact": true, > "file": "GridDhtLocalPartition.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 938, > "method": "tryClear", > "exact": true, > "file": "GridDhtLocalPartition.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 417, > "method": "run", > "exact": true, > "file": "PartitionsEvictManager.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$PartitionEvictionTask", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 7061, > "method": "wrapThreadLoader", > "exact": true, > "file": "IgniteUtils.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": "org.apache.ignite.internal.util.IgniteUtils", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 827, > "method": "body", > "exact": true, > "file": "GridClosureProcessor.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": > "org.apache.ignite.internal.processors.closure.GridClosureProcessor$1", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 120, > "method": "run", > "exact": true, > "file": "GridWorker.java", > "location": "ignite-core-2.8.0-SNAPSHOT.jar", > "class": "org.apache.ignite.internal.util.worker.GridWorker", > "version": "2.8.0-SNAPSHOT" > }, > { > "line": 1149, > "method": "runWorker", > "exact": true, > "file": "ThreadPoolExecutor.java", > "location": "?", > "class": "java.util.concurrent.ThreadPoolExecutor", > "version": "1.8.0_221" > }, > { > "line": 624, > "method": "run", > "exact": true, > "file": "ThreadPoolExecutor.java", > "location": "?", > "class": "java.util.concurrent.ThreadPoolExecutor$Worker", > "version": "1.8.0_221" > }, > { > "line": 748, > "method": "run", > "exact": true, > "file": "Thread.java", > "location": "?", > "class": "java.lang.Thread", > "version": "1.8.0_221" > } > ] > }, > "level": "ERROR" > } > ``` > > Can anyone kindly help? Thanks! > > Best Regards, > --------------------------------------------------------- > Steven Zheng > E-mail: [email protected] >
