Can you export logs from your system in a different format? I don't really
understand how it can be analyzed by a person in this format.
Evgenii
пн, 31 авг. 2020 г. в 00:36, Steven Zheng <closee...@gmail.com>:
> Thanks evgenii, but origin log is not been saved and I can only provide
> json format logs.
> Best Regards,
> ---------------------------------------------------------
> Steven Zheng
> E-mail: closee...@gmail.com
>
>
> Evgenii Zhuravlev <e.zhuravlev...@gmail.com> 于2020年8月28日周五 上午6:45写道：
>
>> Hi,
>>
>> Can you attach logs in normal format? It's really hard to read it. Also,
>> please attach full logs from nodes, not only the stacktrace.
>>
>> Thanks,
>> Evgenii
>>
>> вт, 25 авг. 2020 г. в 19:27, Steven Zheng <closee...@gmail.com>:
>>
>>> Hi community,
>>> Currently I have 25 nodes in my ignite cluster and all of them were
>>> added into the baseline, and I was trying to add another 5 nodes into it.
>>> The data in my cluster is about 8TB and the persistence is enabled.
>>> At first I start all the 5 nodes; then execute in the command line:
>>> ```
>>> bin/control.sh --baseline add ${my_node_id}
>>> ```
>>> Meanwhile, there is still read/write workloads on the cluster.
>>>
>>> After a few minutes, a critical error detected , several nodes crashed
>>> and emits the logs like this(json format):
>>> ```
>>> {
>>> "message": "Critical system error detected. Will be handled
>>> accordingly to configured handler [hnd=StopNodeFailureHandler
>>> [super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
>>> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
>>> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
>>> o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is
>>> corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=-1526563570,
>>> val2=844420635165881]], cacheId=-2021712086, cacheName=MY_SQL_TABLE,
>>> indexName=AFFINITY_KEY, msg=Runtime failure on search row: Row@d00bc1[
>>> key: SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf_KEY
>>> [idHash=2015873411, hash=-2113698004, CARDNUMBER=25323111322, IP=], val:
>>> SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf
>>> [idHash=1729397221, hash=2090272279, MY_BIZ_UID=0, ETL_TIME=1596023962011]
>>> ][ 25323111322, , 0, 1596023962011 ]]]]",
>>> "loggerFqcn": "org.apache.logging.log4j.spi.AbstractLogger",
>>> "thrown": {
>>> "localizedMessage": "B+Tree is corrupted [pages(groupId,
>>> pageId)=[IgniteBiTuple [val1=-1526563570, val2=844420635165881]],
>>> cacheId=-2021712086, cacheName=MY_SQL_TABLE, indexName=AFFINITY_KEY,
>>> msg=Runtime failure on search row: Row@d00bc1[ key:
>>> SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf_KEY
>>> [idHash=2015873411, hash=-2113698004, CARDNUMBER=25323111322, IP=], val:
>>> SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf
>>> [idHash=1729397221, hash=2090272279, MY_BIZ_UID=0, ETL_TIME=1596023962011]
>>> ][ 25323111322, , 0, 1596023962011 ]]",
>>> "message": "B+Tree is corrupted [pages(groupId,
>>> pageId)=[IgniteBiTuple [val1=-1526563570, val2=844420635165881]],
>>> cacheId=-2021712086, cacheName=MY_SQL_TABLE, indexName=AFFINITY_KEY,
>>> msg=Runtime failure on search row: Row@d00bc1[ key:
>>> SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf_KEY
>>> [idHash=2015873411, hash=-2113698004, CARDNUMBER=25323111322, IP=], val:
>>> SQL_FEAT_MY_SQL_TABLE_7f8539ef_08a1_421e_a0b8_60f80cdbbbdf
>>> [idHash=1729397221, hash=2090272279, MY_BIZ_UID=0, ETL_TIME=1596023962011]
>>> ][ 25323111322, , 0, 1596023962011 ]]",
>>> "commonElementCount": 0,
>>> "cause": {
>>> "localizedMessage": "java.lang.IllegalStateException: Item not
>>> found: 27",
>>> "message": "java.lang.IllegalStateException: Item not found: 27",
>>> "commonElementCount": 21,
>>> "cause": {
>>> "localizedMessage": "Item not found: 27",
>>> "message": "Item not found: 27",
>>> "commonElementCount": 21,
>>> "name": "java.lang.IllegalStateException",
>>> "extendedStackTrace": [{
>>> "line": 351,
>>> "method": "findIndirectItemIndex",
>>> "exact": false,
>>> "file": "AbstractDataPageIO.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 459,
>>> "method": "getDataOffset",
>>> "exact": false,
>>> "file": "AbstractDataPageIO.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 501,
>>> "method": "readPayload",
>>> "exact": false,
>>> "file": "AbstractDataPageIO.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 325,
>>> "method": "readIncomplete",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 261,
>>> "method": "doInitFromLink",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 160,
>>> "method": "initFromLink",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 131,
>>> "method": "initFromLink",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 277,
>>> "method": "createRow0",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 262,
>>> "method": "createRow",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 154,
>>> "method": "getLookupRow",
>>> "exact": false,
>>> "file": "AbstractH2ExtrasLeafIO.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 36,
>>> "method": "getLookupRow",
>>> "exact": false,
>>> "file": "AbstractH2ExtrasLeafIO.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 339,
>>> "method": "getRow",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 63,
>>> "method": "getRow",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5218,
>>> "method": "getRow",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 418,
>>> "method": "compare",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 63,
>>> "method": "compare",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5202,
>>> "method": "compare",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5121,
>>> "method": "findInsertionPoint",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 95,
>>> "method": "access$1100",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 299,
>>> "method": "run0",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5714,
>>> "method": "run",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 279,
>>> "method": "run",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5700,
>>> "method": "run",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 169,
>>> "method": "readPage",
>>> "exact": false,
>>> "file": "PageHandler.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 364,
>>> "method": "read",
>>> "exact": false,
>>> "file": "DataStructure.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.DataStructure",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5901,
>>> "method": "read",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2064,
>>> "method": "removeDown",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2083,
>>> "method": "removeDown",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2083,
>>> "method": "removeDown",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 1994,
>>> "method": "doRemove",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> }
>>> ]
>>> },
>>> "name":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTreeRuntimeException",
>>> "extendedStackTrace": [{
>>> "line": 290,
>>> "method": "doInitFromLink",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 160,
>>> "method": "initFromLink",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 131,
>>> "method": "initFromLink",
>>> "exact": false,
>>> "file": "CacheDataRowAdapter.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 277,
>>> "method": "createRow0",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 262,
>>> "method": "createRow",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 154,
>>> "method": "getLookupRow",
>>> "exact": false,
>>> "file": "AbstractH2ExtrasLeafIO.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 36,
>>> "method": "getLookupRow",
>>> "exact": false,
>>> "file": "AbstractH2ExtrasLeafIO.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 339,
>>> "method": "getRow",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 63,
>>> "method": "getRow",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5218,
>>> "method": "getRow",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 418,
>>> "method": "compare",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 63,
>>> "method": "compare",
>>> "exact": false,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5202,
>>> "method": "compare",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5121,
>>> "method": "findInsertionPoint",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 95,
>>> "method": "access$1100",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 299,
>>> "method": "run0",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5714,
>>> "method": "run",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 279,
>>> "method": "run",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5700,
>>> "method": "run",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 169,
>>> "method": "readPage",
>>> "exact": false,
>>> "file": "PageHandler.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 364,
>>> "method": "read",
>>> "exact": false,
>>> "file": "DataStructure.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.DataStructure",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 5901,
>>> "method": "read",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2064,
>>> "method": "removeDown",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2083,
>>> "method": "removeDown",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2083,
>>> "method": "removeDown",
>>> "exact": false,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 1994,
>>> "method": "doRemove",
>>> "exact": true,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> }
>>> ]
>>> },
>>> "name":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException",
>>> "extendedStackTrace": [{
>>> "line": 678,
>>> "method": "corruptedTreeException",
>>> "exact": true,
>>> "file": "H2Tree.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2Tree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2029,
>>> "method": "doRemove",
>>> "exact": true,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 1825,
>>> "method": "removex",
>>> "exact": true,
>>> "file": "BPlusTree.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 453,
>>> "method": "removex",
>>> "exact": true,
>>> "file": "H2TreeIndex.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 805,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "GridH2Table.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.opt.GridH2Table",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 404,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "IgniteH2Indexing.java",
>>> "location": "ignite-indexing-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2828,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "GridQueryProcessor.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.query.GridQueryProcessor",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 440,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "GridCacheQueryManager.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2695,
>>> "method": "finishRemove",
>>> "exact": true,
>>> "file": "IgniteCacheOffheapManagerImpl.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2672,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "IgniteCacheOffheapManagerImpl.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 2439,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "GridCacheOffheapManager.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 615,
>>> "method": "remove",
>>> "exact": true,
>>> "file": "IgniteCacheOffheapManagerImpl.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 4403,
>>> "method": "removeValue",
>>> "exact": true,
>>> "file": "GridCacheMapEntry.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.GridCacheMapEntry",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 678,
>>> "method": "clearInternal",
>>> "exact": true,
>>> "file": "GridDhtCacheEntry.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 1175,
>>> "method": "clearAll",
>>> "exact": true,
>>> "file": "GridDhtLocalPartition.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 938,
>>> "method": "tryClear",
>>> "exact": true,
>>> "file": "GridDhtLocalPartition.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 417,
>>> "method": "run",
>>> "exact": true,
>>> "file": "PartitionsEvictManager.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$PartitionEvictionTask",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 7061,
>>> "method": "wrapThreadLoader",
>>> "exact": true,
>>> "file": "IgniteUtils.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class": "org.apache.ignite.internal.util.IgniteUtils",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 827,
>>> "method": "body",
>>> "exact": true,
>>> "file": "GridClosureProcessor.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class":
>>> "org.apache.ignite.internal.processors.closure.GridClosureProcessor$1",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 120,
>>> "method": "run",
>>> "exact": true,
>>> "file": "GridWorker.java",
>>> "location": "ignite-core-2.8.0-SNAPSHOT.jar",
>>> "class": "org.apache.ignite.internal.util.worker.GridWorker",
>>> "version": "2.8.0-SNAPSHOT"
>>> },
>>> {
>>> "line": 1149,
>>> "method": "runWorker",
>>> "exact": true,
>>> "file": "ThreadPoolExecutor.java",
>>> "location": "?",
>>> "class": "java.util.concurrent.ThreadPoolExecutor",
>>> "version": "1.8.0_221"
>>> },
>>> {
>>> "line": 624,
>>> "method": "run",
>>> "exact": true,
>>> "file": "ThreadPoolExecutor.java",
>>> "location": "?",
>>> "class": "java.util.concurrent.ThreadPoolExecutor$Worker",
>>> "version": "1.8.0_221"
>>> },
>>> {
>>> "line": 748,
>>> "method": "run",
>>> "exact": true,
>>> "file": "Thread.java",
>>> "location": "?",
>>> "class": "java.lang.Thread",
>>> "version": "1.8.0_221"
>>> }
>>> ]
>>> },
>>> "level": "ERROR"
>>> }
>>> ```
>>>
>>> Can anyone kindly help? Thanks!
>>>
>>> Best Regards,
>>> ---------------------------------------------------------
>>> Steven Zheng
>>> E-mail: closee...@gmail.com
>>>
>>
