Hello
We are using Apache Ignite 2.7.5 and we encountered the following in our
ignite. After this error occurred, it seems that the cache have been
corrupted and when we try to access this cache, the same errors occurred. We
also encountered this error with Apache Ignite 2.7.0.
The errors seems to occurred for random cache as well. Sometimes it will
happen in QA_Account cache but another times it happened for QA_Order cache,
etc. The only common denominator at the moment are the issues occurred for
the caches in the same data region and there is a schedule job at the
backend which operation involved iterate all records in the cache. (The job
run every 1 minutes). We do not have any idea what caused this as the error
seems to occur at random time for random cache and this is the only error
messages found in the ignite log.
Please kindly help. Thanks.
[15:01:53,699][SEVERE][query-#127][GridCacheDistributedQueryManager]
<QA_Account> Failed to run query [qry=GridCacheQueryInfo [loc=false,
trans=null, rdc=null, qry=GridCacheQueryAdapter [type=SCAN, clsName=null,
clause=null, filter=null, transform=null, part=null, incMeta=false,
metrics=null, pageSize=1024, timeout=0, incBackups=false, forceLocal=false,
dedup=false, prj=null, keepBinary=false,
subjId=c52f26d5-48c6-4b48-9566-41accfe995ff, taskHash=0, mvccSnapshot=null],
locFut=null, sndId=c52f26d5-48c6-4b48-9566-41accfe995ff, reqId=97142,
incMeta=false, all=false], node=b79e29cd-80ad-4191-9001-5a1454b5e207]
class org.apache.ignite.IgniteException: Runtime failure on bounds:
[lower=SearchRow [key=null, hash=0, cacheId=-703495123], upper=SearchRow
[key=null, hash=0, cacheId=-703495123]]
at
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:48)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.advance(GridCacheQueryManager.java:2996)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.onHasNext(GridCacheQueryManager.java:2965)
at
org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
at
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1266)
at
org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:234)
at
org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109)
at
org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
Runtime failure on bounds: [lower=SearchRow [key=null, hash=0,
cacheId=-703495123], upper=SearchRow [key=null, hash=0, cacheId=-703495123]]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1013)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:985)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2919)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2890)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2884)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2872)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.cursor(GridCacheOffheapManager.java:2075)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$6.onHasNext(IgniteCacheOffheapManagerImpl.java:968)
at
org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
at
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
... 21 more
Caused by: java.lang.IllegalArgumentException: Invalid object type: 107
at
org.apache.ignite.internal.processors.cacheobject.IgniteCacheObjectProcessorImpl.toKeyCacheObject(IgniteCacheObjectProcessorImpl.java:166)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toKeyCacheObject(CacheObjectBinaryProcessorImpl.java:865)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFullRow(CacheDataRowAdapter.java:333)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:167)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:108)
at
org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:92)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:200)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:49)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5512)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5280)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5205)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetCursor.notFound(BPlusTree.java:2899)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown(BPlusTree.java:1309)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doFind(BPlusTree.java:1276)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8700(BPlusTree.java:90)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.find(BPlusTree.java:5289)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1005)
... 30 more
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
