Aaron, Could you provide steps to reproduce the issue? Can this issue be reproduced when there is only one cache and one data region? Or when all caches are in a single data region.
Also please clarify, how you modify caches and what queried do you execute? Denis вт, 2 янв. 2018 г. в 16:26, [email protected] <[email protected]>: > This only happen when I shut down the nodes with this persist-able cache. > > After bring those nodes back, either throw this exception, or the node > hang up forever waiting the initial partition map. > > [WARN ] 2018-01-02 13:19:08.318 [main] [ig] > GridCachePartitionExchangeManager - Still waiting for initial partition map > exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent > [evtNode=TcpDiscoveryNode [id=bfb39f71-fa1d-4547-9b49-b59aed9ecbef, > addrs=[10.31.23.18], sockAddrs=[iZuf62zdiq684kn72aatgjZ/10.31.23.18:47502], > discPort=47502, order=37, intOrder=23, lastExchangeTime=1514899147831, > loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=37, > nodeId8=bfb39f71, msg=null, type=NODE_JOINED, tstamp=1514898728282], > crd=TcpDiscoveryNode [id=237a7983-a2d6-4f39-9f39-75cd8b17299c, > addrs=[10.26.244.207], sockAddrs=[/10.26.244.207:47500], discPort=47500, > order=1, intOrder=1, lastExchangeTime=1514898728181, loc=false, > ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], > exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion > [topVer=37, minorTopVer=0], discoEvt=DiscoveryEvent > [evtNode=TcpDiscoveryNode [id=bfb39f71-fa1d-4547-9b49-b59aed9ecbef, > addrs=[10.31.23.18], sockAddrs=[iZuf62zdiq684kn72aatgjZ/10.31.23.18:47502], > discPort=47502, order=37, intOrder=23, lastExchangeTime=1514899147831, > loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=37, > nodeId8=bfb39f71, msg=null, type=NODE_JOINED, tstamp=1514898728282], > nodeId=bfb39f71, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter > [ignoreInterrupts=false, state=DONE, res=true, hash=993782641], init=true, > lastVer=null, partReleaseFut=PartitionReleaseFuture > [topVer=AffinityTopologyVersion [topVer=37, minorTopVer=0], > futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion > [topVer=37, minorTopVer=0], futures=[]], TxReleaseFuture > [topVer=AffinityTopologyVersion [topVer=37, minorTopVer=0], futures=[]], > AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=37, > minorTopVer=0], futures=[]], DataStreamerReleaseFuture > [topVer=AffinityTopologyVersion [topVer=37, minorTopVer=0], futures=[]]]], > exchActions=null, affChangeMsg=null, initTs=1514898728302, > centralizedAff=false, changeGlobalStateE=null, done=false, state=SRV, > evtLatch=0, remaining=[a364148f-c04f-487e-9935-5298662aa7cc, > e7c123fd-5da8-45ca-a04a-65bafe9ed6c6, e417828d-793f-4f24-bf80-7945321d8f51, > 237a7983-a2d6-4f39-9f39-75cd8b17299c, bed139ea-9ea9-42e4-b296-79ed51477ad0, > d3890e42-6003-4d33-a5f2-52c7c4547f27, > 9ece2c56-ced5-4710-be57-94d6bd529a90], > > > > Previous when we use the DB as backend storage never meet this issue. > attach are those two configuration. > > We have do all the thing, like active the ignite before use it etc. > > > > > > ------------------------------ > [email protected] > > > *From:* [email protected] > *Date:* 2018-01-02 17:30 > *To:* user <[email protected]> > *Subject:* Re: Re: Some question regards the DataStorageConfiguration > when used in multiple data regions. > > Hi Denis, > > Thanks for so quick response the NPE as: > > Caused by: java.lang.NullPointerException > > at > org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.metadata(CacheObjectBinaryProcessorImpl.java:538) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.metadata(CacheObjectBinaryProcessorImpl.java:194) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryContext.metadata(BinaryContext.java:1266) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryReaderExImpl.getOrCreateSchema(BinaryReaderExImpl.java:2005) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryReaderExImpl.<init>(BinaryReaderExImpl.java:284) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryReaderExImpl.<init>(BinaryReaderExImpl.java:183) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryObjectImpl.reader(BinaryObjectImpl.java:830) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryObjectImpl.reader(BinaryObjectImpl.java:845) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.binary.BinaryObjectImpl.field(BinaryObjectImpl.java:308) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.fieldValue(QueryBinaryProperty.java:245) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:139) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.h2.H2RowDescriptor.columnValue(H2RowDescriptor.java:303) > ~[ignite-indexing-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.h2.opt.GridH2KeyValueRowOnheap.getValue(GridH2KeyValueRowOnheap.java:160) > ~[ignite-indexing-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:212) > ~[ignite-indexing-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:40) > ~[ignite-indexing-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:4359) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findInsertionPoint(BPlusTree.java:4279) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$1500(BPlusTree.java:81) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:261) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:4697) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:4682) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:158) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:319) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown(BPlusTree.java:1132) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doFind(BPlusTree.java:1101) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$15800(BPlusTree.java:81) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.find(BPlusTree.java:4597) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.access$5400(BPlusTree.java:4391) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:957) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:938) > ~[ignite-core-2.3.0.jar!/:2.3.0] > > at > org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.find(H2TreeIndex.java:181) > ~[ignite-indexing-2.3.0.jar!/:2.3.0] > > at org.h2.index.BaseIndex.find(BaseIndex.java:128) ~[h2-1.4.195.jar!/:?] > > at org.h2.index.IndexCursor.find(IndexCursor.java:169) > ~[h2-1.4.195.jar!/:?] > > at org.h2.table.TableFilter.next(TableFilter.java:468) > ~[h2-1.4.195.jar!/:?] > > at > org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow(Select.java:1452) > ~[h2-1.4.195.jar!/:?] > > at org.h2.result.LazyResult.hasNext(LazyResult.java:79) > ~[h2-1.4.195.jar!/:?] > > at org.h2.result.LazyResult.next(LazyResult.java:59) ~[h2-1.4.195.jar!/:?] > > at org.h2.command.dml.Select.queryFlat(Select.java:519) > ~[h2-1.4.195.jar!/:?] > > at org.h2.command.dml.Select.queryWithoutCache(Select.java:625) > ~[h2-1.4.195.jar!/:?] > > at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(Query.java:114) > ~[h2-1.4.195.jar!/:?] > > at org.h2.command.dml.Query.query(Query.java:352) ~[h2-1.4.195.jar!/:?] > > at org.h2.command.dml.Query.query(Query.java:333) ~[h2-1.4.195.jar!/:?] > > at org.h2.command.CommandContainer.query(CommandContainer.java:113) > ~[h2-1.4.195.jar!/:?] > > at org.h2.command.Command.executeQuery(Command.java:201) > ~[h2-1.4.195.jar!/:?] > > at > org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:111) > ~[h2-1.4.195.jar!/:?] > > at > org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(IgniteH2Indexing.java:961) > ~[ignite-indexing-2.3.0.jar!/:2.3.0] > > > > Regards > Aaron > ------------------------------ > [email protected] > > > *From:* Denis Mekhanikov <[email protected]> > *Date:* 2018-01-02 16:28 > *To:* user <[email protected]> > *Subject:* Re: Some question regards the DataStorageConfiguration when > used in multiple data regions. > Hi Aaron! > > Looks like warnings are printed due to a bug. I created a ticket for it: > https://issues.apache.org/jira/browse/IGNITE-7347 > This bug doesn't affect anything, so you can just ignore the warnings. > > You've cut out the most interesting part of the stack trace - place, where > NPE occurred. Please provide the full stack trace. > > So, this exception appears, when you update a cache from one data region, > and query a cache from another? It doesn't sound plausible :) Please > provide a reproducer, if it's possible. Or provide more details about your > configuration, query entities and a query. > > Denis > > вт, 2 янв. 2018 г. в 7:12, [email protected] <[email protected]>: > >> >> hi All, >> >> Current we are leveraging the DataStorageConfiguration to persist the >> cache to local file system. >> >> Can currently the persistence enable flag be set for separated data >> Regions? as from our running log. >> >> Even we only set the region A for persistenceEnabled but log said: >> >> [WARN ] 2018-01-02 03:27:43.234 [main] [ig] >> GridCacheDatabaseSharedManager - Page eviction mode for [B_Region] memory >> region is ignored because Ignite Native Persistence is enabled >> >> [WARN ] 2018-01-02 03:27:43.235 [main] [ig] >> GridCacheDatabaseSharedManager - Page eviction mode for [C_Region] memory >> region is ignored because Ignite Native Persistence is enabled >> >> B/C_Region used because some cache refer to those regions. >> >> Now the problem is if any time we update the data structure in the Other >> caches in B/C_Region, this cache in the Region A can not be query any more! >> >> >> [ERROR] 2018-01-02 03:50:06.328 [query-#202%BookerFX%] [ig] >> GridMapQueryExecutor - Failed to execute local query. >> >> org.apache.ignite.IgniteCheckedException: Failed to execute SQL query. >> >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(IgniteH2Indexing.java:968) >> ~[ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer(IgniteH2Indexing.java:1027) >> ~[ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer(IgniteH2Indexing.java:1006) >> ~[ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest0(GridMapQueryExecutor.java:670) >> [ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest(GridMapQueryExecutor.java:516) >> [ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onMessage(GridMapQueryExecutor.java:214) >> [ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor$2.onMessage(GridMapQueryExecutor.java:174) >> [ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage(GridIoManager.java:2332) >> [ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555) >> [ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183) >> [ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126) >> [ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090) >> [ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> [?:1.8.0_131] >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> [?:1.8.0_131] >> >> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] >> >> Caused by: org.h2.jdbc.JdbcSQLException: General error: "class >> org.apache.ignite.IgniteException: Runtime failure on bounds: >> [lower=RowSimple [vals=[null, null, null, 'EURUSD', 0]], upper=RowSimple >> [vals=[null, null, null, 'EURUSD', 4096]]]"; SQL statement: >> >> SELECT >> >> "CacheExample".__Z0._KEY __C0_0, >> >> "CacheExample".__Z0._VAL __C0_1, >> >> __Z0.SEQUENCENUMBER __C0_2 >> >> FROM "BookerGenericDomainEventEntryFX".GENERICDOMAINEVENTENTRY __Z0 >> >> WHERE (__Z0.SEQUENCENUMBER < ?3) AND ((__Z0.AGGREGATEIDENTIFIER = ?1) AND >> (__Z0.SEQUENCENUMBER >= ?2)) >> >> ORDER BY 3 [50000-195] >> >> at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.message.DbException.get(DbException.java:168) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.message.DbException.convert(DbException.java:295) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.Command.executeQuery(Command.java:215) >> ~[h2-1.4.195.jar!/:?] >> >> at >> org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:111) >> ~[h2-1.4.195.jar!/:?] >> >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(IgniteH2Indexing.java:961) >> ~[ignite-indexing-2.3.0.jar!/:2.3.0] >> >> ... 14 more >> >> Caused by: org.apache.ignite.IgniteException: Runtime failure on bounds: >> [lower=RowSimple [vals=[null, null, null, 'EURUSD', 0]], upper=RowSimple >> [vals=[null, null, null, 'EURUSD', 4096]]] >> >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:965) >> ~[ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:938) >> ~[ignite-core-2.3.0.jar!/:2.3.0] >> >> at >> org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.find(H2TreeIndex.java:181) >> ~[ignite-indexing-2.3.0.jar!/:2.3.0] >> >> at org.h2.index.BaseIndex.find(BaseIndex.java:128) ~[h2-1.4.195.jar!/:?] >> >> at org.h2.index.IndexCursor.find(IndexCursor.java:169) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.table.TableFilter.next(TableFilter.java:468) >> ~[h2-1.4.195.jar!/:?] >> >> at >> org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow(Select.java:1452) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.result.LazyResult.hasNext(LazyResult.java:79) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.result.LazyResult.next(LazyResult.java:59) ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.dml.Select.queryFlat(Select.java:519) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.dml.Select.queryWithoutCache(Select.java:625) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(Query.java:114) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.dml.Query.query(Query.java:352) ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.dml.Query.query(Query.java:333) ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.CommandContainer.query(CommandContainer.java:113) >> ~[h2-1.4.195.jar!/:?] >> >> at org.h2.command.Command.executeQuery(Command.java:201) >> ~[h2-1.4.195.jar!/:?] >> >> at >> org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:111) >> ~[h2-1.4.195.jar!/:?] >> >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(IgniteH2Indexing.java:961) >> ~[ignite-indexing-2.3.0.jar!/:2.3.0] >> >> ... 14 more >> >> Caused by: java.lang.NullPointerException >> >> >> >> Cache configuration: >> >> <property name="storagePath" >> value="${persistent.store.base.path:/var/ignite/persistent}/${app.instance.cnt}/store"/> >> <property name="walArchivePath" >> value="${persistent.store.base.path:/var/ignite/persistent}/${app.instance.cnt}/store/wal/archive"/> >> <property name="walPath" >> value="${persistent.store.base.path:/var/ignite/persistent}/${app.instance.cnt}/store/wal"/> >> >> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> >> <property name="name" value="A_Region"/> >> <property name="initialSize" value="#{2L * 1024L * 1024L * 1024L}"/> >> <property name="maxSize" value="#{4L * 1024L * 1024L * 1024L}"/> >> <property name="pageEvictionMode" value="DISABLED"/> >> <property name="persistenceEnabled" value="true"/> >> </bean> >> >> >> >> Thanks for your time! >> >> >> >> Regards >> Aaron >> ------------------------------ >> [email protected] >> >
