Hi Kasnacheev, Thank you very much for the response.. We use v1.9, because 2.x uses mandatory ignite native persistence(local disk) along with durable memory management(RAM), and no option for a **java heap only** cache storage. We wanted to keep away from storing anything in disk. Hence using 1.9 Ignite. Can you please clarify “I believe that cache in question is no longer consistent between nodes on metadata level” ?
1. Like what could have caused & under what conditions this inconsistent state between nodes on cache metadata occurred ? 2. And does 2.x fixes the problem?( Is there a way you can suggest to reproduce this issue, so that we can know for sure 2.x fixes the problem) It appeared from our logs that the cache.put threads hangs forever after this error. Can that be possible with this? From: Ilya Kasnacheev [mailto:[email protected]] Sent: Wednesday, September 26, 2018 12:31 PM To: [email protected] Subject: Re: java.lang.NullPointerException in GridDhtPartitionsExchangeFuture This email is from an external source - exercise caution regarding links and attachments. Hello! This is a bad error message. I believe that cache in question is no longer consistent between nodes on metadata level, and you can't fix that without full restart or at least dropping and recreating the cache. I don't think that you will get much support from community on 1.x since the focus has shifted to 2.x. Have you considered upgrading? Regards, -- Ilya Kasnacheev ср, 26 сент. 2018 г. в 18:50, HEWA WIDANA GAMAGE, SUBASH <[email protected]<mailto:[email protected]>>: This is the only single error from ignite happened after JVM startup. Looks like I only posted the exception stack trace. Here’s the message and from which thread it logged. level: ERROR logger: org.apache.ignite.internal.processors.cache.GridCacheIoManager message: Failed processing message [senderId=57ee6544-e0b3-45cc-bdb5-f3fd37d7db1e, msg=GridDhtPartitionsSingleMessage [parts=null, partCntrs=null, client=false, compress=false, super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], nodeId=57ee6544, evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, time=0, order=1536996016410, nodeOrder=0], flags=0, super=GridCacheMessage [msgId=1, depInfo=null, err=null, skipPrepare=false, cacheId=0, cacheId=0]]]] [IM_GROUP=fsy-pi-dt-ssam ] thread: sys-stripe-2-#3%null% From: Ilya Kasnacheev [mailto:[email protected]<mailto:[email protected]>] Sent: Tuesday, September 25, 2018 11:11 AM To: [email protected]<mailto:[email protected]> Subject: Re: java.lang.NullPointerException in GridDhtPartitionsExchangeFuture This email is from an external source - exercise caution regarding links and attachments. Hello! It's hard to say without reviewing logs, but it seems that there's some inconsistency with regards to cache metadata on nodes. Regards, -- Ilya Kasnacheev вт, 25 сент. 2018 г. в 0:13, HEWA WIDANA GAMAGE, SUBASH <[email protected]<mailto:[email protected]>>: Hi all, We use Ignite 1.9. We could see this in our logs. All we do is cache.get() , cache.put() operations. With this log being seen, is it possible for cache.put or ignite.getOrCreateCache() method calling threads be blocked forever ? (unfortunately we couldn’t get a thread dump to prove that, but from application logs, it looks like it). java.lang.NullPointerException: null at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.updatePartitionSingleMap(GridDhtPartitionsExchangeFuture.java:1446) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processMessage(GridDhtPartitionsExchangeFuture.java:1199) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$100(GridDhtPartitionsExchangeFuture.java:86) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$1.apply(GridDhtPartitionsExchangeFuture.java:1167) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$1.apply(GridDhtPartitionsExchangeFuture.java:1155) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271) at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:228) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceive(GridDhtPartitionsExchangeFuture.java:1155) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(GridCachePartitionExchangeManager.java:1304) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1200(GridCachePartitionExchangeManager.java:116) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:310) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:308) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:1992) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:1974) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:827) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:369) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:293) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:95) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:238) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1222) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:850) at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108) at org.apache.ignite.internal.managers.communication.GridIoManager$7.run(GridIoManager.java:790) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:428) at java.lang.Thread.run(Thread.java:748)
