Failing client node due to not receiving metrics updates-IGNITE-10354

Hi, There is a cluster of four server nodes and six client nodes in production. I was using ignite 2.6.0 version and all six client nodes were failing with below error WARN o.a.i.s.d.tcp.TcpDiscoverySpi - Failing client node due to not receiving metrics updates from client node within 'IgniteConfi

Re: Failing client node due to not receiving metrics updates-IGNITE-10354

Hi Zhenya, Thanks for the quick response. I am checking with the network team as well for any network glitch occurrence. Thanks, Akash On Thu, Jun 10, 2021 at 1:06 PM Zhenya Stanilovsky wrote: > Hello Akash ! > I found that fix mentioned by you is ok. > Why do you think that your network betwee

Failed to perform cache operation (cache is stopped)

Hi, I have four server nodes and six client nodes on ignite cluster. I am using ignite 2.10 version. Some operations are failing due to the CacheStoppedException exception on the server nodes. This has become a blocker issue. Could someone please help me to resolve this issue. *Cache Configuration

Re: Failed to perform cache operation (cache is stopped)

Hi Zhenya, Thanks for the quick response. I believe you are talking about ignite instances. There is single ignite using in application. I also want to point out that I am not using destroyCache() method anywhere in application. I will set IGNITE_QUIET = false and try to grep the required logs.

Re: Failed to perform cache operation (cache is stopped)

Meanwhile I grep the logs with the next occurrence of cache stopped exception,can someone highlight if there is any known bug related to this? I want to check the possible reason for this cache stop exception. On Mon, Sep 6, 2021 at 6:27 PM Akash Shinde wrote: > Hi Zhenya, > Thanks f

Re: Re[2]: Failed to perform cache operation (cache is stopped)

you can >> send it directly to arzamas...@mail.ru >> >> >> Meanwhile I grep the logs with the next occurrence of cache stopped >> exception,can someone highlight if there is any known bug related to this? >> I want to check the possible reason for this cache stop

Re: Re[4]: Failed to perform cache operation (cache is stopped)

e: > > plz share somehow these logs, if you have no ideas how to share, you can > send it directly to arzamas...@mail.ru > > > > > Meanwhile I grep the logs with the next occurrence of cache stopped > exception,can someone highlight if there is any known bug related to th

Re: Re[4]: Failed to perform cache operation (cache is stopped)

Could someone please help me out here to find out the root cause of this problem? This is now happening so frequently. On Wed, Oct 13, 2021 at 3:56 PM Akash Shinde wrote: > Yes, I have set failureDetectionTimeout = 6. > There is no long GC pause. > Core1 GC report > <https

Re: Server Nodes Stopped Unexpectedly

Can someone please help me on this issue ? On Wed, Jul 24, 2019 at 12:04 PM Akash Shinde wrote: > Hi, > Please find attached logs from all server and client nodes.Also attached > gc logs for each node. > > Thanks, > Akash > > > On Tue, Jul 23, 2019 at 8:59 PM Evgeni

Re: Server Nodes Stopped Unexpectedly

at 4:00 PM Evgenii Zhuravlev wrote: > I don't see any specific errors in the logs. For me, it looks like network > problems, moreover, on client nodes it prints messages about connection > problems. Is this issue reproducible? > Evgenii > > пт, 26 июл. 2019 г. в 09:21,

Re: Server Nodes Stopped Unexpectedly

omething like a temporary network > problem. I would recommend to add some network monitoring to be prepared > for the next failure. > > Best Regards, > Evgenii > > пт, 26 июл. 2019 г. в 16:01, Akash Shinde : > >> This issue is not consistent and but occurs sometimes. Doe

Re: Server Nodes Stopped Unexpectedly

ssible > reason of the issue. > > Thanks, > Evgenii > > ср, 28 авг. 2019 г. в 17:56, Akash Shinde : > >> Hi, >> >> Now I have set the failure detection timeout to 12 mills and I am >> still getting this error message intermittently on Ignite 2.6 version. &g

Handling Of Partition loss

Hi, I am trying to recover lost data in case of partition loss. In my ignite configuration native persistence is *off*. I have started event listener on EVT_CACHE_REBALANCE_PART_DATA_LOST event. This listener will get lost partition list using cache.lostPartitions() method. The issue is that liste

Re: Handling Of Partition loss

Can someone please help me on this? Thanks, Akash On Tue, Sep 17, 2019 at 10:40 AM Akash Shinde wrote: > Hi, > I am trying to recover lost data in case of partition loss. > In my ignite configuration native persistence is *off*. > I have started event

Re: Handling Of Partition loss

apache.ignite.IgniteCache#lostPartitions [1] > > 2) I doubt that it is possible since resetLostPartitions method > accepts only cache names. > > [1] > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#lostPartitions-- > > On Tue, 17 Sep 2019 at 16:14

Partition loss due to node segmentation

Hi, Are there any possibilities of partition loss in case of node segmentation? Thanks, Akash

Re: Partition loss due to node segmentation

kash, > > Yes of course. As cluster is segmented, it is very well possible that > cluster segment doesn't have all partitions in that segment. > If you have backup partitions in the same segment, then only you have > complete partitions. > > Best Regards, > Gaurav >

Read through not working as expected in case of Replicated cache

I am using Ignite 2.6 version. I am starting 3 server nodes with a replicated cache and 1 client node. Cache configuration is as follows. Read-through true on but write-through is false. Load data by key is implemented as given below in cache-loader. Steps to reproduce issue: 1) Delete an entry f

Re: Read through not working as expected in case of Replicated cache

, Oct 29, 2019 at 8:53 PM Ilya Kasnacheev wrote: > Hello! > > I remember that we had this issue. Have you tried 2.7.6 yet? > > Regards, > -- > Ilya Kasnacheev > > > вт, 29 окт. 2019 г. в 18:18, Akash Shinde : > >> I am using Ignite 2.6 version. >> >

resetLostPartitions is blocked inside event listener

Hi, I am trying to handle lost partition scenario. I have written event listener listening to EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST event. I want to reset lost partition state of cache after cache loading is done. *Issue:* ignite.resetLostPartitions(caheName) is getting blocked and not com

Re: resetLostPartitions is blocked inside event listener

;> Thanks, >> Prasad >> >> On Thu 7 Nov, 2019, 9:28 PM Ilya Kasnacheev > wrote: >> >>> Hello! >>> >>> It is not advisable to call any blocking methods from event listeners. >>> Just fire resetLostPartitions from another thread. >>

Primary Entry Count are not correct in cache

Hi I have created a cache with following configuration. I started four nodes,each node on different machines. I have loaded this cache with loader. Issue: I am not performing any operation on this cache but I am able to see the primary key count not constant. Its keep on changing after some time. I

How to identify if re-balancing completed

Hi, If there are multiple nodes in cluster and if one of the node/nodes goes down or a new node/nodes is added how to make sure that re-balancing of all cache partitions has been completed successfully? I need this to implement rolling restart. Thanks, Aksah

Re: Local node terminated after segmentation

Hi, Can you please explain on high level how GridGain implementations protects from having two segments that are alive at the same time which could lead to data inconsistency over time? What exactly does it do to achieve this? Regards, A. On Wed, Dec 11, 2019 at 5:48 PM Stanislav Lukyanov wrot

Re: Local node terminated after segmentation

Can someone please help me on this? On Thu, Dec 12, 2019 at 1:11 PM Akash Shinde wrote: > Hi, > > Can you please explain on high level how GridGain implementations protects > from having two segments that are alive at the same time which could lead > to data inconsistency

cache.containsKey returns false

Hi, I am using Ignite 2.6 version. I have partitioned cache, read-through and write-through is enabled. Back-up count is 1 and total number of server nodes in cluster are 3. When I try to get the data from a cache for a key using cache.get(key) method, ignite reads the value from database using

Reloading of cache not working

Hi, I am using ignite 2.6. I am trying refresh few caches which are already loaded during server startup using cache loaders. To refresh caches I am invoking IgniteCache.loadCache method. But it seems that it is not updating data in caches. 1) Is it the expected behavior? 2) Do I have to clear the

Re: Reloading of cache not working

Hi, My question is specifically for clo.apply(key, data) that I invoked in CacheStoreAdapter.loadCache method. So, does this method (clo.apply) overrides value for the keys which are already present in cache or it just skips? My observation is that its not overriding value for the keys which are a

Test Mail

Countdown latch issue with 2.6.0

Hi, Issue: Countdown latched gets reinitialize to original value(4) when one or more (but not all) node goes down. (Partition loss happened) We are using ignite's distributed countdownlatch to make sure that cache loading is completed on all server nodes. We do this to make sure that our kafka con

CountDownLatch issue in Ignite 2.6 version

*Issue:* Countdown latched gets reinitialize to original value(4) when one or more (but not all) node goes down. *(Partition loss happened)* We are using ignite's distributed countdownlatch to make sure that cache loading is completed on all server nodes. We do this to make sure that our kafka con

Re: Countdown latch issue with 2.6.0

Can someone please help me with this issue. On Sat, Jun 6, 2020 at 6:45 PM Akash Shinde wrote: > Hi, > Issue: Countdown latched gets reinitialize to original value(4) when one > or more (but not all) node goes down. (Partition loss happened) > > We are using ignite's distri

Re: Countdown latch issue with 2.6.0

Hi, I have created jira for this issue https://issues.apache.org/jira/browse/IGNITE-13132 Thanks, Akash On Sun, Jun 7, 2020 at 9:29 AM Akash Shinde wrote: > Can someone please help me with this issue. > > On Sat, Jun 6, 2020 at 6:45 PM Akash Shinde wrote: > >> Hi, >>

Exception while running sql inside ignite transaction

Hi, I am facing below exception while executing query on cache. This exception is not consistent. I am starting transaction in optimistic mode. Could someone please advice. try (Transaction transaction = igniteTx .txStart(TransactionConcurrency.OPTIMISTIC, TransactionIsolation.SERIALIZABLE)) {

Re: ***UNCHECKED*** Executing SQL on cache using affinnity key

Hi Evgenii, Are you saying that if the affinityKey column is present in WHERE clause of SQL statement, the SQL engine will detect a node where this key is placed and execute this query only on one node. I just want to make sure that SQL gets executed only on node where the affinity key is placed.

Partitions distribution across nodes

Hi , I am loading cache in partition aware mode.I have started four nodes. Out of these four node two nodes are loading only backup partitions and other two nodes are loading only primary partitions. As per my understanding each node should have backup and primary partition both. But in my cluste

Re: Partitions distribution across nodes

Hi, I introduced the delay of 5 seconds and it worked. 1) What is exchange process and how to identify whether exchange process is finished? 2) I am doing partition key aware data loading and I want to start load process from server node only and not from the client node. I just want to initiate

Re: Query execution too long even after providing index

Hello guys, I am also facing the similar problem. Does community users have any solution for this? This has become blocking issue for me. Can someone please help? Thanks, Akash On Mon, Sep 10, 2018 at 8:33 AM Prasad Bhalerao < prasadbhalerao1...@gmail.com> wrote: > Guys, is there any solution

Cache Metrics

Hi, I have captured below two ignite cache metrics. 1) igniteCache.metrics().getSize() 2)igniteCache .metrics().getOffHeapEntriesCount() I started three nodes with distrusted load. When I started filling the cache I observed OffHeapEntries counts are approximately double of cache size. Why off

Error in write-through

Hi, I have started four ignite nodes and configured cache in distributed mode. When I initiated thousands of requests to write the data on this cache(write through enabled) , facing below error. >From logs we can see this error is occurring while witting to oracle database.(using cache write-throu

After upgrading 2.7 getting Unexpected error occurred during unmarshalling

Hi, When execute affinity.partition(key), I am getting following exception on Ignite 2.7. Stacktrace: 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR o.a.i.i.p.task.GridTaskWorker - Error deserializing job response: GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2

Re: nodes getting disconnected from cluster

Hi, Someone could please help me on this issue. Thanks, Akash On Thu, Jan 3, 2019 at 5:46 PM Akash Shinde wrote: > Hi, > > I am getting " Timed out waiting for message delivery receipt" WARN > message in my logs. > But I am sure that it is not happening because of long

Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling

halling of an instance of the class: org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. This is working fine on Ignite 2.6 version but giving problem on 2.7. Is this a bug or am I doing something wrong? Can someone please help? On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde wr

Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling

; > Mikael > Den 2019-01-08 kl. 12:21, skrev Akash Shinde: > > Before submitting the affinity task ignite first gets the affinity cached > function (AffinityInfo) by submitting the cluster wide task "AffinityJob". > But while in the process of retrieving the output of th

Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling

Added d...@ignite.apache.org. Should I log Jira for this issue? Thanks, Akash On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde wrote: > Hi, > > No both nodes, client and server are running on Ignite 2.7 version. I am > starting both server and client from Intellij IDE. > >

JTA transactions in Iginte

Hello, My requirement is to achieve ignite cache update operation and database write operation in single transaction. Right now I am trying to use JTA transaction management. I have created below IgniteConfiguration using transactionConfiguration. But I am not sure how to configure XAResourc

Off heap and JVM heap memory allocation

Hi, I have 32 GB RAM machine to configure IGNITE node. Out of this 32 GB memory I want to allocate 16 GB for *off heap* space and remaining 16 GB for *JVM heap* space(processing). My cache will be configured as only off heap and JVM heap space will be use for processing. Is there any way to do this

code hangs up on cache().removeAll(set) operation

Hi, My application hangs up when I execute following code. I tried to debug the ignite source code but no luck. Complete thread dump is attached in this mail. *cache().removeAll(set)* * Can someone please advise? * *Thread dump is as follows.* Name: local-task-pool-0 State: WAITING Total blo