Re: CountDownLatch issue in Ignite 2.6 version
Prasad, Please don't use the dev list for the questions regarding the product usage, dev list used for development-related activities. To see how this configuration used for countDownLatch you can take a look at these 2 methods: https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/DataStructuresProcessor.java#L1187 https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/DataStructuresProcessor.java#L495 Evgenii пн, 8 июн. 2020 г. в 20:43, Prasad Bhalerao : > I just checked the ignite doc for atomic configuration. > But it doesn't say that it is applicable to distributed data structures. > > Is it really applicable to distributed data structures like count down > latch > > On Tue 9 Jun, 2020, 7:26 AM Prasad Bhalerao wrote: > >> Hi, >> I was under the impression that countdown latch is implemented in >> replicated cache. So when any number of nodes go down it does not loose >> it's state. >> >> Can you please explain why atmoc data structures are using 1 back when >> its state is very important? >> >> Can we enforce atomic data structures to use replicated cache? >> >> Which cache does ignite use to store atomic data structures? >> >> Thanks >> Prasad >> >> On Mon 8 Jun, 2020, 11:58 PM Evgenii Zhuravlev > wrote: >> >>> Hi, >>> >>> By default, cache, that stores all atomic structures has only 1 backup, >>> so, after losing all data for this certain latch, it recreates it. To >>> change the default atomic configuration use >>> IgniteConfiguration.setAtomicConfiguration. >>> >>> Evgenii >>> >>> сб, 6 июн. 2020 г. в 06:20, Akash Shinde : >>> *Issue:* Countdown latched gets reinitialize to original value(4) when one or more (but not all) node goes down. *(Partition loss happened)* We are using ignite's distributed countdownlatch to make sure that cache loading is completed on all server nodes. We do this to make sure that our kafka consumers starts only after cache loading is complete on all server nodes. This is the basic criteria which needs to be fulfilled before starts actual processing We have 4 server nodes and countdownlatch is initialized to 4. We use cache.loadCache method to start the cache loading. When each server completes cache loading it reduces the count by 1 using countDown method. So when all the nodes completes cache loading, the count reaches to zero. When this count reaches to zero we start kafka consumers on all server nodes. But we saw weird behavior in prod env. The 3 server nodes were shut down at the same time. But 1 node is still alive. When this happened the count down was reinitialized to original value i.e. 4. But I am not able to reproduce this in dev env. Is this a bug, when one or more (but not all) nodes goes down then count re initializes back to original value? Thanks, Akash >>>
Re: CountDownLatch issue in Ignite 2.6 version
I just checked the ignite doc for atomic configuration. But it doesn't say that it is applicable to distributed data structures. Is it really applicable to distributed data structures like count down latch On Tue 9 Jun, 2020, 7:26 AM Prasad Bhalerao Hi, > I was under the impression that countdown latch is implemented in > replicated cache. So when any number of nodes go down it does not loose > it's state. > > Can you please explain why atmoc data structures are using 1 back when its > state is very important? > > Can we enforce atomic data structures to use replicated cache? > > Which cache does ignite use to store atomic data structures? > > Thanks > Prasad > > On Mon 8 Jun, 2020, 11:58 PM Evgenii Zhuravlev wrote: > >> Hi, >> >> By default, cache, that stores all atomic structures has only 1 backup, >> so, after losing all data for this certain latch, it recreates it. To >> change the default atomic configuration use >> IgniteConfiguration.setAtomicConfiguration. >> >> Evgenii >> >> сб, 6 июн. 2020 г. в 06:20, Akash Shinde : >> >>> *Issue:* Countdown latched gets reinitialize to original value(4) when >>> one or more (but not all) node goes down. *(Partition loss happened)* >>> >>> We are using ignite's distributed countdownlatch to make sure that cache >>> loading is completed on all server nodes. We do this to make sure that our >>> kafka consumers starts only after cache loading is complete on all server >>> nodes. This is the basic criteria which needs to be fulfilled before starts >>> actual processing >>> >>> >>> We have 4 server nodes and countdownlatch is initialized to 4. We use >>> cache.loadCache method to start the cache loading. When each server >>> completes cache loading it reduces the count by 1 using countDown method. >>> So when all the nodes completes cache loading, the count reaches to zero. >>> When this count reaches to zero we start kafka consumers on all server >>> nodes. >>> >>> But we saw weird behavior in prod env. The 3 server nodes were shut >>> down at the same time. But 1 node is still alive. When this happened the >>> count down was reinitialized to original value i.e. 4. But I am not able to >>> reproduce this in dev env. >>> >>> Is this a bug, when one or more (but not all) nodes goes down then >>> count re initializes back to original value? >>> >>> Thanks, >>> Akash >>> >>
Re: CountDownLatch issue in Ignite 2.6 version
Hi, I was under the impression that countdown latch is implemented in replicated cache. So when any number of nodes go down it does not loose it's state. Can you please explain why atmoc data structures are using 1 back when its state is very important? Can we enforce atomic data structures to use replicated cache? Which cache does ignite use to store atomic data structures? Thanks Prasad On Mon 8 Jun, 2020, 11:58 PM Evgenii Zhuravlev Hi, > > By default, cache, that stores all atomic structures has only 1 backup, > so, after losing all data for this certain latch, it recreates it. To > change the default atomic configuration use > IgniteConfiguration.setAtomicConfiguration. > > Evgenii > > сб, 6 июн. 2020 г. в 06:20, Akash Shinde : > >> *Issue:* Countdown latched gets reinitialize to original value(4) when >> one or more (but not all) node goes down. *(Partition loss happened)* >> >> We are using ignite's distributed countdownlatch to make sure that cache >> loading is completed on all server nodes. We do this to make sure that our >> kafka consumers starts only after cache loading is complete on all server >> nodes. This is the basic criteria which needs to be fulfilled before starts >> actual processing >> >> >> We have 4 server nodes and countdownlatch is initialized to 4. We use >> cache.loadCache method to start the cache loading. When each server >> completes cache loading it reduces the count by 1 using countDown method. >> So when all the nodes completes cache loading, the count reaches to zero. >> When this count reaches to zero we start kafka consumers on all server >> nodes. >> >> But we saw weird behavior in prod env. The 3 server nodes were shut down >> at the same time. But 1 node is still alive. When this happened the count >> down was reinitialized to original value i.e. 4. But I am not able to >> reproduce this in dev env. >> >> Is this a bug, when one or more (but not all) nodes goes down then count >> re initializes back to original value? >> >> Thanks, >> Akash >> >
Re: CountDownLatch issue in Ignite 2.6 version
Hi, By default, cache, that stores all atomic structures has only 1 backup, so, after losing all data for this certain latch, it recreates it. To change the default atomic configuration use IgniteConfiguration.setAtomicConfiguration. Evgenii сб, 6 июн. 2020 г. в 06:20, Akash Shinde : > *Issue:* Countdown latched gets reinitialize to original value(4) when > one or more (but not all) node goes down. *(Partition loss happened)* > > We are using ignite's distributed countdownlatch to make sure that cache > loading is completed on all server nodes. We do this to make sure that our > kafka consumers starts only after cache loading is complete on all server > nodes. This is the basic criteria which needs to be fulfilled before starts > actual processing > > > We have 4 server nodes and countdownlatch is initialized to 4. We use > cache.loadCache method to start the cache loading. When each server > completes cache loading it reduces the count by 1 using countDown method. > So when all the nodes completes cache loading, the count reaches to zero. > When this count reaches to zero we start kafka consumers on all server > nodes. > > But we saw weird behavior in prod env. The 3 server nodes were shut down > at the same time. But 1 node is still alive. When this happened the count > down was reinitialized to original value i.e. 4. But I am not able to > reproduce this in dev env. > > Is this a bug, when one or more (but not all) nodes goes down then count > re initializes back to original value? > > Thanks, > Akash >
CountDownLatch issue in Ignite 2.6 version
*Issue:* Countdown latched gets reinitialize to original value(4) when one or more (but not all) node goes down. *(Partition loss happened)* We are using ignite's distributed countdownlatch to make sure that cache loading is completed on all server nodes. We do this to make sure that our kafka consumers starts only after cache loading is complete on all server nodes. This is the basic criteria which needs to be fulfilled before starts actual processing We have 4 server nodes and countdownlatch is initialized to 4. We use cache.loadCache method to start the cache loading. When each server completes cache loading it reduces the count by 1 using countDown method. So when all the nodes completes cache loading, the count reaches to zero. When this count reaches to zero we start kafka consumers on all server nodes. But we saw weird behavior in prod env. The 3 server nodes were shut down at the same time. But 1 node is still alive. When this happened the count down was reinitialized to original value i.e. 4. But I am not able to reproduce this in dev env. Is this a bug, when one or more (but not all) nodes goes down then count re initializes back to original value? Thanks, Akash
