Re: Ignite Data loss
Hi, Please attach full logs from all nodes for investigation. ср, 29 апр. 2020 г. в 06:48, krkumar24061...@gmail.com < krkumar24061...@gmail.com>: > Checked the logs and nothing like that happened where went down. > > Thanx and Regards, > KR Kumar > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite Data loss
Checked the logs and nothing like that happened where went down. Thanx and Regards, KR Kumar -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite Data loss
You are using PARTITIONED caches, and some nodes might have left the grid causing data loss look for this message: Topology snapshot [ .. servers=1, clients=0] see if you are seeing servers appear or disappear? Try changing your cache config to REPLICATED to see whether this issue re-occurs? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite Data loss
Hi Alex - Here is the cache configuration for the cache
CacheConfiguration cacheConfig = new
CacheConfiguration<>();
cacheConfig.setCacheMode(CacheMode.PARTITIONED);
cacheConfig.setRebalanceMode(CacheRebalanceMode.ASYNC);
cacheConfig.setRebalanceDelay(rebalanceDelay);
cacheConfig.setName("eventCache-" + tenantRunId + "-" +
tenantId);
cacheConfig.setBackups(1);
cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
IgniteCache cache =
IgniteContextWrapper.getInstance().getEngine()
.getOrCreateCache(cacheConfig);
Here is how i am looking at the size
logger.info("Events already persisted into store for Tenant : " + tenantId
+ " are "
+
cache.sizeLong(CachePeekMode.PRIMARY));
Thanx and Regards,
KR Kumar
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite Data loss
Hi, Are you using expiry policies? https://apacheignite.readme.io/docs/expiry-policies Share your config? How are you measuring the # of entries? Thanks, Alex -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ignite Data loss
Hi Guys - I have ignite cache with persistence enabled. Three days back I was having 11 billion events in the cache and yesterday it became around 10 billion and now its around 8.9 billion. Ignite data is constantly going down on its own. Am I doing something wrong with my config or have you guys faced similar problem ever?? Thanx and Regards, KR Kumar -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite data loss
Thanks for your reply guys.
I am not sure if I have really solved the problem but this is how i fixed
it. Initially i was adding the nodes to the baseline topology through code.
I have removed that and now adding the nodes thru control.sh which case i am
not loosing any data. I do not know the root cause of the issue, but its a
temp. fix that's working for me for now.
The code that i am using to set the baseline topology is as follows:
// logger.info("Setting the baseline topology ...");
//
engine.cluster().setBaselineTopology(engine.cluster().forServers().nodes());
// logger.info("Baseline topology is set");
Thanx and Regards,
KR Kumar
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: Re: Ignite data loss
Hello! Is it possible that you had disabled WAL before restarting a cluster? Doing this may cause data loss in some scenarios. From: MikaelSent: Saturday, November 16, 2019 9:58 AMTo: user@ignite.apache.orgSubject: Re: Ignite data loss Hi!So it does not matter what node you restart ? it's always that one that keeps the data ?Are all 4 nodes part of the baseline topology ?I pretty much have the same setup but with 3 nodes and have not had any problems at all, not sure what it could be ?If you turn on all logging you will see all it does at startup and can see if there is something weird going on, it's lots of information but it usually gives a good indication if there is any problem, nothing else in the logs ? if the nodes clear everything at startup there should be something in the logs.MikaelDen 2019-11-16 kl. 04:16, skrev KR Kumar:Hi Guys - I have a four node cluster with native persistence enabled. Its a partitioned cache and sync rebalance is enabled. When I restart the cluster the first node that starts retain the data and all the other nodes data is deleted and all the ignite data files are turned into 4096 byte files. Am I missing something? or some configuration that I missing. Following is the cache configuration: CacheConfigurationbyte[]> cacheConfig = new CacheConfigurationbyte[]>(); cacheConfig.setCacheMode(CacheMode.PARTITIONED); cacheConfig.setRebalanceMode(CacheRebalanceMode.SYNC);// cacheConfig.setRebalanceDelay(30); cacheConfig.setName("eventCache-" + tenantRunId + "-" + tenantId); cacheConfig.setBackups(1); cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC); cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC); IgniteCachebyte[]> cache = IgniteContextWrapper.getInstance().getEngine() .getOrCreateCache(cacheConfig); Here is the configuration of ignite Any quick pointers ?? Thanx and Regards,KR Kumar
Re: Ignite data loss
Hi!
So it does not matter what node you restart ? it's always that one that
keeps the data ?
Are all 4 nodes part of the baseline topology ?
I pretty much have the same setup but with 3 nodes and have not had any
problems at all, not sure what it could be ?
If you turn on all logging you will see all it does at startup and can
see if there is something weird going on, it's lots of information but
it usually gives a good indication if there is any problem, nothing else
in the logs ? if the nodes clear everything at startup there should be
something in the logs.
Mikael
Den 2019-11-16 kl. 04:16, skrev KR Kumar:
Hi Guys - I have a four node cluster with native persistence enabled.
Its a partitioned cache and sync rebalance is enabled. When I restart
the cluster the first node that starts retain the data and all the
other nodes data is deleted and all the ignite data files are turned
into 4096 byte files. Am I missing something? or some configuration
that I missing.
Following is the cache configuration:
CacheConfiguration cacheConfig = new
CacheConfiguration();
cacheConfig.setCacheMode(CacheMode.PARTITIONED);
cacheConfig.setRebalanceMode(CacheRebalanceMode.SYNC);
//cacheConfig.setRebalanceDelay(30);
cacheConfig.setName("eventCache-"+ tenantRunId+ "-"+ tenantId);
cacheConfig.setBackups(1);
cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
IgniteCache cache =
IgniteContextWrapper.getInstance().getEngine()
.getOrCreateCache(cacheConfig);
Here is the configuration of ignite
class="org.apache.ignite.configuration.IgniteConfiguration">
Any quick pointers ??
Thanx and Regards,
KR Kumar
Ignite data loss
Hi Guys - I have a four node cluster with native persistence enabled. Its a
partitioned cache and sync rebalance is enabled. When I restart the cluster
the first node that starts retain the data and all the other nodes data is
deleted and all the ignite data files are turned into 4096 byte files. Am I
missing something? or some configuration that I missing.
Following is the cache configuration:
CacheConfiguration cacheConfig = new CacheConfiguration();
cacheConfig.setCacheMode(CacheMode.PARTITIONED);
cacheConfig.setRebalanceMode(CacheRebalanceMode.SYNC);
// cacheConfig.setRebalanceDelay(30);
cacheConfig.setName("eventCache-" + tenantRunId + "-" + tenantId);
cacheConfig.setBackups(1);
cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.
FULL_SYNC);
IgniteCache cache =
IgniteContextWrapper.getInstance().getEngine()
.getOrCreateCache(cacheConfig);
Here is the configuration of ignite
Any quick pointers ??
Thanx and Regards,
KR Kumar
