Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hello! Maybe you should try having more frequent checkpoints? As for WAL archive, you can try. Regards, -- Ilya Kasnacheev вт, 5 февр. 2019 г. в 14:35, radha : > Hi, > > I increased the java heap size , then i am able to put 160GB of data. > Without any node failure. > Ignite JVM

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hi, I increased the java heap size , then i am able to put 160GB of data. Without any node failure. Ignite JVM configuration: -server -Xms8g -Xmx8g -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC I am afaraid, if one of ignite-server fails , and if i am unable

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hello! Is it possible that you have deleted/lost some of WAL files from this instance? If not, I'm afraid we can only figure it out if you share your PDS files (wal + checkpoint dirs) of affected instance. Regards, -- Ilya Kasnacheev пн, 4 февр. 2019 г. в 10:07, radha jai : > I am using the

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

I am using the default WAL mode. I think its LOG_ONLY. The crashed ignite server log is below: {"type":"log","host":"ignite-cluster-ap-ignite-10","level":"INFO","systemid":"296b639f","system":"ignite-service","time":"2019-01-31

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hello! It's hard to say outright. Can you provide full log before node crash? Is there a chance that you ran out of disk space? What's your WALMode? Regards, -- Ilya Kasnacheev пт, 1 февр. 2019 г. в 08:16, radha jai : > Hi, >Ignite has been deployed on k8s has 12 ignite-servers, which

checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hi, Ignite has been deployed on k8s has 12 ignite-servers, which are spread out one on each worker node. The limits are 1 CPU 32GB RAM, with maximum of 8 CPU and 64GB. Each ignite-server has a WAL and Persistent storage volume of 30GB. Getting below error after inserting the 60GB of data

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

KR, any chance you can provide a reproducer? It would really help us properly debug your issue. If not, can we get a copy of your configuration? On Thu, Oct 12, 2017 at 10:31 AM, KR Kumar wrote: > Hi AG, > > Thanks for responding to the thread. I have tried with 2.3 and

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hi AG, Thanks for responding to the thread. I have tried with 2.3 and I still face the same problem. Just to further explore, I killed ignite instance with kill -9 and a reboot, both situations, ignite just hangs during restart. Thanx and Regards KR Kumar -- Sent from:

Re: checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hi, This should never happen in BACKGROUND mode unless you have a hard power kill for your Ignite node (which is not your case). I've reviewed the related parts of the code and found that there were a few tickets fixed in 2.3 that may have caused this issue (e.g. IGNITE-5772). Can you try

checkpoint marker is present on disk, but checkpoint record is missed in WAL

Hi Guys - I am using ignite persistence with a 8 node cluster. Currently in dev/poc stages. I get following exception when i try to restart the node after I killed the process with "kill . I have a shutdown hook to the code in which I am shutting down Ignite with G.stop(false). I read in a blog