Hello! Let's talk about persistence performance.
First of all, you have to understand how checkpointing works. It works by writing a whole 4k page if you happen to add or update an object to it. This means when you have a lot of updates there will be a lot of writes. When you have a write-heavy process, you can often mitigate this by having large checkpointPageBufferSize and rare checkpoints: https://apacheignite.readme.io/docs/durable-memory-tuning#section-checkpointing-buffer-size You can have checkpointPageBufferSize as large as 1/3~1/2 of your durable memory size at the expense of durable memory. So you can decrease your data region, increase your checkpointing page buffer. You should also set your checkpoint frequency to something like 300000 (5 minutes): https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/DataStorageConfiguration.html#getCheckpointFrequency-- https://apacheignite.readme.io/docs/persistence-checkpointing When checkpointing is rare there is less chance than the same page will be written to disk multiple times. I hope that you have already set WALMode to LOG_ONLY, did you? Moreover, you can try and disableWal() before populating your database and enableWal() after data ingestion is complete. This is not fit for steady state but when you have burst of writes it works nicely: https://apacheignite.readme.io/docs/write-ahead-log#section-wal-activation-and-deactivation Regards, -- Ilya Kasnacheev пн, 3 дек. 2018 г. в 22:58, kellan <[email protected]>: > I'm Using 2.6 on AWS. Like I mentioned, my Ignite cluster is running on i3 > instances which have local storage, so burst shouldn't be a problem. > > The trend I've noticed is that my writes-per-second increases, while the > size of each write decreases, and the number of PUT operations per second > and CPU usage also decreases. > > I don't know if this is applicable here, but while running Kafka I ran into > problems like this when I didn't have enough memory dedicated to page > cache, > but I don't know if this should be a consideration with Ignite. I'm > following Ignite's performance guidelines and dedicating less than 70% of > my > available memory to Ignite Durable Memory and the Java Heap. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
