Re: Warning in the logs - Writes are very slow

2020-03-16 Thread krkumar24061...@gmail.com
Hi Mike - Thanks for the reply. Please take a look at the point to point details. Ignite need to sync memory and disk time to time, if you write rate is bigger than disk speed that means Ignite will never able to sync memory and disk, that's why it starts to throttle your writes. Its a

Re: Warning in the logs - Writes are very slow

2020-03-16 Thread krkumar24061...@gmail.com
HI - Thanks for getting back to me. This is happening in our test environment and we have 4TB gp2 disks i.e. general purpose SSD with 12000 IOPS. Its a very write intensive work where we compress records that are anything between 1kb to 4kb and push them to ignite. Right now the throughput is

Re: Warning in the logs - Writes are very slow

2020-03-16 Thread Ilya Kasnacheev
Hello! Let me add to the previous answers: This message means that checkpoint page buffer is almost exhausted. It is recommended to increase its size (dataRegionCfg.checkpointPageBufferSize) if you have a lot of writes. Please see

Re: Warning in the logs - Writes are very slow

2020-03-13 Thread Mikhail
Hi Kumar, Ignite need to sync memory and disk time to time, if you write rate is bigger than disk speed that means Ignite will never able to sync memory and disk, that's why it starts to throttle your writes. So ignite added 1.6ms delay to each write, to be more accurate, not for each writes,

Re: Warning in the logs - Writes are very slow

2020-03-13 Thread akorensh
Hi, Follow the recommendations for persistence tuning here: https://www.gridgain.com/docs/latest/perf-troubleshooting-guide/persistence-tuning How much data is being saved to disk? Are you able to adjust the rate of data flow and see whether it makes a difference. Can you try using

Re: Warning in the logs - Writes are very slow

2020-03-13 Thread akorensh
Hi, Follow the recommendations for persistence tuning here: https://www.gridgain.com/docs/latest/perf-troubleshooting-guide/persistence-tuning How much data is being saved to disk? Are you able to adjust the rate of data flow and see whether it makes a difference. Can you try using

Warning in the logs - Writes are very slow

2020-03-13 Thread krkumar24061...@gmail.com
Hi All - I see this message in the logs [2020-03-12 15:23:29,896][INFO ][comcastprod-1-StoreFlushWorker-7][PageMemoryImpl] Throttling is applied to page modifications [*percentOfPartTime=0.54*, markDirty=8185 pages/sec, checkpointWrite=14893 pages/sec, estIdealMarkDirty=8182 pages/sec,

Warning in the logs - Writes are very slow

2020-03-13 Thread krkumar24061...@gmail.com
Hi All - I see this message in the logs [2020-03-12 15:23:29,896][INFO ][comcastprod-1-StoreFlushWorker-7][PageMemoryImpl] Throttling is applied to page modifications [*percentOfPartTime=0.54*, markDirty=8185 pages/sec, checkpointWrite=14893 pages/sec, estIdealMarkDirty=8182 pages/sec,