[ https://issues.apache.org/jira/browse/IGNITE-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Rakov reassigned IGNITE-6334: ---------------------------------- Assignee: Ivan Rakov > Throttle writing threads during ongoing checkpoint with token bucket algorithm > ------------------------------------------------------------------------------ > > Key: IGNITE-6334 > URL: https://issues.apache.org/jira/browse/IGNITE-6334 > Project: Ignite > Issue Type: Improvement > Components: persistence > Affects Versions: 2.1 > Reporter: Ivan Rakov > Assignee: Ivan Rakov > Fix For: 2.3 > > Attachments: opsec3.jpg > > > We've received several negative pieces of feedback about LFS performance with > enabled persistence. Ignite node stops responding to user operations under > intensive load. Typical operations/second graph is attached. > Zero dropdowns happen during ongoing checkpoint when checkpoint buffer > (memory segment that accumulates old versions of dirty pages that are not yet > written in current checkpoint) is overflowed. > In general, performance decrease is inevitable - writing in memory is always > faster than writing to disk. Though, we can avoid zero dropdowns if we'll > throttle threads that generate dirty pages. > We can manage amount of throttle time with tocken bucket algorithm: > 1) Before checkpoint start, we calculate ratio K = (number of checkpoint > pages) / (size of checkpoint buffer) and initialize non-negative atomic > marker counter > 2) Every checkpointing thread increments marker counter once per K written > pages > 3) Any thread that makes page dirty should decrement marker counter. Thread > should wait if marker counter is zero. > Such algorithm makes buffer overflow impossible. If activity is intensive and > constant, user threads will write at the speed of the disk. On the other > hand, user threads will write at maximum speed in case of burst activity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)