[
https://issues.apache.org/jira/browse/IGNITE-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Rakov reassigned IGNITE-6334:
----------------------------------
Assignee: Ivan Rakov
> Throttle writing threads during ongoing checkpoint with token bucket algorithm
> ------------------------------------------------------------------------------
>
> Key: IGNITE-6334
> URL: https://issues.apache.org/jira/browse/IGNITE-6334
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
> Affects Versions: 2.1
> Reporter: Ivan Rakov
> Assignee: Ivan Rakov
> Fix For: 2.3
>
> Attachments: opsec3.jpg
>
>
> We've received several negative pieces of feedback about LFS performance with
> enabled persistence. Ignite node stops responding to user operations under
> intensive load. Typical operations/second graph is attached.
> Zero dropdowns happen during ongoing checkpoint when checkpoint buffer
> (memory segment that accumulates old versions of dirty pages that are not yet
> written in current checkpoint) is overflowed.
> In general, performance decrease is inevitable - writing in memory is always
> faster than writing to disk. Though, we can avoid zero dropdowns if we'll
> throttle threads that generate dirty pages.
> We can manage amount of throttle time with tocken bucket algorithm:
> 1) Before checkpoint start, we calculate ratio K = (number of checkpoint
> pages) / (size of checkpoint buffer) and initialize non-negative atomic
> marker counter
> 2) Every checkpointing thread increments marker counter once per K written
> pages
> 3) Any thread that makes page dirty should decrement marker counter. Thread
> should wait if marker counter is zero.
> Such algorithm makes buffer overflow impossible. If activity is intensive and
> constant, user threads will write at the speed of the disk. On the other
> hand, user threads will write at maximum speed in case of burst activity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
