[
https://issues.apache.org/jira/browse/IGNITE-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174424#comment-16174424
]
ASF GitHub Bot commented on IGNITE-6334:
----------------------------------------
GitHub user glukos opened a pull request:
https://github.com/apache/ignite/pull/2710
IGNITE-6334 Throttle writing threads during ongoing checkpoint
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-6334
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/2710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2710
----
commit cc8cc25a81dc98f02212dd6b499b5e030426d0dd
Author: Ivan Rakov <[email protected]>
Date: 2017-09-21T08:16:00Z
IGNITE-6334 Throttle writing threads during ongoing checkpoint with token
bucket algorithm
----
> Throttle writing threads during ongoing checkpoint with token bucket algorithm
> ------------------------------------------------------------------------------
>
> Key: IGNITE-6334
> URL: https://issues.apache.org/jira/browse/IGNITE-6334
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
> Affects Versions: 2.1
> Reporter: Ivan Rakov
> Assignee: Ivan Rakov
> Fix For: 2.3
>
> Attachments: opsec3.jpg
>
>
> We've received several negative pieces of feedback about LFS performance with
> enabled persistence. Ignite node stops responding to user operations under
> intensive load. Typical operations/second graph is attached.
> Zero dropdowns happen during ongoing checkpoint when checkpoint buffer
> (memory segment that accumulates old versions of dirty pages that are not yet
> written in current checkpoint) is overflowed.
> In general, performance decrease is inevitable - writing in memory is always
> faster than writing to disk. Though, we can avoid zero dropdowns if we'll
> throttle threads that generate dirty pages.
> We can manage amount of throttle time with tocken bucket algorithm:
> 1) Before checkpoint start, we calculate ratio K = (number of checkpoint
> pages) / (size of checkpoint buffer) and initialize non-negative atomic
> marker counter
> 2) Every checkpointing thread increments marker counter once per K written
> pages
> 3) Any thread that makes page dirty should decrement marker counter. Thread
> should wait if marker counter is zero.
> Such algorithm makes buffer overflow impossible. If activity is intensive and
> constant, user threads will write at the speed of the disk. On the other
> hand, user threads will write at maximum speed in case of burst activity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
