[
https://issues.apache.org/jira/browse/BOOKKEEPER-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624370#comment-13624370
]
Ivan Kelly commented on BOOKKEEPER-564:
---------------------------------------
{quote}
but since checkpointing is a behavior of ledger storage, it would be better to
keep checkpoint in ledger storage rather than in SyncThread,
{quote}
So why have a sync thread? The checkpoint/mark is a property of the ledger
storage. It signifies the point in history to which the ledger storage has
synced to disk, and therefore the point from which the bookie needs to start
applying entries from the journal on a restart. The actual journal only uses it
as a convenience mechanism for GC.
So we should give the ledger storage absolute control over checkpointing. The
ledger storage should also take care of storing the mark also, since we store
it in the ledger directories. The LedgerStorage should have a method,
LedgerStorage#getSyncedMark() which gives back the mark. This method can be
used on recovery to find where to start applying the journal from. Also there
can be a background thread to gc the journals based on what #getSyncedMark
returns.
Also we'll need to give the mark to the ledger storage at some point. pre
BOOKKEEPER-572, this can be done after adding to ledger storage, but before
adding to the journal. post BOOKKEEPER-572, the mark should be taken from the
journal before adding to it, and give to the ledger storage after it has been
applied to the journal.
> Better checkpoint mechanism
> ---------------------------
>
> Key: BOOKKEEPER-564
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-564
> Project: Bookkeeper
> Issue Type: Improvement
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.3.0
>
> Attachments: 0002-BOOKKEEPER-564-Better-checkpoint-mechanism.patch,
> BOOKKEEPER-564.patch
>
>
> Currently, SyncThread made a checkpoint too frequently, which affects
> performance. data is writing to entry logger file might be blocked by syncing
> same entry logger file, which affect bookie to achieve higher throughput. We
> could schedule checkpoint only when rotating an entry log file. so new
> incoming entries would be written to newer entry log file and old entry log
> file could be synced.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
