[
https://issues.apache.org/jira/browse/BOOKKEEPER-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626331#comment-13626331
]
Sijie Guo commented on BOOKKEEPER-564:
--------------------------------------
{quote}
So we should give the ledger storage absolute control over checkpointing. The
ledger storage should also take care of storing the mark also, since we store
it in the ledger directories. The LedgerStorage should have a method,
LedgerStorage#getSyncedMark() which gives back the mark. This method can be
used on recovery to find where to start applying the journal from. Also there
can be a background thread to gc the journals based on what #getSyncedMark
returns.
Also we'll need to give the mark to the ledger storage at some point. pre
BOOKKEEPER-572, this can be done after adding to ledger storage, but before
adding to the journal. post BOOKKEEPER-572, the mark should be taken from the
journal before adding to it, and give to the ledger storage after it has been
applied to the journal.
{quote}
you could just pass the checkpoint when adding entry to ledger storage. so for
interleaved ledger storage, when it rolls an entry log file, it used its kept
checkpoint to do checkpointing; for skip list based ledger storage, each skip
list kept a checkpoint, when it snapshot current skip list, it used the
checkpoint kept in snapshotted skip list to do checkpointing. I don't see how
difficult that BOOKKEEPER-572 could be addressed based on this jira.
> Better checkpoint mechanism
> ---------------------------
>
> Key: BOOKKEEPER-564
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-564
> Project: Bookkeeper
> Issue Type: Improvement
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.3.0
>
> Attachments: 0002-BOOKKEEPER-564-Better-checkpoint-mechanism.patch,
> BOOKKEEPER-564.patch, BOOKKEEPER-564.patch
>
>
> Currently, SyncThread made a checkpoint too frequently, which affects
> performance. data is writing to entry logger file might be blocked by syncing
> same entry logger file, which affect bookie to achieve higher throughput. We
> could schedule checkpoint only when rotating an entry log file. so new
> incoming entries would be written to newer entry log file and old entry log
> file could be synced.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
