[
https://issues.apache.org/jira/browse/BOOKKEEPER-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628141#comment-13628141
]
Ivan Kelly commented on BOOKKEEPER-564:
---------------------------------------
{quote}the integration is easy due to skiplist extends
InterleavedLedgerStorage. but if another implementation of LedgerStorage is not
based InterleavedLedgerStorage, it has to implement all the things in
InterleavedLedgerStorage again.{quote}
Which part needs to be reimplemented? storing the log mark? Perhaps the new
ledger storage stores it in a different way. It's quite conceivable that it may
store the log mark in the footer of a log file, rather than in an individual
file for example.
I can't think of anything else I've moved into ledger storage, which a new
ledger storage wouldn't have to implement anyhow, like checkpointing and
flushing, if it even considers them different things.
{quote}
"Journal(ServerConfiguration conf, LogMark lastLogMark)" looks like that if I
passed any mark, the journal could replay starting from the passed mark. but it
couldn't, since Journal gc its journal files. {quote}
It couldn't have gc'd unless the ledger storage had indicated that that mark
had been synced to. And if the ledger storage has indicated this, it shouldn't
expect though journals still to be there.
This doesn't change whether you're having bookie asked for the last sync mark
and gc, or passing it through Checkpoint#complete. Whats more, with
Checkpoint#complete, you're having the ledger storage driving actions on the
journal, rather than the bookie driving the action. The bookie owns the
journal, so it can tell it what to do. For the ledger storage, this is not the
case
{quote}
And Checkpoint#complete is the way to bridge ledger storage and journal, which
tells journal that ledger storage already synced until this checkpoint
{quote}
So the journal storage tells the journal that it has synced to the checkpoint.
This means that the checkpoint is something that is inheritly a property of the
ledger storage. The ledger storage + the mark/checkpoint tells you something
about the state of the system. The journal + mark/checkpoint tells you nothing.
> Better checkpoint mechanism
> ---------------------------
>
> Key: BOOKKEEPER-564
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-564
> Project: Bookkeeper
> Issue Type: Improvement
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.3.0
>
> Attachments: 0001-BOOKKEEPER-564-Better-checkpoint-mechanism.patch,
> 0001-BOOKKEEPER-564-Better-checkpoint-mechanism.patch,
> 0002-BOOKKEEPER-564-Better-checkpoint-mechanism.patch, BOOKKEEPER-564.patch,
> BOOKKEEPER-564.patch
>
>
> Currently, SyncThread made a checkpoint too frequently, which affects
> performance. data is writing to entry logger file might be blocked by syncing
> same entry logger file, which affect bookie to achieve higher throughput. We
> could schedule checkpoint only when rotating an entry log file. so new
> incoming entries would be written to newer entry log file and old entry log
> file could be synced.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
