[
https://issues.apache.org/jira/browse/BOOKKEEPER-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412907#comment-13412907
]
Matteo Merli commented on BOOKKEEPER-220:
-----------------------------------------
On a slightly related topic, we've been thinking about the
ManagedCursor.markDelete() method. Its goal is to persist the last-message
acknowledged by a client and it's implemented updating a z-node with the
current acked-position. As we previously discussed, this generates a lot of
updates into the cursors z-nodes on ZK.
To circumvent this issue, we thought it may be possible to keep the cursors
mark-deleted persisted positions directly in BK. That would mean using a
separate BK ledger for each cursor and that markDelete() just appends the new
position to the end of that ledger (after N updates the ledger will be deleted
and the cursor will append to a new one).
When doing a recovery (when the ML is reopened), we'd just need to read the
last entry in each of the cursors ledgers. The only drawback would be in having
more BK ledgers.
Another schema could be to use a single BK ledger for all the cursors inside
the same ML, and periodically snapshot all the cursors to avoid having to
rescan the whole ledger when there is a slow consumer.
Do you see any potential problem with these approach? Or a better way to do the
same?
> Managed Ledger proposal
> -----------------------
>
> Key: BOOKKEEPER-220
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-220
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client
> Reporter: Matteo Merli
> Assignee: Matteo Merli
> Fix For: 4.2.0
>
> Attachments: 0001-BOOKKEEPER-220-Managed-Ledger-proposal.patch,
> 0001-BOOKKEEPER-220-Managed-Ledger-proposal.patch,
> 0001-BOOKKEEPER-220-Managed-Ledger-proposal.patch
>
>
> The ManagedLedger design is based on our need to manage a set of ledgers,
> with a single writer (at any point in time) and a set on consumers that read
> entries from it.
> The ManagedLedger also takes care of periodically closing ledgers to have a
> "reasonable" sized sets of ledgers that can individually deleted when no more
> needed.
> I've put on github the interface proposal (along with an early WIP
> implementation)
> http://github.com/merlimat/managed-ledger
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
