[
https://issues.apache.org/jira/browse/BOOKKEEPER-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270478#comment-13270478
]
Rakesh R commented on BOOKKEEPER-237:
-------------------------------------
Thanks Ivan for the comments.
bq. The ack quorum is the number of bookies who must acknowledge the entry
before it is acked to the client application
I have gone through BookKeeper-208, 'write quorum' is using the RRDS algo for
interleaving ?
Also will reform the ensemble if any Bookie in this ensemble is slow/dead and
timedout ?
But, I was thinking to avoid the interleaving of entries and ensemble
reformation on write entry failures to make recovery simple. Here the idea is,
all the ensemble Bookies will be having the replicas in best case. Ack quorum
also set to the ensemble size. Here, assured replica = quorum size.
Please see the doc section '1.4.4 Example: How it works'
For any ledger,
in best case, total replicas = ensemble size
in worst case, total replicas = quorum size, which is the assured/majority
replicas.
Also, I feel able to handle intermittent network problems as by default this
approach will tolerate (ensemble - quorum) failures.
How client will read?
For an inprogress ledger, bkclient looks to the entire ensemble for reading.
For a closed ledger, bkclient looks to the CLOSED ensemble for reading.
{quote} I don't think its necessary to maintain a mapping of bookies to
ledgers. To rereplicate a ledger, it is necessary to read the whole ledger
metadata for the ledger, so a mapping of which bookies should contain the
ledger is straightforward to build on the fly.{quote}
I was trying to avoid multiple ZK calls to read ledger metadata for the
detection. Only for the failed ledgers, Accountant would read the ledger
metadata and initiate the rereplication.
Also, the map contains the _inprogressreplica information, so the new
Accountant would be knowing about the already initiated rereplicas.
{quote}
The rereplication process should be distributed across all bookies. Each bookie
should run a replication process which takes the first available unreplicated
ledger off the queue and rereplicates it.
{quote}
Yeah, its good. If I understand correctly, instead of assigning work to the
Bookie, Accountant would keep the under replicated ledgers. Bookies (which is
not an existing replica holder for that ledger) would takes the unreplicated
ledger and after finish send ack to the Accountant. I feel, we should define
proper locking here to avoid concurrency?
> Automatic recovery of under-replicated ledgers and its entries
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-237
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-237
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: Auto Recovery and Bookie sync-ups.pdf
>
>
> As per the current design of BookKeeper, if one of the BookKeeper server
> dies, there is no automatic mechanism to identify and recover the under
> replicated ledgers and its corresponding entries. This would lead to losing
> the successfully written entries, which will be a critical problem in
> sensitive systems. This document is trying to describe few proposals to
> overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
