[
https://issues.apache.org/jira/browse/ZOOKEEPER-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998468#comment-12998468
]
Flavio Junqueira commented on ZOOKEEPER-1001:
---------------------------------------------
Ben and I discussed this issue offline, and I'd like to summarize our
discussion.
To enable readers to read from a ledger open for writing we need to have a
couple of things in place:
# An openLedger call that does not try to recover the ledger if it can't find
the corresponding ledger znode on zookeeper;
# A notification mechanism to let the reader know how far it can read into a
ledger and when.
For the first point, we can add a new call to the api for unsafely opening a
ledger. This call is unsafe in the sense that the end of the ledger is not
known and the client needs to figure out what entries have been committed so
that it can safely read them.
This last point brings us to the discussion of how to notify the reader that
new entries have been committed (we don't want a reader either waiting too long
to read new entries or trying to guess when new entries have come in).
We see it possibly implemented in one of two ways: inside or outside
bookkeeper. Implementing it inside bookkeeper is not trivial, since our
replication protocol does not require bookies to talk to each other and no
bookie individually knows when an entry has been committed. The second option
seems to be more feasible. The writer can communicate periodically to one or
more readers the entries that have been committed either directly or through
zookeeper.
One alternative for a reader to learn what entries have been committed is to
read the last hint from bookies. We have a mechanism implemented that enables a
reader to determine what was the last entry confirmed at the time the last
entry of a ledger was generated. This approach, however, has the drawback that
the hint is behind the last committed entry, since an entry can only have a
previous entry as a hint.
Thoughts or questions?
> Read from open ledger
> ---------------------
>
> Key: ZOOKEEPER-1001
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1001
> Project: ZooKeeper
> Issue Type: New Feature
> Components: contrib-bookkeeper
> Reporter: Flavio Junqueira
>
> The BookKeeper client currently does not allow a client to read from an open
> ledger. That is, if the creator of a ledger is still writing to it (and the
> ledger is not closed), then an attempt to open the same ledger for reading
> will execute the code to recover the ledger, assuming that the ledger has not
> been correctly closed.
> It seems that there are applications that do require the ability to read from
> a ledger while it is being written to, and the main goal of this jira is to
> discuss possible implementations of this feature.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
