[
https://issues.apache.org/jira/browse/ZOOKEEPER-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012416#comment-13012416
]
Flavio Junqueira commented on ZOOKEEPER-1001:
---------------------------------------------
I also agree that it is not strictly necessary for the HDFS journal to have the
NN connecting to ZooKeeper, since the information of what entries have been
committed can be conveyed in other ways, e.g., by explicitly sending from the
writer to the reader(s). Bookies can't provide the hints you're referring to
because no single bookie has a global view of a ledger. Consequently, I can
only see those hints working either via ZooKeeper or via direct communication.
Since we use a ZooKeeper instance to manage a bookie pool and the BK client
talks to ZooKeeper today, the discussion so far seems to converge to using
ZooKeeper to pass the information on what entries are committed. One idea we
have suggested in this jira to do it through the BK client is to have the
writer periodically writing to the CLOSED znode the last committed entry, and
having the reader(s) reading this value. The mechanism to write the CLOSED
znode is there already, so there is a small amount of code change needed. We
need essentially to introduce a call to the BK api that forces the BK client to
write the CLOSED znode.
Given that some applications, including the NN, might learn of committed
entries through ZooKeeper, it seems to make sense to incorporate such accesses
to ZooKeeper to the BK client, but I don't think we have reached agreement on
it yet and leaving to the application to implement it when needed is still an
option. If we do it in the way I suggest, then an application would not be
forced to use ZooKeeper for such hints, and direct communication would also be
possible.
Also, If we end up going this direction of periodically writing to the CLOSED
znode, we should change the name of the znode to something else, since it
doesn't really represent any longer what the znode does.
> Read from open ledger
> ---------------------
>
> Key: ZOOKEEPER-1001
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1001
> Project: ZooKeeper
> Issue Type: New Feature
> Components: contrib-bookkeeper
> Reporter: Flavio Junqueira
> Attachments: zk-1001-design-doc.pdf, zk-1001-design-doc.pdf
>
>
> The BookKeeper client currently does not allow a client to read from an open
> ledger. That is, if the creator of a ledger is still writing to it (and the
> ledger is not closed), then an attempt to open the same ledger for reading
> will execute the code to recover the ledger, assuming that the ledger has not
> been correctly closed.
> It seems that there are applications that do require the ability to read from
> a ledger while it is being written to, and the main goal of this jira is to
> discuss possible implementations of this feature.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
