[
https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403293#comment-13403293
]
Suresh Srinivas commented on HDFS-3077:
---------------------------------------
Based on previous discussions, HDFS-3092 was decided to be the server side for
JournalProtocol and would provide JournalNode capability. HDFS-3077 would be
the client side implementation of Quorum based Journal Writer. In the first
phase we decided to go with the Quorum based journal writer and decided to
abandon the BK style writer proposed in 3092, for the time being. But it left
the window open for writing other types of writes based on 3092 server side.
See -
https://issues.apache.org/jira/browse/HDFS-3092?focusedCommentId=13271098&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13271098
I have not looked at the patch deeply enough. But here are my high level
comments:
# 3092 added the capabilities to fence and epoch semantics to JournalProtocol.
I believe adding support for muliple namespaces should be done as an additional
change and not all in one big patch. This could be added to 3092 or to trunk
post merging 3092.
# Quorum is a semantics on the client/writer side and not the server side
policy. Hence the protocol for journaling should be generic enough. So lets not
call it QJournalProtocol and make it generic, allowing other types of
clients/writers.
# Considerable work has gone into HDFS-3092. If you see some functionality
missing in 3092, lets discuss and add it there, instead of copying code and
changing it separately.
My preference is to see HDFS-3092 committed as a generic journal node
functionality and 3077 enable Quorum write mechanism. Also 3092 has been in
development in open, in incremental fashion. I think we should follow this,
instead of attaching a big patch from github.
> Quorum-based protocol for reading and writing edit logs
> -------------------------------------------------------
>
> Key: HDFS-3077
> URL: https://issues.apache.org/jira/browse/HDFS-3077
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: ha, name-node
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-3077-partial.txt, hdfs-3077.txt,
> qjournal-design.pdf, qjournal-design.pdf
>
>
> Currently, one of the weak points of the HA design is that it relies on
> shared storage such as an NFS filer for the shared edit log. One alternative
> that has been proposed is to depend on BookKeeper, a ZooKeeper subproject
> which provides a highly available replicated edit log on commodity hardware.
> This JIRA is to implement another alternative, based on a quorum commit
> protocol, integrated more tightly in HDFS and with the requirements driven
> only by HDFS's needs rather than more generic use cases. More details to
> follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
