[
https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245346#comment-13245346
]
Suresh Srinivas commented on HDFS-3092:
---------------------------------------
bq. Your design does not seem to consider the possibility of multiple
concurrent logs, which you may want to have for federation.
For HDFS editlogs, my feeling is that there will only be three JDs. One on the
active namenode, second on the standby and a third JD on one of the machines.
In federation, one has to configure a JD per Federated namespace. Alternative
is to use BookKeeper, since it could make the deployment simpler for federated
large cluster.
bq. There has been comments about comparing the different approaches discussed,
and I was wondering what criteria you have been thinking of using to compare
them.
I think the comment was more about comparing the design and complexity of
deployment and not benchmarks for two systems. Performance is not the
motivation for this jira.
bq. I was wondering about how reads to the log are executed if writes only have
to reach a majority quorum. Once it is time to read, how does the reader gets a
consistent view of the log? One JD alone may not have all entries, so I suppose
the reader may need to read from multiple JDs to get a consistent view? Do the
transaction identifiers establish the order of entries in the log? One quick
note is that I don't see why a majority is required; bk does not require a
majority.
We decided on majority quorum to keep the design simple, though it is strictly
not necessary. A JD in JournalList is supposed to have all the entries and any
JD from the list can be used to read the journals.
bq. Here are some notes I took comparing the bk approach with the one in this
jira, in the case you're interested
I noticed that as well. After we went thourgh many issues that this solution
had to take care of, the solution looks very similar to BK. That is comforting
:-)
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>
> Key: HDFS-3092
> URL: https://issues.apache.org/jira/browse/HDFS-3092
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha, name-node
> Affects Versions: 0.24.0, 0.23.3
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Attachments: MultipleSharedJournals.pdf
>
>
> Currently standby namenode relies on reading shared editlogs to stay current
> with the active namenode, for namespace changes. BackupNode used streaming
> edits from active namenode for doing the same. This jira is to explore using
> journal protocol based editlog streams for the standby namenode. A daemon in
> standby will get the editlogs from the active and write it to local edits. To
> begin with, the existing standby mechanism of reading from a file, will
> continue to be used, instead of from shared edits, from the local edits.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
