[
https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026025#comment-14026025
]
Aaron T. Myers commented on HADOOP-10641:
-----------------------------------------
bq. Not sure where this leaves QJM. I thought it satisfies all these
requirements including the Hadoop's aim.
The NameNode QuorumJournalManager and JNs are expressly for storing HDFS NN
edit logs. Not for general purpose consensus, not for use by other projects
like HBase, etc.
bq. Hadoop common is chosen for the coordination engine interface as the lowest
common ancestor.
But a separate project could just as well be a common ancestor, just like both
Hadoop and HBase separately depend on ZooKeeper. There's no actual need for it
to be in Hadoop Common if HBase is to use it.
bq. It could be used for anything Hadoop from here: HDFS, HBase, Yarn.
But seems like it could also be used for arbitrary, non-Hadoop things, correct?
If so, why put it in Hadoop?
bq. I don't have other use cases for it in my mind at the moment. Did not have
enough experience with TLPs and Incubators, but thought it needs like something
bigger. Say, in this case, implementations for more than one CE. Sorry if I
misunderstood you, but if you want to take it to Apache Incubator I'll be on
your side.
I personally don't think there's any good reason for this to start out as part
of a larger project, and honestly think there are several downsides. For
example, Hadoop's release cadence is too slow for a new project like this,
there's not much expertise in Hadoop for the general problem of distributed
consensus, possible desire for other non-Hadoop projects to want to use it, etc.
> Introduce Coordination Engine
> -----------------------------
>
> Key: HADOOP-10641
> URL: https://issues.apache.org/jira/browse/HADOOP-10641
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Attachments: HADOOP-10641.patch, HADOOP-10641.patch,
> HADOOP-10641.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of
> events in a distributed system. In order to be reliable CE should be
> distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC,
> zab) and have different implementations, depending on use cases, reliability,
> availability, and performance requirements.
> CE should have a common API, so that it could serve as a pluggable component
> in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and
> HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
