[
https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027601#comment-14027601
]
Rakesh R commented on HADOOP-10641:
-----------------------------------
bq.The intent of this jira is not to solve the general problem of distributed
consensus. That is, I do not propose to build an implementation of paxos or
other coordination algorithms here. This is only to introduce a common
interface, so that real implementations such as ZooKeeper could be plugged into
hadoop projects.
This sounds interesting. Thanks for the effort!. If I understood the discussion
correctly here, the idea is to build a quorum based replication. For example,
the events(I think this represents data) are submitted as proposals to a quorum
of nodes. In ZooKeeper terms, Leader proposes values to the Followers. Now
Leader wait for acknowledgements from a quorum of Followers before considering
a proposal committed. Also, Leader queues COMMIT(zxid) events to all Followers
so that all other nodes learn the events. This ensures that the events will be
reached to all nodes in the system. Adding one more point, in general ZK
provides strong ordering guarantees.
Sometime back ZooKeeper folks initiated discussions to decouple ZAB from
ZooKeeper, so that users can make use of this and can define their own models
and reliably replicate the data. There is a related JIRA ZOOKEEPER-1931 talks
similar feature, now this is in initial dev stage. Please have a look at this.
I hope this would help to define a common interface, also an opportunity for us
to know more about the use cases.
Regards,
Rakesh
> Introduce Coordination Engine
> -----------------------------
>
> Key: HADOOP-10641
> URL: https://issues.apache.org/jira/browse/HADOOP-10641
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Attachments: HADOOP-10641.patch, HADOOP-10641.patch,
> HADOOP-10641.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of
> events in a distributed system. In order to be reliable CE should be
> distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC,
> zab) and have different implementations, depending on use cases, reliability,
> availability, and performance requirements.
> CE should have a common API, so that it could serve as a pluggable component
> in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and
> HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
