[
https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054589#comment-14054589
]
Konstantin Shvachko commented on HADOOP-10641:
----------------------------------------------
I was thinking about ATM's suggestion of taking this as a separate project some
place other than Hadoop. The main problem with that is cross dependencies.
'Coordination Engine' project depends on types from Hadoop common, and HDFS
will depend on the CE project. Since Common and HDFS are packaged together this
will be hard to break unless CE project is a part of Common. Currently the
interface uses Configuration, logging, and looking forward it will be rather
tightly integrated with Hadoop RPC layer, since many RPC calls should trigger
coordination.
I think the patch looks good now. +1
I see how the interfaces can be the base for HDFS and HBase coordination. I
also see that ZKCoordinationEngine needs more work. I think once we start using
it we will see where it should evolve.
I propose to commit this upon Jenkins approval and if there are no objections.
This should free up the implementation of coordination for HBase and HDFS.
> Introduce Coordination Engine
> -----------------------------
>
> Key: HADOOP-10641
> URL: https://issues.apache.org/jira/browse/HADOOP-10641
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Attachments: HADOOP-10641.patch, HADOOP-10641.patch,
> HADOOP-10641.patch, hadoop-coordination.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of
> events in a distributed system. In order to be reliable CE should be
> distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC,
> zab) and have different implementations, depending on use cases, reliability,
> availability, and performance requirements.
> CE should have a common API, so that it could serve as a pluggable component
> in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and
> HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
