[
https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020426#comment-14020426
]
Plamen Jeliazkov commented on HADOOP-10641:
-------------------------------------------
[~szetszwo], thanks for the review! It was nice to finally see a face at the
Summit as well. :)
[~atm], thanks for the comments! I think I am outside of that discussion, most
likely [~cos] or [~shv] can comment better on where to take the project.
I posted a new patch around the same time your review came in; there were
mistakes in the way agreement executions work.
* ProtoBuf is certainly a nice choice for serialization. However, we shouldn't
need to bind ourselves to any one serialization format. This is why we use
Serializable. It is certainly possible to have the writeObject call write out a
ProtoBuf of the proposal itself, for example, and read the values back using
ProtoBuf as well. This is feasible with the current interfaces.
* Good point on version compatibility. AFAIK, version compatibility would take
place once the quorum is established as prior to that there is no communication
between the engines. So the coordination engine, as part of bootstrap, should
perform a version check against its quorum peers. Perhaps this means extending
the API, or making it part of a larger interface?
(VersionedCoordinationEngine)? [~shv] might be able to comment better.
* Please see my new patch. The idea is indeed to make the agreement execute on
some callBack object, SampleLearner in this case. The new patch should show the
test making use of it.
* Yes we can probably do some refactoring here. I'll work on a new patch.
* Yes we can add details for ZkCoordinationEngine. Unsure of any clear
advantages and disadvantages. The only thing that comes to my mind right away
is that it may be possible to build Paxos directly into the CoordinationEngine
implementation, thus co-locating the coordination service with the server /
application itself, rather than having to make RPC calls and wait for
responses, like with ZooKeeper(s). I don't think the intent of this work is
really to compare any one coordination mechanism with another but so much as
provide a common interface for which one can implement whichever they prefer.
> Introduce Coordination Engine
> -----------------------------
>
> Key: HADOOP-10641
> URL: https://issues.apache.org/jira/browse/HADOOP-10641
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Attachments: HADOOP-10641.patch, HADOOP-10641.patch,
> HADOOP-10641.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of
> events in a distributed system. In order to be reliable CE should be
> distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC,
> zab) and have different implementations, depending on use cases, reliability,
> availability, and performance requirements.
> CE should have a common API, so that it could serve as a pluggable component
> in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and
> HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
