[
https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078724#comment-14078724
]
Konstantin Shvachko commented on HADOOP-10641:
----------------------------------------------
Steve, I am glad we are talking about the same thing now, which is the
CoordinationEngine interface.
# It seems that you cite HADOOP-9361 as an example of a specification of an
API. Same as Andrew I could not find specifications or any documents linked
there. May be you can clarify. Is it a TLA+ spec of HDFS?
# If FileSystem specs are done and the tests developed according to specs,
should that be sufficient to safeguard FS behaviour of any internal changes
including introduction of CE? Which I assume is your main concern with this
jira.
If I introduce an implementation of CE using ZK and it does not break the tests
and therefore does not alter FileSystem semantics, isn't that a verification of
the implementation.
# It looks like you propose to introduce a new requirement for Hadoop
contributions, that new features should be _formally specified and
mathematically proven_. I think this should be discussed in a separate thread
before it can be enforced. Would be good to hear what people think.
# Generally great minds were thinking about disciplined software development
techniques way back, probably starting from Dijkstra and Donald Knuth. I found
them very useful dealing with complex algorithms, not sure about APIs.
> Introduce Coordination Engine interface
> ---------------------------------------
>
> Key: HADOOP-10641
> URL: https://issues.apache.org/jira/browse/HADOOP-10641
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Attachments: HADOOP-10641.patch, HADOOP-10641.patch,
> HADOOP-10641.patch, hadoop-coordination.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of
> events in a distributed system. In order to be reliable CE should be
> distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC,
> zab) and have different implementations, depending on use cases, reliability,
> availability, and performance requirements.
> CE should have a common API, so that it could serve as a pluggable component
> in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and
> HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
