[
https://issues.apache.org/jira/browse/HADOOP-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Szalay-Beko updated HADOOP-16579:
--------------------------------------
Description:
Currently in Hadoop we are using [ZooKeeper version
3.4.13|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L90].
ZooKeeper 3.5.5 is the latest stable Apache ZooKeeper release. It contains
many new features (including SSL related improvements which can be very
important for production use; see [the release
notes|https://zookeeper.apache.org/doc/r3.5.5/releasenotes.html]).
Apache Curator is a high level ZooKeeper client library, that makes it easier
to use the low level ZooKeeper API. Currently [in Hadoop we are using Curator
2.13.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L91]
and [in Ozone we use Curator
2.12.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/pom.ozone.xml#L146].
Curator 2.x is supporting only the ZooKeeper 3.4.x releases, while Curator 3.x
is compatible only with the new ZooKeeper 3.5.x releases. Fortunately, the
latest Curator 4.x versions are compatible with both ZooKeeper 3.4.x and 3.5.x.
(see [the relevant Curator
page|https://curator.apache.org/zk-compatibility.html]). Many Apache projects
have already migrated to Curator 4 (like HBase, Phoenix, Druid, etc.), other
components are doing it right now (e.g. Hive).
*The aims of this task are* to:
- change Curator version in Hadoop to the latest stable 4.x version (currently
4.2.0)
- also make sure we don't have multiple ZooKeeper versions in the classpath to
avoid runtime problems (it is
[recommended|https://curator.apache.org/zk-compatibility.html] to exclude the
ZooKeeper which come with Curator, so that there will be only a single
ZooKeeper version used runtime in Hadoop)
In this ticket we still don't want to change the default ZooKeeper version in
Hadoop, we only want to make it possible for the community to be able to build
/ use Hadoop with the new ZooKeeper (e.g. if they need to secure the ZooKeeper
communication with SSL, what is only supported in the new ZooKeeper version).
Upgrading to Curator 4.x should keep Hadoop to be compatible with both
ZooKeeper 3.4 and 3.5.
was:
Currently in Hadoop we are using [ZooKeeper version
3.4.13|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L90].
ZooKeeper 3.5.5 is the latest stable Apache ZooKeeper release. It contains
many new features (including SSL related improvements which can be very
important for production use; see [the release
notes|https://zookeeper.apache.org/doc/r3.5.5/releasenotes.html]).
Apache Curator is a high level ZooKeeper client library, that makes it easier
to use the low level ZooKeeper API. Currently [in Hadoop we are using Curator
2.13.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L91]
and [in Ozone we use Curator
2.12.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/pom.ozone.xml#L146].
Curator 2.x is supporting only the ZooKeeper 3.4.x releases, while Curator 3.x
is compatible only with the new ZooKeeper 3.5.x releases. Fortunately, the
latest Curator 4.x versions are compatible with both ZooKeeper 3.4.x and 3.5.x.
(see [the relevant Curator
page|https://curator.apache.org/zk-compatibility.html]). Many Apache projects
have already migrated to Curator 4 (like HBase, Phoenix, Druid, etc.), other
components are doing it right now (e.g. Hive).
*The aims of this task are* to:
- change Curator version in Hadoop to the latest stable 4.x version (currently
4.2.0)
- also make sure we don't have multiple ZooKeeper versions in the classpath to
avoid runtime problems (it is
[recommended|https://curator.apache.org/zk-compatibility.html] to exclude the
ZooKeeper which come with Curator, so that there will be only a single
ZooKeeper version used in Hadoop)
In this ticket we still don't want to change the default ZooKeeper version in
Hadoop, we only want to make it possible for the community to be able to build
/ use Hadoop with the new ZooKeeper (e.g. if they need to secure the ZooKeeper
communication with SSL, what is only supported in the new ZooKeeper version).
Upgrading to Curator 4.x should keep Hadoop to be compatible with both
ZooKeeper 3.4 and 3.5.
> Upgrade to Apache Curator 4.2.0 in Hadoop
> -----------------------------------------
>
> Key: HADOOP-16579
> URL: https://issues.apache.org/jira/browse/HADOOP-16579
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Mate Szalay-Beko
> Priority: Major
>
> Currently in Hadoop we are using [ZooKeeper version
> 3.4.13|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L90].
> ZooKeeper 3.5.5 is the latest stable Apache ZooKeeper release. It contains
> many new features (including SSL related improvements which can be very
> important for production use; see [the release
> notes|https://zookeeper.apache.org/doc/r3.5.5/releasenotes.html]).
> Apache Curator is a high level ZooKeeper client library, that makes it easier
> to use the low level ZooKeeper API. Currently [in Hadoop we are using Curator
> 2.13.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L91]
> and [in Ozone we use Curator
> 2.12.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/pom.ozone.xml#L146].
> Curator 2.x is supporting only the ZooKeeper 3.4.x releases, while Curator
> 3.x is compatible only with the new ZooKeeper 3.5.x releases. Fortunately,
> the latest Curator 4.x versions are compatible with both ZooKeeper 3.4.x and
> 3.5.x. (see [the relevant Curator
> page|https://curator.apache.org/zk-compatibility.html]). Many Apache projects
> have already migrated to Curator 4 (like HBase, Phoenix, Druid, etc.), other
> components are doing it right now (e.g. Hive).
> *The aims of this task are* to:
> - change Curator version in Hadoop to the latest stable 4.x version
> (currently 4.2.0)
> - also make sure we don't have multiple ZooKeeper versions in the classpath
> to avoid runtime problems (it is
> [recommended|https://curator.apache.org/zk-compatibility.html] to exclude the
> ZooKeeper which come with Curator, so that there will be only a single
> ZooKeeper version used runtime in Hadoop)
> In this ticket we still don't want to change the default ZooKeeper version in
> Hadoop, we only want to make it possible for the community to be able to
> build / use Hadoop with the new ZooKeeper (e.g. if they need to secure the
> ZooKeeper communication with SSL, what is only supported in the new ZooKeeper
> version). Upgrading to Curator 4.x should keep Hadoop to be compatible with
> both ZooKeeper 3.4 and 3.5.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
