[
https://issues.apache.org/jira/browse/HADOOP-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084282#comment-17084282
]
Wei-Chiu Chuang edited comment on HADOOP-16579 at 4/15/20, 5:52 PM:
--------------------------------------------------------------------
I don't think we (at Cloudera) tested Hadoop 3.3 on ZK 3.4 ensemble. Sorry in
our set up we update ZK servers to 3.5 before updating the Curator in Hadoop.
In HADOOP-16763, it seems the problem is that both ZK 3.4 and 3.5 jars are in
the classpath. It's not in the build time classpath. I've checked it via {{mvn
dependency:tree}}. So must be a runtime-only problem.
was (Author: jojochuang):
I don't think we tested Hadoop 3.3 on ZK 3.4 ensemble. Sorry in our set up we
update ZK servers to 3.5 before updating the Curator in Hadoop.
In HADOOP-16763, it seems the problem is that both ZK 3.4 and 3.5 jars are in
the classpath. It's not in the build time classpath. I've checked it via \{{mvn
dependency:tree}}. So must be a runtime-only problem.
> Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.6 in Hadoop
> -------------------------------------------------------------
>
> Key: HADOOP-16579
> URL: https://issues.apache.org/jira/browse/HADOOP-16579
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Mate Szalay-Beko
> Assignee: Norbert Kalmár
> Priority: Major
> Fix For: 3.3.0
>
>
> *Update:* the original idea was to only update Curator but keep the old
> ZooKeeper version in Hadoop. However, we encountered some run-time
> backward-incompatibility during unit tests with Curator 4.2.0 and ZooKeeper
> 3.5.5. We haven't really investigated deeply these issues, but upgraded to
> ZooKeeper 3.5.5 (and later to 3.5.6). We had to do some minor fixes in the
> unit tests (and also had to change some deprecated Curator API calls), but
> [the latest PR|https://github.com/apache/hadoop/pull/1656] seems to be stable.
> ZooKeeper 3.5.6 just got released during our work. (I think the official
> announcement will get out maybe tomorrow, but it is already available in
> maven central or on the [Apache ZooKeeper ftp
> site|https://www-eu.apache.org/dist/zookeeper/]). It is considered to be a
> stable version, contains some minor fixes and improvements, plus some CVE
> fixes. See the [release
> notes|https://github.com/apache/zookeeper/blob/branch-3.5.6/zookeeper-docs/src/main/resources/markdown/releasenotes.md].
>
> ----
> Currently in Hadoop we are using [ZooKeeper version
> 3.4.13|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L90].
> ZooKeeper 3.5.5 is the latest stable Apache ZooKeeper release. It contains
> many new features (including SSL related improvements which can be very
> important for production use; see [the release
> notes|https://zookeeper.apache.org/doc/r3.5.5/releasenotes.html]).
> Apache Curator is a high level ZooKeeper client library, that makes it easier
> to use the low level ZooKeeper API. Currently [in Hadoop we are using Curator
> 2.13.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/hadoop-project/pom.xml#L91]
> and [in Ozone we use Curator
> 2.12.0|https://github.com/apache/hadoop/blob/7f9073132dcc9db157a6792635d2ed099f2ef0d2/pom.ozone.xml#L146].
> Curator 2.x is supporting only the ZooKeeper 3.4.x releases, while Curator
> 3.x is compatible only with the new ZooKeeper 3.5.x releases. Fortunately,
> the latest Curator 4.x versions are compatible with both ZooKeeper 3.4.x and
> 3.5.x. (see [the relevant Curator
> page|https://curator.apache.org/zk-compatibility.html]). Many Apache projects
> have already migrated to Curator 4 (like HBase, Phoenix, Druid, etc.), other
> components are doing it right now (e.g. Hive).
> *The aims of this task are* to:
> - change Curator version in Hadoop to the latest stable 4.x version
> (currently 4.2.0)
> - also make sure we don't have multiple ZooKeeper versions in the classpath
> to avoid runtime problems (it is
> [recommended|https://curator.apache.org/zk-compatibility.html] to exclude the
> ZooKeeper which come with Curator, so that there will be only a single
> ZooKeeper version used runtime in Hadoop)
> In this ticket we still don't want to change the default ZooKeeper version in
> Hadoop, we only want to make it possible for the community to be able to
> build / use Hadoop with the new ZooKeeper (e.g. if they need to secure the
> ZooKeeper communication with SSL, what is only supported in the new ZooKeeper
> version). Upgrading to Curator 4.x should keep Hadoop to be compatible with
> both ZooKeeper 3.4 and 3.5.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
