keith-turner commented on issue #316: Add support for ZooKeeper 3.5 URL: https://github.com/apache/fluo-muchos/pull/316#issuecomment-581618750 I took a look at what is in the zookeeper 3.4 and 3.5 dirs and found found the following. The contents of ZK/lib differs a lot between 3.4 and 3.5. ```bash $ ls zookeeper-3.4.14/**/*.jar zookeeper-3.4.14/dist-maven/zookeeper-3.4.14.jar zookeeper-3.4.14/lib/audience-annotations-0.5.0.jar zookeeper-3.4.14/lib/slf4j-api-1.7.25.jar zookeeper-3.4.14/dist-maven/zookeeper-3.4.14-javadoc.jar zookeeper-3.4.14/lib/jline-0.9.94.jar zookeeper-3.4.14/lib/slf4j-log4j12-1.7.25.jar zookeeper-3.4.14/dist-maven/zookeeper-3.4.14-sources.jar zookeeper-3.4.14/lib/log4j-1.2.17.jar zookeeper-3.4.14/dist-maven/zookeeper-3.4.14-tests.jar zookeeper-3.4.14/lib/netty-3.10.6.Final.jar $ ls apache-zookeeper-3.5.6-bin/**/*.jar apache-zookeeper-3.5.6-bin/lib/audience-annotations-0.5.0.jar apache-zookeeper-3.5.6-bin/lib/log4j-1.2.17.jar apache-zookeeper-3.5.6-bin/lib/commons-cli-1.2.jar apache-zookeeper-3.5.6-bin/lib/netty-buffer-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jackson-annotations-2.9.10.jar apache-zookeeper-3.5.6-bin/lib/netty-codec-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jackson-core-2.9.10.jar apache-zookeeper-3.5.6-bin/lib/netty-common-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jackson-databind-2.9.10.jar apache-zookeeper-3.5.6-bin/lib/netty-handler-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/javax.servlet-api-3.1.0.jar apache-zookeeper-3.5.6-bin/lib/netty-resolver-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jetty-http-9.4.17.v20190418.jar apache-zookeeper-3.5.6-bin/lib/netty-transport-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jetty-io-9.4.17.v20190418.jar apache-zookeeper-3.5.6-bin/lib/netty-transport-native-epoll-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jetty-security-9.4.17.v20190418.jar apache-zookeeper-3.5.6-bin/lib/netty-transport-native-unix-common-4.1.42.Final.jar apache-zookeeper-3.5.6-bin/lib/jetty-server-9.4.17.v20190418.jar apache-zookeeper-3.5.6-bin/lib/slf4j-api-1.7.25.jar apache-zookeeper-3.5.6-bin/lib/jetty-servlet-9.4.17.v20190418.jar apache-zookeeper-3.5.6-bin/lib/slf4j-log4j12-1.7.25.jar apache-zookeeper-3.5.6-bin/lib/jetty-util-9.4.17.v20190418.jar apache-zookeeper-3.5.6-bin/lib/zookeeper-3.5.6.jar apache-zookeeper-3.5.6-bin/lib/jline-2.11.jar apache-zookeeper-3.5.6-bin/lib/zookeeper-jute-3.5.6.jar apache-zookeeper-3.5.6-bin/lib/json-simple-1.1.1.jar ``` It seems both 3.4 and 3.5 have a lib dir, but we are not currently adding ZK/lib in accumulo-env.sh. So I suspect the ZK 3.4 dependencies are currently supplied elsewhere on the classpath. @karthick-rn suggested possibly always adding ZK/lib to the classpath, does anyone know if this works? > I prefer the latter, simply because it won't change the CLASSPATH & any previous Zookeeper versions would still work. Are there good reasons to support ZK 3.4 and 3.5? If not we always have the option of making ZK 3.5 the minimum supported version of ZK for muchos (we can add a sanity check during setup). If this makes maintaining muchos simpler, it may be a good way to go. > We may also have to copy these 2 zookeeper*.jar files to hadoop common lib & hadoop hdfs lib to make sure the same version of Zookeeper is being picked across the cluster. This may not be an issue for Accumulo. I think the order in which items are listed on the classpath is important for deciding which classes are used (see specification order section at the end of [this doc](https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html)). So if two ZK versions are listed on the classpath, the one listed first will be chosen. So if accumulo-env.sh puts ZK_HOME first on the classpath, then maybe it will always use that version even if Hadoop has it. However, not sure if this holds for java 11. > I am also wondering if we should refine the addition of ${HADOOP_HOME}/share/hadoop/common/lib/* to the classpath and only include the specific jars required for ADLS Gen2. One tricky thing with that is when you change the version of Hadoop, the versions of deps will change and their transitive deps may also change.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
