Hi, I am working on branch 'master'. But since Kafka and zookeeper versions in 1.4 and in master branch are exactly the same, so I believe the issue exists for 1.4 as well. (let me know if anybody has tested and found no).
When I set up a multi-node container based cluster (./provisioner/docker/docker-hadoop.sh -c 3), I noticed this error, which is fatal to Kafka-server: "java.lang.NoClassDefFoundError:org/apache/zookeeper/AsyncCallback$MultiCallback " Logs are in below. Root cause of this problem is 'AsyncCallback MultiCallback' is defined in Zookeeper only since 3.4.7. However, Bigtop master (and 1.4) included Zookeeper 3.4.6. See here for similar discussion from Spark community: https://mail-archives.apache.org/mod_mbox/spark-reviews/201811.mbox/%[email protected]%3E Kafka version is upgraded to 2.1.1 in Bigtop 1.4. That's why I didn't such error in previous releases like Bigtop 1.3. In Bigtop 1.3, Kafka version is 0.10.2.2, and it runs well with Zookeeper 3.4.6. -Guodong == master branch or 1.4.0: Error: Could not start Service[kafka-server]: Execution of '/usr/bin/systemctl start kafka-server' returned 1: Job for kafka-server.service failed because the control process exited with error code. See "systemctl status kafka-server.service" and "journalctl -xe" for details. Wrapped exception: Execution of '/usr/bin/systemctl start kafka-server' returned 1: Job for kafka-server.service failed because the control process exited with error code. See "systemctl status kafka-server.service" and "journalctl -xe" for details. Error: /Stage[main]/Kafka::Server/Service[kafka-server]/ensure: change from stopped to running failed: Could not start Service[kafka-server]: Execution of '/usr/bin/systemctl start kafka-server' returned 1: Job for kafka-server.service failed because the control process exited with error code. See "systemctl status kafka-server.service" and "journalctl -xe" for details. [root@63f5c5a99d8d /]# systemctl status kafka-server.service ● kafka-server.service - LSB: Kafka Server Loaded: loaded (/etc/rc.d/init.d/kafka-server; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Mon 2019-04-15 03:48:45 UTC; 1min 29s ago Docs: man:systemd-sysv-generator(8) Apr 15 03:48:42 63f5c5a99d8d systemd[1]: Starting LSB: Kafka Server... Apr 15 03:48:42 63f5c5a99d8d kafka-server[9344]: Starting Kafka Server (kafka-server):[ OK ] Apr 15 03:48:42 63f5c5a99d8d kafka-server[9344]: Starting (kafka-server): Apr 15 03:48:42 63f5c5a99d8d su[9370]: (to kafka) root on none Apr 15 03:48:45 63f5c5a99d8d kafka-server[9344]: Failure to start Kafka Server (kafka-server). Return value: 1[FAILED] Apr 15 03:48:45 63f5c5a99d8d systemd[1]: kafka-server.service: control process exited, code=exited status=1 Apr 15 03:48:45 63f5c5a99d8d systemd[1]: Failed to start LSB: Kafka Server. Apr 15 03:48:45 63f5c5a99d8d systemd[1]: Unit kafka-server.service entered failed state. Apr 15 03:48:45 63f5c5a99d8d systemd[1]: kafka-server.service failed. = /var/log/kafka/kafka-server.out java.lang.NoClassDefFoundError: org/apache/zookeeper/AsyncCallback$MultiCallback at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1693) at kafka.server.KafkaServer.kafka$server$KafkaServer$$createZkClient$1(KafkaServer.scala:348) at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:372) at kafka.server.KafkaServer.startup(KafkaServer.scala:202) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38) at kafka.Kafka$.main(Kafka.scala:75) at kafka.Kafka.main(Kafka.scala) Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.AsyncCallback$MultiCallback at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 more [2019-04-15 03:55:24,859] INFO shutting down (kafka.server.KafkaServer) [2019-04-15 03:55:24,865] INFO shut down completed (kafka.server.KafkaServer) [2019-04-15 03:55:24,865] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable) [2019-04-15 03:55:24,868] INFO shutting down (kafka.server.KafkaServer)
