For the Kafka issue, I'm wondering if this helps? Also, please confirm you are using HDP 2.2 (specifically HDP 2.2.0.0)?
Thanks. ---- If you are managing a HDP 2.2 cluster that includes Kafka, you must adjust the Kafka configuration to send metrics to the Ambari Metrics system. From Ambari Web, browse to Services > Kafka > Configs and edit the kafka-env template found under Advanced kafka-env to include the following: : # Add kafka sink to classpath and related dependencies if [ -e "/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar" ]; then export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/* fi ________________________________________ From: Frank Eisenhauer <[email protected]> Sent: Saturday, April 18, 2015 8:32 AM To: Ambari User Subject: Ambari 2.0 - Storm not starting Hi All, are there any known incompatibilities between Ambari 2.0.0 and Kafka/Storm? Since the Update to Ambari 2.0 Kafka and Storm Services are failing on start. There are a lot of error entries in Storm nimbus.log: 2015-04-18 14:34:08 b.s.d.nimbus [ERROR] Error when processing event java.lang.NullPointerException: null at clojure.lang.Numbers.ops(Numbers.java:942) ~[clojure-1.5.1.jar:na] at clojure.lang.Numbers.isZero(Numbers.java:90) ~[clojure-1.5.1.jar:na] at backtype.storm.util$partition_fixed.invoke(util.clj:868) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.AFn.applyToHelper(AFn.java:163) [clojure-1.5.1.jar:na] at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na] at clojure.core$apply.invoke(core.clj:617) ~[clojure-1.5.1.jar:na] at clojure.lang.AFn.applyToHelper(AFn.java:163) [clojure-1.5.1.jar:na] at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[clojure-1.5.1.jar:na] at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na] at clojure.core$partial$fn__4190.doInvoke(core.clj:2396) ~[clojure-1.5.1.jar:na] at clojure.lang.RestFn.invoke(RestFn.java:408) ~[clojure-1.5.1.jar:na] at backtype.storm.util$map_val$iter__274__278$fn__279.invoke(util.clj:291) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na] at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na] at clojure.lang.Cons.next(Cons.java:39) ~[clojure-1.5.1.jar:na] at clojure.lang.RT.next(RT.java:598) ~[clojure-1.5.1.jar:na] at clojure.core$next.invoke(core.clj:64) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__6034.invoke(protocols.clj:146) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__6005$G__6000__6014.invoke(protocols.clj:19) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__6026.invoke(protocols.clj:54) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13) ~[clojure-1.5.1.jar:na] at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na] at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na] at backtype.storm.util$map_val.invoke(util.clj:290) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.daemon.nimbus$compute_executors.invoke(nimbus.clj:435) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.daemon.nimbus$compute_executor__GT_component.invoke(nimbus.clj:446) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:339) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.daemon.nimbus$mk_assignments$iter__6522__6526$fn__6527.invoke(nimbus.clj:665) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na] at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na] at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na] at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__6026.invoke(protocols.clj:54) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13) ~[clojure-1.5.1.jar:na] at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na] at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na] at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:664) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.RestFn.invoke(RestFn.java:410) ~[clojure-1.5.1.jar:na] at backtype.storm.daemon.nimbus$fn__6755$exec_fn__1142__auto____6756$fn__6761$fn__6762.invoke(nimbus.clj:994) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3$ at backtype.storm.daemon.nimbus$fn__6755$exec_fn__1142__auto____6756$fn__6761.invoke(nimbus.clj:993) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-$ at backtype.storm.timer$schedule_recurring$this__3798.invoke(timer.clj:99) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.timer$mk_timer$fn__3781$fn__3782.invoke(timer.clj:50) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.timer$mk_timer$fn__3781.invoke(timer.clj:42) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] 2015-04-18 14:34:08 b.s.util [ERROR] Halting process: ("Error when processing an event") java.lang.RuntimeException: ("Error when processing an event") at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:322) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na] at backtype.storm.daemon.nimbus$nimbus_data$fn__6128.invoke(nimbus.clj:85) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.timer$mk_timer$fn__3781$fn__3782.invoke(timer.clj:68) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at backtype.storm.timer$mk_timer$fn__3781.invoke(timer.clj:42) [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] 2015-04-18 14:34:08 b.s.d.nimbus [INFO] Shutting down master A manual start of kafka broker results in: [2015-04-18 14:41:03,964] FATAL (kafka.Kafka$) java.lang.ClassNotFoundException: org.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporter at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at kafka.utils.Utils$.createObject(Utils.scala:438) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:59) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:58) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34) at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:58) at kafka.Kafka$.main(Kafka.scala:36) at kafka.Kafka.main(Kafka.scala)
