The stack trace you posted is from the storm-kafka-monitor component, which allows Storm UI to show how far behind the latest offset a Kafka spout is. As far as I know, the monitor runs in a separate JVM from your topology components, so that error log shouldn't prevent your topology from running normally, it'll just cause the offset lag display in Storm UI to not work.
2018-04-21 15:52 GMT+02:00 Yu Watanabe <[email protected]>: > Hello . > > I would like to ask question regarding to kafka spout setup for initial > topology deployment. My storm version is "1.2.1" and kafka version is > storm-kafka version is "1.1.0" > > I am getting below error when worker node is launched. > > Unable to get offset lags for kafka. Reason: > java.lang.IllegalArgumentException: > zk-node > '/kafka-cluster-1/brokers/topics/myfirsttopic/492864cd-f2d5-4f19-ac05-ff1a5a4ed165' > dose not exists. at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil. > getOldConsumerOffsetsFromZk(KafkaOffsetLagUtil.java:387) at > org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.getOffsetLags(KafkaOffsetLagUtil.java:268) > at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil. > main(KafkaOffsetLagUtil.java:124) > > I have referenced online document and set "ignoreZkOffsets" to true. > > http://storm.apache.org/releases/1.2.1/storm-kafka.html > > Below is my code snippet. > > // Build zookeeper instance > BrokerHosts hosts = new ZkHosts( zkConnString, brokerZkPath ); > > // Build configuration instance for Spout > SpoutConfig spoutConfig = new SpoutConfig( hosts, topicName, zkRoot + > "/" + topicName , UUID.randomUUID().toString() ); > > spoutConfig.ignoreZkOffsets = true; > > // Build Multischeme instance > spoutConfig.scheme = new SchemeAsMultiScheme( new StringScheme() ); > > // Build Kafka spout > KafkaSpout kafkaSpout = new KafkaSpout(spoutConfig); > > Do I need any additional configuration to stop storm looking into > Zookeeper for offsets at initial topology deployment? > > Best Regards, > Yu Watanabe > > -- > Yu Watanabe > > LinkedIn : jp.linkedin.com/in/yuwatanabe1 >
