Hello . I would like to ask question regarding to kafka spout setup for initial topology deployment. My storm version is "1.2.1" and kafka version is storm-kafka version is "1.1.0"
I am getting below error when worker node is launched. Unable to get offset lags for kafka. Reason: java.lang.IllegalArgumentException: zk-node '/kafka-cluster-1/brokers/topics/myfirsttopic/492864cd-f2d5-4f19-ac05-ff1a5a4ed165' dose not exists. at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.getOldConsumerOffsetsFromZk(KafkaOffsetLagUtil.java:387) at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.getOffsetLags(KafkaOffsetLagUtil.java:268) at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.main(KafkaOffsetLagUtil.java:124) I have referenced online document and set "ignoreZkOffsets" to true. http://storm.apache.org/releases/1.2.1/storm-kafka.html Below is my code snippet. // Build zookeeper instance BrokerHosts hosts = new ZkHosts( zkConnString, brokerZkPath ); // Build configuration instance for Spout SpoutConfig spoutConfig = new SpoutConfig( hosts, topicName, zkRoot + "/" + topicName , UUID.randomUUID().toString() ); spoutConfig.ignoreZkOffsets = true; // Build Multischeme instance spoutConfig.scheme = new SchemeAsMultiScheme( new StringScheme() ); // Build Kafka spout KafkaSpout kafkaSpout = new KafkaSpout(spoutConfig); Do I need any additional configuration to stop storm looking into Zookeeper for offsets at initial topology deployment? Best Regards, Yu Watanabe -- Yu Watanabe LinkedIn : jp.linkedin.com/in/yuwatanabe1
