Hi, Are we talking about the regular KafkaSpout or the Trident spout?
The regular KafkaSpout uses the KafkaConsumer class under the hood, and commits offsets via the commitSync method https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#commitSync-java.util.Map-. Could you elaborate on what happens in your case, e.g. where in Zookeeper are the offsets ending up? 2018-05-04 19:25 GMT+02:00 Eric Hokanson <[email protected]>: > We're working on upgrading our Storm cluster from v1.0.X to v1.2.1. We're > taking advantage of this upgrade by moving to a newer Kafka v0.10.2 server > from our older v0.8 server and using the built-in new Storm Kafka spout > verses a custom Kafka spout we had before. We've got everything up and > working now on Storm 1.2.1 except for the fact that Storm insists that the > Kafka offsets should be written to Zookeeper instead of to Kafka like they > should be on newer consumers. I've made sure we're using the latest 1.2.1 > storm-kafka-client and I've tried various versions of the Kafka client > including the latest v1.1.0 but I can't spot any place where this can be > specified. What am I missing? > > -- > Eric >
