This would be the regular KafkaSpout, i.e.: new
KafkaSpout<>(KafkaSpoutConfig.builder("k10server:9092", "topic")
.setProp(p.getComponentProperties("KafkaConsumer")).setProcessingGuarantee(ProcessingGuarantee.AT_MOST_ONCE).build();The offsets end up in the standard old pre-v0.10 location: /consumers/ group.id/offsets/ On Fri, May 4, 2018 at 1:41 PM, Stig Rohde Døssing <[email protected]> wrote: > Hi, > > Are we talking about the regular KafkaSpout or the Trident spout? > > The regular KafkaSpout uses the KafkaConsumer class under the hood, and > commits offsets via the commitSync method https://kafka.apache.org/10/ > javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#commitSync- > java.util.Map-. Could you elaborate on what happens in your case, e.g. > where in Zookeeper are the offsets ending up? > > 2018-05-04 19:25 GMT+02:00 Eric Hokanson <[email protected]>: > >> We're working on upgrading our Storm cluster from v1.0.X to v1.2.1. >> We're taking advantage of this upgrade by moving to a newer Kafka v0.10.2 >> server from our older v0.8 server and using the built-in new Storm Kafka >> spout verses a custom Kafka spout we had before. We've got everything up >> and working now on Storm 1.2.1 except for the fact that Storm insists that >> the Kafka offsets should be written to Zookeeper instead of to Kafka like >> they should be on newer consumers. I've made sure we're using the latest >> 1.2.1 storm-kafka-client and I've tried various versions of the Kafka >> client including the latest v1.1.0 but I can't spot any place where this >> can be specified. What am I missing? >> >> -- >> Eric >> > > -- *Eric Hokanson* Sr Software Engineer | *Return Path* w | 303-999-3270 m | 970-412-2728 [email protected] <https://returnpath.sigstr.net/uc/58c1b1f3825be97ab9b7975f> [image: Powered by Sigstr] <https://returnpath.sigstr.net/uc/58c1b1f3825be97ab9b7975f/watermark>
