Thanks, I was not aware of the last Kafka update in the area :)
On Wed, Jul 26, 2017 at 1:26 PM, Stig Rohde Døssing <[email protected]> wrote: > Hi. > > You should be able to do this with the storm-kafka-client spout and a > sufficiently new Kafka cluster (0.11, see the documentation at > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 122%3A+Add+Reset+Consumer+Group+Offsets+tooling). Kafka ships with the > kafka/bin/kafka-consumer-groups.sh script. You can use this script with > the --reset-offsets option to reset offsets for a consumer group based on a > bunch of conditions (timestamp, offset and others). > > Note that I haven't tried to do this, but I'd imagine you'd need to do > something like this: > * Ensure your kafka spout is configured to use the UNCOMMITTED_EARLIEST or > UNCOMMITTED_LATEST as its FirstPollOffsetStrategy. > * Find the consumer group your spout is running as. > * Stop (kill) the topology containing the spout you want to reset. You > need to do this to avoid having the spout commit offsets while you're > working with the kafka-consumer-groups tool. If you reset the offset while > the spout is running, it will ignore the reset. > * Use kafka-consumer-groups.sh to reset the offsets for your consumer > group. Reference the KIP link to see how to do this, or just run > kafka-consumer-groups.sh to get it to print usage. > * Redeploy your topology > > 2017-07-26 11:24 GMT+02:00 Elyahou Ittah <[email protected]>: > >> Is there an option to set the spout kafka offset to a specific offset ? >> >> It can be very usefull for replaying kafka log from specific time or >> offset. >> >> Thanks >> > >
