Re: Resetting offsets

2017-05-04 Thread Dana Powers
kafka-python, yes. On May 4, 2017 2:28 AM, "Paul van der Linden" wrote: Thanks everyone. @Dana is that using the kafka-python library? On Thu, May 4, 2017 at 4:52 AM, Dana Powers wrote: > Requires stopping your existing consumers, but otherwise should work: > > from kafka import KafkaConsumer

Re: Resetting offsets

2017-05-04 Thread Paul van der Linden
Thanks everyone. @Dana is that using the kafka-python library? On Thu, May 4, 2017 at 4:52 AM, Dana Powers wrote: > Requires stopping your existing consumers, but otherwise should work: > > from kafka import KafkaConsumer, TopicPartition, OffsetAndMetadata > > def reset_offsets(group_id, topic,

Re: Resetting offsets

2017-05-03 Thread Dana Powers
Requires stopping your existing consumers, but otherwise should work: from kafka import KafkaConsumer, TopicPartition, OffsetAndMetadata def reset_offsets(group_id, topic, bootstrap_servers): consumer = KafkaConsumer(bootstrap_servers=bootstrap_servers, group_id=group_id) consumer.assign([Top

Re: Resetting offsets

2017-05-03 Thread Ben Stopford
Hu is correct, there isn't anything currently, but there is an active proposal: https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+Reset+Consumer+Group+Offsets+tooling On Wed, May 3, 2017 at 1:23 PM Hu Xi wrote: > Seems there is no command line out of box, but if you could write a

Re: Resetting Offsets

2015-02-18 Thread Suren
Reading offsets looks like it's compatible across 0.8.1 and 0.8.2. However, we cannot use the update logic in ImportZkOffsets, since we want to store offsets in the broker in 0.8.2. It looks like SimpleConsumer.commitOffsets() would work with either version. Is there a better way? -Suren

Re: Resetting Offsets

2015-02-18 Thread Michal Michalski
See https://cwiki.apache.org/confluence/display/KAFKA/System+Tools and check the following: GetOffsetShell (not very accurate - will set your offsets to much smaller values than you really need; we log offsets frequently in application logs and get it from there) ImportZkOffsets Kind regards, Mich