Is there any way to reset the offset in an automatic way? It is very painful to set them one by one through zookeeper CLI!
On Wed, Apr 5, 2017 at 7:04 PM, Ali Nazemian <alinazem...@gmail.com> wrote: > Hi Simon, > > Yeah. I should have applied the retention strategy that is much safer. Do > you think my current problem will be solved by resetting the offset for > those topics? How can I force the storm topologies to use the new topics? > > Cheers, > Ali > > On Wed, Apr 5, 2017 at 6:44 PM, Simon Elliston Ball < > si...@simonellistonball.com> wrote: > >> It's generally not a good idea to delete these topics, it would be better >> to lower retention, or change the offset rules to latest on the storm >> topologies and ignore the problematic messages until they age off >> naturally. >> >> If you really have to delete and purge the logs to recreate be careful >> about partitioning, and also make sure you bounce all the metron storm >> topologies to force connection against the new topics and offset reset. >> >> Simon >> >> On 5 Apr 2017, at 10:32, Ali Nazemian <alinazem...@gmail.com> wrote: >> >> Hi all, >> >> I have had some issue to clear the "enrichments" and "indexing" Kafka >> topics, so I have tried to delete and recreate these topics. I have had >> some troubles to clear those topics. I tried to remove those topics and >> create them again. Unfortunately, Kafka did not remove the topic gracefully >> so I tried to remove partitions from disk manually and remove the related >> config files from zookeeper by following this instruction: >> https://github.com/darrenfu/bigdata/issues/6 >> >> After I have managed to remove those topics, Metron has been broken and >> give me the following warning: >> >> 2017-04-05 18:28:37.063 o.a.s.k.KafkaUtils [WARN] >> Partition{host=gsoclabsn02.test.telstra.com:6667, topic=indexing, >> partition=10} Got fetch request with offset out of range: [1227] >> >> >> I can see the messages inside parser topics, but there is not message >> inside "enrichments" and "indexing" topics. Do you know what is the problem >> and how can I fix it? >> >> Thanks, >> Ali >> >> > > > -- > A.Nazemian > -- A.Nazemian
