This strategy is working. It has been fixed. Thanks Simon and Casey. On Wed, Apr 5, 2017 at 11:33 PM, Ali Nazemian <alinazem...@gmail.com> wrote:
> Since Metron is the only consumer for the entire Kafka bus, I think I > should be able to set a low retention policy for the Kafka internal > "__consumer_offsets" topic to clear it. In addition to > the "__consumer_offsets" I guess, removing all of the consumers from the > zookeeper will help. > > On Wed, Apr 5, 2017 at 11:31 PM, Casey Stella <ceste...@gmail.com> wrote: > >> Sorry, misspoke. I meant to say "the new consumer API" not "consumer >> groups" in the previous email. >> >> On Wed, Apr 5, 2017 at 9:26 AM, Casey Stella <ceste...@gmail.com> wrote: >> >>> To my knowledge wiping the topic offsets in zookeeper through the zkcli >>> is the only way and yep, it's painful. It's unclear whether the move to >>> the new kafka spout which uses consumer groups that can be stored in kafka >>> rather than zookeeper will help in this. Someone who is more of a kafka >>> expert than me should speak, but my understanding is that should make >>> deleting topics less of a pain going forward. >>> >>> On Wed, Apr 5, 2017 at 5:27 AM, Ali Nazemian <alinazem...@gmail.com> >>> wrote: >>> >>>> Is there any way to reset the offset in an automatic way? It is very >>>> painful to set them one by one through zookeeper CLI! >>>> >>>> On Wed, Apr 5, 2017 at 7:04 PM, Ali Nazemian <alinazem...@gmail.com> >>>> wrote: >>>> >>>>> Hi Simon, >>>>> >>>>> Yeah. I should have applied the retention strategy that is much safer. >>>>> Do you think my current problem will be solved by resetting the offset for >>>>> those topics? How can I force the storm topologies to use the new topics? >>>>> >>>>> Cheers, >>>>> Ali >>>>> >>>>> On Wed, Apr 5, 2017 at 6:44 PM, Simon Elliston Ball < >>>>> si...@simonellistonball.com> wrote: >>>>> >>>>>> It's generally not a good idea to delete these topics, it would be >>>>>> better to lower retention, or change the offset rules to latest on the >>>>>> storm topologies and ignore the problematic messages until they age off >>>>>> naturally. >>>>>> >>>>>> If you really have to delete and purge the logs to recreate be >>>>>> careful about partitioning, and also make sure you bounce all the metron >>>>>> storm topologies to force connection against the new topics and offset >>>>>> reset. >>>>>> >>>>>> Simon >>>>>> >>>>>> On 5 Apr 2017, at 10:32, Ali Nazemian <alinazem...@gmail.com> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I have had some issue to clear the "enrichments" and "indexing" Kafka >>>>>> topics, so I have tried to delete and recreate these topics. I have had >>>>>> some troubles to clear those topics. I tried to remove those topics and >>>>>> create them again. Unfortunately, Kafka did not remove the topic >>>>>> gracefully >>>>>> so I tried to remove partitions from disk manually and remove the related >>>>>> config files from zookeeper by following this instruction: >>>>>> https://github.com/darrenfu/bigdata/issues/6 >>>>>> >>>>>> After I have managed to remove those topics, Metron has been broken >>>>>> and give me the following warning: >>>>>> >>>>>> 2017-04-05 18:28:37.063 o.a.s.k.KafkaUtils [WARN] >>>>>> Partition{host=gsoclabsn02.test.telstra.com:6667, topic=indexing, >>>>>> partition=10} Got fetch request with offset out of range: [1227] >>>>>> >>>>>> >>>>>> I can see the messages inside parser topics, but there is not message >>>>>> inside "enrichments" and "indexing" topics. Do you know what is the >>>>>> problem >>>>>> and how can I fix it? >>>>>> >>>>>> Thanks, >>>>>> Ali >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> A.Nazemian >>>>> >>>> >>>> >>>> >>>> -- >>>> A.Nazemian >>>> >>> >>> >> > > > -- > A.Nazemian > -- A.Nazemian
