Re: Dynamically deleting kafka topics does not remove partitions from kafkaConsumer

Hello, I've being working on a fix for this, I posted more details on the JIRA ticket. Regards, Edward 2018-05-07 5:51 GMT+02:00 Tzu-Li (Gordon) Tai : > Ah, correct, sorry for the incorrect link. > Thanks Ted! > > > On 7 May 2018 at 11:43:12 AM, Ted Yu (yuzhih...@gmail.com)

Re: Dynamically deleting kafka topics does not remove partitions from kafkaConsumer

Ah, correct, sorry for the incorrect link. Thanks Ted! On 7 May 2018 at 11:43:12 AM, Ted Yu (yuzhih...@gmail.com) wrote: It seems the correct JIRA should be FLINK-9303 On Sun, May 6, 2018 at 8:29 PM, Tzu-Li (Gordon) Tai wrote: Hi Edward, Thanks for brining this up, and I

Re: Dynamically deleting kafka topics does not remove partitions from kafkaConsumer

It seems the correct JIRA should be FLINK-9303 On Sun, May 6, 2018 at 8:29 PM, Tzu-Li (Gordon) Tai wrote: > Hi Edward, > > Thanks for brining this up, and I think your suggestion makes sense. > The problem is that the Kafka

Re: Dynamically deleting kafka topics does not remove partitions from kafkaConsumer

Hi Edward, Thanks for brining this up, and I think your suggestion makes sense. The problem is that the Kafka consumer has no notion of "closed" partitions at the moment, so statically assigned partitions to the Kafka client is never removed and is always continuously requested for records. For

Dynamically deleting kafka topics does not remove partitions from kafkaConsumer

Hello all, We have a kafka consumer listening to a topic pattern "topic-*" with a partition discovery interval. We eventually add new topics and this is working perfectly, the consumer discover the new topics (and partitions) and listen to them. But we also remove topics eventually and in this