Re: Backfilling From Kafka Topics With Varied Ingestion Rates

2020-02-12 Thread Robert Metzger
You can find more information here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface On Wed, Feb 12, 2020 at 11:30 AM Akshay Aggarwal < akshay.aggar...@flipkart.com> wrote: > Thanks Aljoscha. Is there a JIRA where this is getting tracked? > > ~Akshay > > On W

Re: Backfilling From Kafka Topics With Varied Ingestion Rates

2020-02-12 Thread Akshay Aggarwal
Thanks Aljoscha. Is there a JIRA where this is getting tracked? ~Akshay On Wed, Feb 12, 2020 at 1:56 PM Aljoscha Krettek wrote: > Hi, > > I'm afraid your analysis is 100% correct. Currently there's no > out-of-box feature for dealing with this but our work on a new source > interface ([1]) will

Re: Backfilling From Kafka Topics With Varied Ingestion Rates

2020-02-12 Thread Aljoscha Krettek
Hi, I'm afraid your analysis is 100% correct. Currently there's no out-of-box feature for dealing with this but our work on a new source interface ([1]) will enable us to add a feature that we call "event-time alignment" where source readers would slow down reading from certain source partiti