Hi Ahmed,
Thanks for the additional information about the current assignment strategy
problem that I had overlooked. Initially, I thought that Kafka alone was
not sufficient for the flip, but now that you've broadened the scope, I am
excited to work on it for the common Flink source interfaces.
I
Hi Hongshun,
I personally see this as a common challenge for source connectors, it might
be a good FLIP candidate for the common flink-source interfaces.
I still see a couple of challenges that might be covered in the FLIP, for
example currently the Kafka source doesn't wait for reader registratio
Hi Ahmed,
I want to do two things in this flip.
1. Add current assignments to ReaderRegistrationEvent. Thus, enumerator can
know the overall assignments after restarting the jon.
2. Add an distribution strategy interface to the kafka enumerator, thus can
flexibly choose how to assign the split base
Hi Ahmed,
Thanks for your question.
> Do you intend to keep the current assignment in the enumerator
state or are you only going to use per session assignment status and keep
it in memory?
Not keep in state, because it's be different with the real distribution
when redistribute state if the parre
Hi Hongshun
There is currently a Jira tracking this issue [1].
>Therefore, I want the enumerator to select a reader based on the current
assignment situation. Each time, the enumerator should choose the reader
whose assigned split number is the smallest.
I am not sure I follow how you intend to
Hi devs,
I'd like to push a new FLIP xxx: Kafka Enumerator assigns splits based on
the current assignment situation. Currently, when the Kafka enumerator
finds a new split, it will choose a reader using the following algorithm:
```java
static int getSplitOwner(TopicPartition tp, int numReaders)
