curcur commented on pull request #11725: URL: https://github.com/apache/flink/pull/11725#issuecomment-629578740
Resolved 2. Avoid duplicated code 1. `KafkaShuffleFetcher` now extends `KafkaFetcher` instead of `AbstractFetcher` 2. I also did a little clean up on the code of `KafkaFetcher`. I know it is not quite relevant, but I guess you will think it is necessary as well. 3. The current "ending of stream" logic in KafkaFetcher a bit strange: if any partition has a record signaled as "END_OF_STREAM", the fetcher will stop running. Notice that the signal is coming from the deserializer, which means from Kafka data itself. But it is possible that other topics and partitions still have data to read. Finishing reading Partition0 can not guarantee that Partition1 also finishes. It is possible I misunderstand what an "END_OF_STREAM" signal means. 4. All Kafka related tests passed after the change. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
