Leo Zhang created FLINK-14729:
---------------------------------
Summary: Multi-topics consuming from KafkaTableSource
Key: FLINK-14729
URL: https://issues.apache.org/jira/browse/FLINK-14729
Project: Flink
Issue Type: New Feature
Components: Connectors / Kafka
Reporter: Leo Zhang
Fix For: 1.10.0
Hi, all. I propose a new functionality of KafkaTableSource which can consume
multiple topics at the same time.
*Design plan*
* Add a new constructor in KafkaTableSource which accepts topics with List
type as one parameter.
* Modify the existed one which only accepts one topic as string type to call
the proposed one to finish the instantiation. That is to say, wrap this topic
in a list and pass it to the multi-topics-consuming constructor.
* Modify the overridden method createKafkaConsumer in KafkaTableSource to pass
topics as List instead of String.
* Replace the field topic with topics as List type in KafkaTableSourceBase
and modify every place using topic with topics. So we just need to modify the
constructor KafkaTableSourceBase, method getDataStream, and equals and hashCode.
*Test plan*
There is less to do as KafkaTableSource is based on FlinkKafkaConsumer which
already supports consuming multiple topics and is tested well. Of course, we
can easily add further more tests if needed.
So what's your opinion?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
