Hi all,
Thanks for your involvement. The benefits can be summarized here for a
multi-topics consuming via KafkaTableSource,
First, It boosts developing and managing a source with so many topics,
especially when the schema of topics is the same.
Second, It is aligned with existed FlinkKafkaConsumer and it is also
easy to achieve this goal just leveraging on the existed APIs.
I'll open a VOTE for this feature now and please join this.
Best,
Leo
On 11/27/19 3:12 PM, Gyula Fóra wrote:
Hi all,
I think the same reasoning applies here as with the regular Kafka consumer
sources.
Those support multiple topics, even topic patterns assuming the same schema.
Some times you have so many topics that you can't create a source for each
or as with the pattern the topics might be changing over time.
Cheers
Gyula
On Wed, Nov 27, 2019, 07:36 Jingsong Li <jingsongl...@gmail.com> wrote:
Hi leo:
Can you share why KafkaSource supports multiple topics?
Why can't we use multiple sources and then union them?
Best,
Jingsong Lee
On Wed, Nov 27, 2019 at 8:19 AM leo <leozhangs...@qq.com> wrote:
Hi,
As Flink does not own the data, so it can not forbid this. It's the
responsibility of user to deserialize messages from these topics with
custom schema deser.
On 11/26/19 10:55 AM, Jun Zhang wrote:
If the data formats of the Multi-topics are different, what do you
plan to do?
On 11/26/2019 08:53，leo<leozhangs...@qq.com>
<mailto:leozhangs...@qq.com> wrote：
Hi, all.
I want to propose an improvement of KafkaTableSource which can
consume
multiple topics at the same time.
Here is proposal,
https://issues.apache.org/jira/browse/FLINK-14729
Best,
Leo
--
Best, Jingsong Lee
