[
https://issues.apache.org/jira/browse/FLINK-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-18449:
----------------------------
Description:
In streaming api, we can use regex to find topic and enable partiton discovery
by setting non-negative value for
`{{flink.partition-discovery.interval-millis}}`. However, it's not work in
table api. We propose the following new connector options:
{code}
'topic' = 'topic' // for single topic
'topic' = 'topic-1, topic-2,..., topic-n', // for list topic
'topic-pattern' = 'topic*' // for use of regex, "topic"
and "topic-pattern" can't coexist
'scan.topic-partition-discovery.interval' = '5s' // for both topic discovery
{code}
was:In streaming api, we can use regex to find topic and enable partiton
discovery by setting non-negative value for
`{{flink.partition-discovery.interval-millis}}`. However, it's not work in
table api. I think we can add options such as 'topic-regex' and
'{{partition-discovery.interval-millis}}' in WITH block for users.
> Make topic discovery and partition discovery configurable for
> FlinkKafkaConsumer in Table API
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-18449
> URL: https://issues.apache.org/jira/browse/FLINK-18449
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka, Table SQL / API
> Affects Versions: 1.11.0
> Reporter: Shengkai Fang
> Assignee: Shengkai Fang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> In streaming api, we can use regex to find topic and enable partiton
> discovery by setting non-negative value for
> `{{flink.partition-discovery.interval-millis}}`. However, it's not work in
> table api. We propose the following new connector options:
> {code}
> 'topic' = 'topic' // for single topic
> 'topic' = 'topic-1, topic-2,..., topic-n', // for list topic
> 'topic-pattern' = 'topic*' // for use of regex,
> "topic" and "topic-pattern" can't coexist
> 'scan.topic-partition-discovery.interval' = '5s' // for both topic discovery
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
