[
https://issues.apache.org/jira/browse/FLINK-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17569301#comment-17569301
]
JasonLee commented on FLINK-28595:
----------------------------------
I think Afzal Mazhar is right that we should put the filter forward and I'm
happy to fix it
Can someone assign this to me,thanks
> KafkaSource should not read metadata of unmatched regex topics
> --------------------------------------------------------------
>
> Key: FLINK-28595
> URL: https://issues.apache.org/jira/browse/FLINK-28595
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Reporter: Afzal Mazhar
> Priority: Minor
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When we use a regex to subscribe to topics, the current connector gets a list
> of all topics, then runs describe against all of them, and finally filters by
> the regex pattern. This is not performant, as well as could possibly trigger
> audit alarms against sensitive topics that do not match the regex.
> Proposed fix: move the regex filtering from the TopicPatternSubscriber's
> set() down into KafkaSubscriberUtils getAllTopicMetadata(). Get the list of
> topics, filter by pattern (if any), then get metadata. Create appropriate
> tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
