Afzal Mazhar created FLINK-28595:
------------------------------------
Summary: KafkaSource should not read metadata of unmatched regex
topics
Key: FLINK-28595
URL: https://issues.apache.org/jira/browse/FLINK-28595
Project: Flink
Issue Type: Improvement
Components: Connectors / Kafka
Reporter: Afzal Mazhar
When we use a regex to subscribe to topics, the current connector gets a list
of all topics, then runs describe against all of them, and finally filters by
the regex pattern. This is not performant, as well as could possibly trigger
audit alarms against sensitive topics that do not match the regex.
Proposed fix: move the regex filtering from the TopicPatternSubscriber's set()
down into KafkaSubscriberUtils getAllTopicMetadata(). Get the list of topics,
filter by pattern (if any), then get metadata. Create appropriate tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
