Lucas Brutschy created KAFKA-20171:
--------------------------------------
Summary: Support regular expression topic patterns for streams
groups
Key: KAFKA-20171
URL: https://issues.apache.org/jira/browse/KAFKA-20171
Project: Kafka
Issue Type: Task
Components: streams, group-coordinator
Reporter: Lucas Brutschy
The streams rebalance protocol \(KIP\-1071\) does not currently support
pattern\-based topic subscription. Using regex patterns for source topics with
group.protocol=streams results in an UnsupportedOperationException stating
Pattern subscriptions are not supported with the STREAMS protocol \(see
KafkaStreamsTest.java:1877\). This limits the flexibility of streams
applications that need to dynamically consume from topics matching a pattern.
As specified in KIP\-1071, source topics should be specifiable using Google
RE2/J regular expression patterns. The broker should resolve regex patterns
against current topic metadata when processing heartbeats. If a regex matches
zero topics or required source topics are missing, the group should enter
NOT\_READY state with a MISSING\_SOURCE\_TOPICS status. The broker\-side data
model already includes fields for sourceTopicRegex in the topology records
\(see StreamsCoordinatorRecordHelpers.java\), but the actual resolution of
regex patterns to concrete topics and the corresponding client\-side changes
are not implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
