kfaraz commented on code in PR #14828: URL: https://github.com/apache/druid/pull/14828#discussion_r1295753660
########## docs/development/extensions-core/kafka-supervisor-reference.md: ########## @@ -138,11 +138,20 @@ The following example demonstrates supervisor spec with `lagBased` autoScaler an } ``` ## Ingesting from multiple topics -To ingest from multiple topics, you have to set `multiTopic` in the supervisor IO config to `true`. Multiple topics -can be passed as a regex pattern as the value for `topic` in the IO config. For example, to ingest data from clicks and -impressions, you will set `topic` to `clicks|impressions` in the IO config. If new topics are added to the cluster that -match the regex, druid will automatically start ingesting from those new topics. If you enable multi-topic -ingestion for a datasource, downgrading will cause the ingestion to fail for that datasource. + +To ingest data from multiple topics, you have to set `topicPattern` in the supervisor IO config and not set `topic`. +Multiple topics can be passed as a regex pattern as the value for `topicPattern` in the IO config. For example, to +ingest data from clicks and impressions, you will set `topicPattern` to `clicks|impressions` in the IO config. +Similarly, you can use `metrics-.*` as the value for `topicPattern` if you want to ingest from all the topics that +start with `metrics-`. If new topics are added to the cluster that match the regex, Druid will automatically start +ingesting from those new topics. If you enable multi-topic ingestion for a datasource, downgrading to a version +lesser than 28.0.0 will cause the ingestion for that datasource to fail. + +When ingesting data from multiple topics, the partitions are assigned based on the hashcode of topic and the id of the Review Comment: ```suggestion When ingesting data from multiple topics, the partitions are assigned based on the hashcode of the topic name and the id of the ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
