abhishekagarwal87 commented on issue #18008: URL: https://github.com/apache/druid/issues/18008#issuecomment-2915528554
It was indeed simpler to extend the single supervisor to read from multiple topics in the same cluster. You also get the benefit of auto-discovering new topics against a regex and don't have to make any changes to supervisor for ingestion to continue. With multi-supervisor approach, you would need to add one every time a new topic is added. That's ok when topics sit in different kafka clusters since you would need to pass on kafka cluster settings for the new topic anyway. IMO a multi-supervisor approach does seem cleaner than stuffing in various topic-related settings into the same supervisor. You can also choose to write a different supervisor altogether that knows how to consume from topics from multiple clusters, reusing what code you can reuse. The task logic will remain the same as it but the supervisor will be different. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
