jtuglu-netflix commented on issue #18008: URL: https://github.com/apache/druid/issues/18008#issuecomment-2915293457
> Perhaps reading from multiple topics in a single supervisor was more straightforward. Regarding this, I think that was a much simpler issue since it was still only reading from a single consumer (single region). The consumer can already natively handle reading from multi-topic partitions, so it was a matter of just plugging into that functionality. The difference here is we have N sets of consumers, so the ingest lag, throughput values, etc. differ across each consumer. While the draft I raised above works well if task counts are inflated enough, with the current task scaling scheme, you can run into asymmetrical lag where you have a laggard region. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
