jtuglu-netflix commented on PR #18148: URL: https://github.com/apache/druid/pull/18148#issuecomment-2979089944
> A higher-level comment is that we shouldn't just make this change without some confidence that our solution makes lives better for a good number of use cases. You should first build a test setup that can be used to simulate query congestion at data level along with metrics that reflect the degree of the congestion, throughput, fairness. Once such a system is in place, thats when you can craft few strategies and using your test setup to measure what strategy is the best. Yes, of course – we've observed this issue in our largest cluster numerous times (daily) where larger, low-priority queries beat smaller, shorter queries to the processing threads. This causes backup when processing each segment takes a while (on the order of mins in some extreme cases). By default this would be turned off, but it gives us an extra lever to turn to reduce the impact of these heavy-hitter queries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
