LakshSingla commented on issue #17139: URL: https://github.com/apache/druid/issues/17139#issuecomment-2373851056
I am still reviewing the changes, so it might be moot once I complete the review: > they do not need meaningful CPU or memory resources beyond what is needed to gather partition statistics for global sorts The controller can consume up to a maximum of 300MB of heap space for the collection of statistics, and this maximum seems a lot given that it will be per query, if compared with the default value of something like `maxSubqueryBytes`. For reference, a broker with 20 GB of heap space, and 50 max concurrent queries and no lookups would allocate 0.5 * 1/50 * (20GB) of heap space per query for inlining results which is 200MB, while each Dart query can theoretically take up to 300MBs. This parity will be more for brokers with smaller heap sizes. Do we require some limiting for the brokers at the moment (like we do with subqueries) or would we take this up once we start tuning concurrency? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
