capistrant commented on code in PR #17892: URL: https://github.com/apache/druid/pull/17892#discussion_r2039618196
########## docs/operations/basic-cluster-tuning.md: ########## @@ -40,11 +40,10 @@ The biggest contributions to heap usage on Historicals are: - Partial unmerged query results from segments - The stored maps for [lookups](../querying/lookups.md). -A general rule-of-thumb for sizing the Historical heap is `(0.5GiB * number of CPU cores)`, with an upper limit of ~24GiB. +A general rule-of-thumb for sizing the Historical heap is `(0.5GiB * number of CPU cores)`. -This rule-of-thumb scales using the number of CPU cores as a convenient proxy for hardware size and level of concurrency (note: this formula is not a hard rule for sizing Historical heaps). - -Having a heap that is too large can result in excessively long GC collection pauses, the ~24GiB upper limit is imposed to avoid this. +This is a starting point, not a hard rule for sizing Historical heaps. +Note that with certain garbage collectors, having a large heap can result in excessively long GC pauses. For heaps larger than about 24GiB, we recommend using a collector that can handle large heaps, such as Shenandoah or ZGC. Review Comment: good to hear that about ZGC. I guess I'm not so concerned with pushing this out. Someone deploying open source druid and scaling historical heaps upwards of 100GiB sizes likely has the skills to navigate the best practices around choosing a collector and tuning it if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
