This is an automated email from the ASF dual-hosted git repository.
gwphua pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new b5e24cc09e0 Documentation in Basic Cluster Tuning using Group By
Metrics (#19083)
b5e24cc09e0 is described below
commit b5e24cc09e020c3a6b15949691db24ccc9dc5e11
Author: Virushade <[email protected]>
AuthorDate: Sun Mar 8 01:25:02 2026 +0800
Documentation in Basic Cluster Tuning using Group By Metrics (#19083)
* Docs for tuning groupby
* Docs
* Update docs/operations/basic-cluster-tuning.md
Co-authored-by: aho135 <[email protected]>
* SpilledQueries docs
* Update basic-cluster-tuning.md
Co-authored-by: aho135 <[email protected]>
---------
Co-authored-by: aho135 <[email protected]>
---
docs/operations/basic-cluster-tuning.md | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/docs/operations/basic-cluster-tuning.md
b/docs/operations/basic-cluster-tuning.md
index 4cbd9c964a7..51f8cb0f037 100644
--- a/docs/operations/basic-cluster-tuning.md
+++ b/docs/operations/basic-cluster-tuning.md
@@ -334,6 +334,25 @@ Non-nested GroupBy queries require 1 merge buffer per
query, while a nested Grou
The number of merge buffers determines the number of GroupBy queries that can
be processed concurrently.
+#### Using metrics to tune GroupBy buffer configuration
+
+Druid can emit metrics that help you right-size merge buffers and related
GroupBy configuration. These metrics are available when the
`GroupByStatsMonitor` module is enabled by adding
`org.apache.druid.server.metrics.GroupByStatsMonitor` to
`druid.monitoring.monitors`. See the [metrics reference](metrics.md) for full
details.
+
+##### Sizing `druid.processing.buffer.sizeBytes`
+
+- `mergeBuffer/maxBytesUsed`: peak merge buffer bytes used by any single
GroupBy query within the emission period. If this value consistently approaches
`druid.processing.buffer.sizeBytes`, consider increasing the buffer size.
+- `groupBy/maxSpilledBytes`: peak bytes spilled to disk by any single GroupBy
query. Non-zero values indicate that merge buffers are too small to hold
intermediate results in memory, causing disk spill. Increasing
`druid.processing.buffer.sizeBytes` reduces spilling. You can also adjust
`druid.query.groupBy.maxOnDiskStorage` to control how much spilling is allowed
before a query fails.
+- `groupBy/spilledQueries`: number of GroupBy queries spilled to disk within
the emission period. Non-zero values may indicate that your buffer size is too
small and should be increased to avoid performance issues caused by excessive
spilling.
+
+##### Sizing `druid.processing.numMergeBuffers`
+
+- `mergeBuffer/pendingRequests`: number of queries waiting to acquire merge
buffers. Persistently non-zero values indicate merge buffer pool exhaustion;
consider increasing `druid.processing.numMergeBuffers`.
+- `mergeBuffer/maxAcquisitionTimeNs`: peak time in nanoseconds that any single
GroupBy query waited to acquire merge buffers. High values suggest contention
on the merge buffer pool; increasing `druid.processing.numMergeBuffers` can
reduce wait times.
+
+##### Sizing `druid.query.groupBy.maxMergingDictionarySize`
+
+- `groupBy/maxMergeDictionarySize`: peak on-heap merge dictionary size in
bytes for any single GroupBy query. If this approaches
`druid.query.groupBy.maxMergingDictionarySize`, queries may spill to disk if
`druid.query.groupBy.maxOnDiskStorage` is configured or fail. If this happens,
consider increasing the dictionary size limit.
+
<a name="connection-pool"></a>
## Connection pool guidelines
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
