clintropolis commented on code in PR #20273:
URL: https://github.com/apache/druid/pull/20273#discussion_r4075368940
##########
processing/src/main/java/org/apache/druid/java/util/http/client/pool/ResourcePool.java:
##########
@@ -355,6 +612,235 @@ public void close()
}
}
+ private static class AdaptiveResourceHolderPerKey<K, V> extends
PooledResources<V>
+ {
+ private static final int CREATE_ATTEMPTS = 3;
Review Comment:
what do you think about making this configurable? I'm sort of worried about
this amplifying problems a sad historical might be having by retrying
connections in exchange for being to handle unlikely blips.
I think we could consider removing the old strategies if this was
configurable since overall this mode seems better in every way and the only
major difference is this retrying, so if it could be turned off then I think
there is low value in retaining the old strategies. That said, I think it would
be fine to leave the old strategies for a release or 2 too just in case.
##########
docs/operations/metrics.md:
##########
@@ -553,6 +553,26 @@ These metrics are emitted when
`druid.auth.emitAuthMetrics` is set to `true`.
|`segment/rowCount/avg`| The average number of rows per segment on a
historical. `SegmentStatsMonitor` must be enabled.| `dataSource`, `tier`,
`priority`|Varies. See [segment
optimization](../operations/segment-optimization.md) for guidance on optimal
segment sizes. |
|`segment/rowCount/range/count`| The number of segments in a bucket.
`SegmentStatsMonitor` must be enabled.| `dataSource`, `tier`, `priority`,
`range`|Varies|
+### HTTP client connection pools
+
+These metrics are only available if the `HttpClientPoolMonitor` module is
included in `druid.monitoring.monitors`.
+They cover the connection pools that Druid services use to talk to each other,
one emission per remote end. The
+`server` dimension is that remote end, and the `httpClient` dimension names
the client that pools the connections to
+it: `client` and `escalatedClient` are configured by `druid.broker.http`,
`global` and `escalatedGlobal` by
+`druid.global.http`. A remote end that a service has stopped talking to keeps
being reported, with zeroes, until the
+service is restarted.
+
+|Metric|Description|Dimensions|Normal value|
+|------|-----------|----------|------------|
+|`httpClient/pool/opened`|Number of connections opened.|`httpClient`,
`server`|Varies. Steady churn on an idle cluster points at connections being
discarded too eagerly.|
+|`httpClient/pool/closed`|Number of connections closed, whether they were
broken, unused for too long, or surplus.|`httpClient`, `server`|Varies|
+|`httpClient/pool/errored`|Number of failures while opening, health checking,
or closing a connection.|`httpClient`, `server`|0|
+|`httpClient/pool/timedOut`|Number of connections discarded for being unused
longer than `unusedConnectionTimeout`.|`httpClient`, `server`|Varies|
+|`httpClient/pool/taken`|Number of connections handed to a caller, that is,
the number of requests that got a connection.|`httpClient`, `server`|Varies|
+|`httpClient/pool/returned`|Number of connections given back by a
caller.|`httpClient`, `server`|Close to `httpClient/pool/taken`|
+|`httpClient/pool/used`|Number of connections in the hands of callers at the
time of the emission, that is, the requests in flight to that remote end. A
level, not a per period count.|`httpClient`, `server`|<=
`druid.<service>.http.numConnections`. Sitting at that ceiling means requests
are waiting for a connection.|
Review Comment:
ci website build doesn't seem to like the "<=" in the last column
https://github.com/apache/druid/actions/runs/35212764228/job/105173968390?pr=20273#step:7:104
presumably the line that follows would also have the same problem since it
has the same pattern
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
