[
https://issues.apache.org/jira/browse/SOLR-18616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18136840#comment-18136840
]
Chris M. Hostetter commented on SOLR-18616:
-------------------------------------------
I briefly looked into writing a customized, Otel, variant of basic logic in
dropwizard's InstrumentedEE10Handler, and quickly realized that the way otel
metrics have been implemented in Solr (and exposed in Solr's {{/admin/metrics}}
handler) creates a "chicken and egg" problem for any sort of metric
registration in customized jetty classes...
AFAICT there does not seem to be any OTel equivalent of the
{{com.codahale.metrics.SharedMetricRegistries}} global factory that the
dropwizard jetty plugins used in Solr-9. Even if there *IS* an equivilent,
Solr's {{/admin/metrics}} handler doesn't look for / inspect any "global"
metric registries to return all their metrics – it only knows about the metric
registries managed by the {{SolrMetricManager}} (which seems to be why even
things like {{OtelRuntimeJvmMetrics}} are solr specific: initing
{{RuntimeTelemetryBuilder}} using a custom {{OpenTelemetry}} subclass that
hooks back into the {{{}SolrMetricManager{}}}.
But since {{SolrMetricManager}} doesn't exist when jetty is initializing -- and
doesn't exist until Jetty loads the solr webapp and the {{CoreContainer}} is
initialized -- there is no way any custom jetty plugins could register their
metrics with the {{SolrMetricManager}}
---
I think the best we might be able to do is something like the jetty
{{StatisticsHandler}} wrapper -- that doesn't know anything about OTel, it just
uses simple counters and exposes them via public methods -- and then
{{SolrMetricManager}} could reach down into the Jetty server layer to get an a
refrence to that handler and wrap each of it's public counters in a gauge?
> Solr 10.+ no longer provides any jetty metrics
> ----------------------------------------------
>
> Key: SOLR-18616
> URL: https://issues.apache.org/jira/browse/SOLR-18616
> Project: Solr
> Issue Type: Sub-task
> Reporter: Chris M. Hostetter
> Priority: Major
>
> [https://lists.apache.org/thread.html/8yzggfvrohv09pqs2wgq1y1nfsq80xhc]
>
> Prior to Solr 10, Solr leveraged 2 "dropwizard" provided jetty plugins in
> order to report some key jetty metrics...
> * InstrumentedEE10Handler
> * InstrumentedQueuedThreadPool
> Those were removed by SOLR-17855, w/o any replacement:
> [https://github.com/apache/solr/commit/cee53df8e0a]
>
> The major impacts of this loss are:
> # no other (existing) solr metrics provide visibility into the number of
> HTTP requests, and their associated responses that do *NOT* a valid Solr
> collection/core
> ** so for example: there is no indication in the existing 10.x metrics of a
> huge spike of abnormal {{PUT}} requests to a patch that doesn't exist.
> # no other (existing) solr metrics provide visibility into the thread
> utilization of the jetty (request) thread pool
> ** so for example: there is no indication in the existing 10.x metrics that
> the current number of currently executing requests is growing continuously
> and getting close to saturating the max number of threads
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
