[
https://issues.apache.org/jira/browse/SOLR-18594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl updated SOLR-18594:
-------------------------------
Description:
Resource attributes identify the sender of OTLP telemetry -- {{service.name}},
{{service.instance.id}}, {{service.namespace}}, {{host.name}} and so on -- and
the OTLP-to-Prometheus mapping turns {{service.name}} into the {{job}} label
and {{service.instance.id}} into {{instance}}. Solr's traces follow the
conventions; its metrics ignore them entirely.
h2. Current behaviour
{{SolrMetricManager.meterProvider()}} never calls {{setResource(...)}}, so
metrics carry {{Resource.getDefault()}}. Visible on any node, no OTLP needed:
{code}
$ curl -s 'http://localhost:8983/solr/admin/metrics' | grep target_info
target_info{service_name="unknown_service:java",telemetry_sdk_language="java",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.65.0"}
1
{code}
{{OtelTracerConfigurator.prepareConfiguration()}} does default
{{OTEL_SERVICE_NAME}} to {{solr}}, merge {{OTEL_RESOURCE_ATTRIBUTES}} and
inject {{host.name}} from {{solr.host.advertise}} -- but only into the
autoconfigured SDK, which metrics do not use. Setting either variable has no
effect on metrics.
For push users every node arrives as {{job="unknown_service:java"}} with no
{{instance}}, so a multi-node cluster collapses into one series and the shipped
dashboard's {{sum by (instance)(...)}} is meaningless. No configuration fixes
this today.
h2. Scope
* Build the metrics resource from the standard inputs ({{OTEL_SERVICE_NAME}} /
{{otel.service.name}}, {{OTEL_RESOURCE_ATTRIBUTES}} /
{{otel.resource.attributes}}), so one set of variables configures both signals.
Where the {{opentelemetry}} module is active, reuse the resource the
autoconfigured SDK resolved, so traces and metrics cannot disagree and
operator-added {{ResourceProvider}}s are included.
* Solr defaults, applied only where the operator has not set them:
{{service.name=solr}} (as tracing already does),
{{service.instance.id=<host>:<port>}}, {{host.name}}. {{host:port}} rather than
a UUID because it becomes the {{instance}} label, making a pushed series match
a direct scrape of the same node -- which is what lets one dashboard serve both
routes.
* Honour the standard exporter variables as an alternative to
{{solr.metrics.otlp*}} -- {{OTEL_METRICS_EXPORTER}},
{{OTEL_EXPORTER_OTLP_ENDPOINT}} / {{OTEL_EXPORTER_OTLP_METRICS_ENDPOINT}},
{{OTEL_EXPORTER_OTLP_PROTOCOL}} / {{OTEL_EXPORTER_OTLP_METRICS_PROTOCOL}},
{{OTEL_METRIC_EXPORT_INTERVAL}} -- with {{solr.metrics.otlp*}} taking
precedence, documented.
Ordering already works: {{CoreContainer}} calls
{{OpenTelemetryConfigurator.initializeOpenTelemetrySdk()}} immediately before
constructing {{SolrMetricManager}}.
h2. Out of scope
* Shipping OpenTelemetry's resource detectors
({{io.opentelemetry.instrumentation:opentelemetry\-resources}}). Operators can
add the jar to the module's lib directory and autoconfigure picks it up via
SPI; worth a ref-guide line.
* Solr-specific attribute names for what the conventions cover. Cluster
grouping is {{service.namespace}} or {{k8s.cluster.name}}, set by the operator.
h2. Acceptance criteria
* {{OTEL_SERVICE_NAME}} and {{OTEL_RESOURCE_ATTRIBUTES}}, as environment
variables or {{otel.*}} system properties, are reflected in the metrics
resource and override Solr's defaults.
* {{target_info}} reports {{service_name="solr"}} and the node's
{{service_instance_id}} out of the box.
* A node pushing through an OpenTelemetry Collector lands in Prometheus as
{{job="solr"}} with {{instance}} equal to that node's {{host:port}}, matching a
direct scrape.
* Trace and metric resources are identical when the module is active.
* Documented on the OpenTelemetry ref-guide page for both signals, linked from
{{metrics\-reporting.adoc}}.
This closes the last gap in SOLR-18536, whose Collector config has to
substitute {{service.name}} itself and cannot produce a per-node {{instance}}
at all.
was:
When pushing metrics and traces over OTLP, it is common to set various resource
attribtues like service.name, service.instance.id, service.namespace and more,
which can then be used in rules and dashboards.
Currently Tracing can be configured using OTEL semantic conventions environment
variable ([https://opentelemetry.io/docs/concepts/semantic-conventions/)] but
metrics cannot.
This task is to unify the configuration of OTEL through environment variables.
> Honour OTEL resource attribute conventions for metrics
> ------------------------------------------------------
>
> Key: SOLR-18594
> URL: https://issues.apache.org/jira/browse/SOLR-18594
> Project: Solr
> Issue Type: Sub-task
> Components: metrics
> Reporter: Jan Høydahl
> Assignee: Jan Høydahl
> Priority: Major
>
> Resource attributes identify the sender of OTLP telemetry --
> {{service.name}}, {{service.instance.id}}, {{service.namespace}},
> {{host.name}} and so on -- and the OTLP-to-Prometheus mapping turns
> {{service.name}} into the {{job}} label and {{service.instance.id}} into
> {{instance}}. Solr's traces follow the conventions; its metrics ignore them
> entirely.
> h2. Current behaviour
> {{SolrMetricManager.meterProvider()}} never calls {{setResource(...)}}, so
> metrics carry {{Resource.getDefault()}}. Visible on any node, no OTLP needed:
> {code}
> $ curl -s 'http://localhost:8983/solr/admin/metrics' | grep target_info
> target_info{service_name="unknown_service:java",telemetry_sdk_language="java",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.65.0"}
> 1
> {code}
> {{OtelTracerConfigurator.prepareConfiguration()}} does default
> {{OTEL_SERVICE_NAME}} to {{solr}}, merge {{OTEL_RESOURCE_ATTRIBUTES}} and
> inject {{host.name}} from {{solr.host.advertise}} -- but only into the
> autoconfigured SDK, which metrics do not use. Setting either variable has no
> effect on metrics.
> For push users every node arrives as {{job="unknown_service:java"}} with no
> {{instance}}, so a multi-node cluster collapses into one series and the
> shipped dashboard's {{sum by (instance)(...)}} is meaningless. No
> configuration fixes this today.
> h2. Scope
> * Build the metrics resource from the standard inputs ({{OTEL_SERVICE_NAME}}
> / {{otel.service.name}}, {{OTEL_RESOURCE_ATTRIBUTES}} /
> {{otel.resource.attributes}}), so one set of variables configures both
> signals. Where the {{opentelemetry}} module is active, reuse the resource the
> autoconfigured SDK resolved, so traces and metrics cannot disagree and
> operator-added {{ResourceProvider}}s are included.
> * Solr defaults, applied only where the operator has not set them:
> {{service.name=solr}} (as tracing already does),
> {{service.instance.id=<host>:<port>}}, {{host.name}}. {{host:port}} rather
> than a UUID because it becomes the {{instance}} label, making a pushed series
> match a direct scrape of the same node -- which is what lets one dashboard
> serve both routes.
> * Honour the standard exporter variables as an alternative to
> {{solr.metrics.otlp*}} -- {{OTEL_METRICS_EXPORTER}},
> {{OTEL_EXPORTER_OTLP_ENDPOINT}} / {{OTEL_EXPORTER_OTLP_METRICS_ENDPOINT}},
> {{OTEL_EXPORTER_OTLP_PROTOCOL}} / {{OTEL_EXPORTER_OTLP_METRICS_PROTOCOL}},
> {{OTEL_METRIC_EXPORT_INTERVAL}} -- with {{solr.metrics.otlp*}} taking
> precedence, documented.
> Ordering already works: {{CoreContainer}} calls
> {{OpenTelemetryConfigurator.initializeOpenTelemetrySdk()}} immediately before
> constructing {{SolrMetricManager}}.
> h2. Out of scope
> * Shipping OpenTelemetry's resource detectors
> ({{io.opentelemetry.instrumentation:opentelemetry\-resources}}). Operators
> can add the jar to the module's lib directory and autoconfigure picks it up
> via SPI; worth a ref-guide line.
> * Solr-specific attribute names for what the conventions cover. Cluster
> grouping is {{service.namespace}} or {{k8s.cluster.name}}, set by the
> operator.
> h2. Acceptance criteria
> * {{OTEL_SERVICE_NAME}} and {{OTEL_RESOURCE_ATTRIBUTES}}, as environment
> variables or {{otel.*}} system properties, are reflected in the metrics
> resource and override Solr's defaults.
> * {{target_info}} reports {{service_name="solr"}} and the node's
> {{service_instance_id}} out of the box.
> * A node pushing through an OpenTelemetry Collector lands in Prometheus as
> {{job="solr"}} with {{instance}} equal to that node's {{host:port}}, matching
> a direct scrape.
> * Trace and metric resources are identical when the module is active.
> * Documented on the OpenTelemetry ref-guide page for both signals, linked
> from {{metrics\-reporting.adoc}}.
> This closes the last gap in SOLR-18536, whose Collector config has to
> substitute {{service.name}} itself and cannot produce a per-node {{instance}}
> at all.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
