[
https://issues.apache.org/jira/browse/HDDS-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-17350:
----------------------------------
Labels: pull-request-available (was: )
> Ozone client builds the OpenTelemetry SDK at start-up even when tracing is
> disabled
> -----------------------------------------------------------------------------------
>
> Key: HDDS-17350
> URL: https://issues.apache.org/jira/browse/HDDS-17350
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Client
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
> Labels: pull-request-available
>
> *Problem*
> `RpcClient` calls `TracingUtil.initTracing("client", conf)` when it is
> created. With tracing disabled
> (`ozone.tracing.enabled=false`, the default), `TracingUtil.initialize()` only
> skips the setup when
> `ozone.tracing.client.application-aware` is also false. But that option
> defaults to true (HDDS-13803). Unless the
> application has registered its own `GlobalOpenTelemetry`, every client
> therefore runs `initOzoneSdk()` →
> `buildSdkTracerProvider()`. That means:
> - parsing the sampler and span-sampling config (the "Sampling Trace Config" /
> "Sampling Span Config" log lines);
> - building an `OtlpGrpcSpanExporter` and a `BatchSpanProcessor`, which starts
> a background export thread;
> - building the `SdkTracerProvider` and `OpenTelemetrySdk`, and registering it
> as `GlobalOpenTelemetry`.
> In application-aware mode none of this is used unless a span is already
> active. Spans are created only when
> `Span.current()` is valid (`shouldCreateTracingProxy`), and a client never
> starts a root span
> (`canStartSpanWithoutParent()` returns `tracingEnabled`). For the usual
> client process with no tracing of its own,
> the SDK, exporter and thread are never used.
> The cost shows up wherever many short-lived client JVMs start: YARN
> `ContainerLocalizer` JVMs, MapReduce tasks, CLI
> commands. In a fresh JVM on a secure compose cluster, timed with step markers
> and DEBUG logs, `FileSystem.get()` for
> `ofs://` takes about 600 ms. About **80 ms** of that runs from the first
> `TracingUtil` line to "Initialized tracing
> service: client (enabled=false, applicationAware=true)", including loading
> the OpenTelemetry SDK and OTLP exporter
> classes. A MapReduce job starts about four such cold clients one after
> another on its critical path, so this is
> roughly 0.3 s per job.
> *Proposed change*
> Build the SDK lazily in application-aware mode:
> 1. In `initialize()`, when tracing is disabled and application-aware is on:
> - if the application registered a real `GlobalOpenTelemetry`, adopt it, as
> today; this is cheap;
> - otherwise keep the no-op tracer and record that an SDK may be needed,
> without building it.
> 2. On the first span request that has a valid parent, build Ozone's SDK once,
> under the existing lock, and
> continue as today. The parent can come from `Span.current()` in
> `shouldCreateTracingProxy` and the child-span
> paths, or from a non-empty propagated context in `importSpan`.
> `hasUsableTracer()` must treat "SDK pending" as
> usable, so those checks still pass.
> 3. Nothing changes when `ozone.tracing.enabled=true`: the SDK is built
> eagerly, so root spans can start at once.
> *Compatibility*
> There's no change in which spans are created or exported. The differences:
> - the export thread and exporter only exist once a traced call happens;
> - in application-aware mode, Ozone registers its SDK as `GlobalOpenTelemetry`
> at the first traced call instead of at
> client creation;
> - a co-resident library that reads `GlobalOpenTelemetry` before that sees the
> default no-op instance, as it would
> in a process with tracing fully disabled. This is worth calling out in the
> docs.
> *Tests*
> - With tracing disabled and application-aware on: after client creation no
> `SdkTracerProvider` or
> `BatchSpanProcessor` exists, and `GlobalOpenTelemetry` isn't set.
> - The same setup with an active application span around an Ozone call: the
> SDK is built once, and the call produces
> a child span.
> - An application-registered `GlobalOpenTelemetry` is still adopted without
> building Ozone's SDK.
> - `ozone.tracing.enabled=true`: unchanged, eager SDK.
> - Optional: the cold-client probe shows `FileSystem.get()` about 80 ms
> shorter.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
