Amrit Sarkar created SOLR-18682:
-----------------------------------
Summary: Collection API commands run by the Overseer start a new
trace instead of continuing the caller's
Key: SOLR-18682
URL: https://issues.apache.org/jira/browse/SOLR-18682
Project: Solr
Issue Type: Bug
Components: tracing
Reporter: Amrit Sarkar
When the Collection API runs on the Overseer, a traced request like {{RELOAD}}
or {{CREATE}} shows up as two unrelated traces: one for the HTTP request, and
one for the command span (e.g. {{ReloadCollectionCmd}}) with all its
{{/admin/cores}} sub-requests. You can't follow the request from the client
down to the cores.
With the distributed Collection API it already works, because the command runs
in an executor that carries the caller's context. The Overseer path loses it:
the message goes through the ZK queue, and {{CollApiCmds.TraceAwareCommand}}
starts its span with no parent. {{TestDistributedTracing}} (added in
SOLR-17011) currently asserts the two separate traces.
*Proposed fix*
* {{CollectionsHandler.submitCollectionApiCommand}} adds the sampled caller's
trace context to the Overseer message, next to {{async}} and the calling lock
id.
* {{TraceAwareCommand}} uses it as the parent of the command span, and drops
those keys before the command reads the message (MODIFYCOLLECTION validates
every message key against the collection state).
* Messages without the keys (from an older node during a rolling upgrade, or an
unsampled request) start a new trace, as today.
Config Set API messages take the same path and can get the same fix in a
follow-up.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]