1fanwang opened a new issue, #66906:
URL: https://github.com/apache/airflow/issues/66906

   ### Description
   
   AIP-59 added OpenTelemetry traces inside the scheduler, Dag processor, and 
task supervisor. The CLI entry points that drive those subsystems — `airflow 
tasks test`, `airflow dags trigger`, `airflow dags test`, `airflow backfill 
create` — are not wrapped in spans. When one of those commands is invoked from 
a wrapper that already carries trace context (a CI pipeline, a parent workflow, 
a debugger harness, a shell that received a `TRACEPARENT` env var), the inbound 
trace context dies at the CLI binary. The resulting task and DagRun spans show 
up as a *separate* trace; nothing stitches them back to the caller.
   
   The asymmetry is most visible when an external orchestrator triggers a Dag 
via the CLI and then tries to follow the work through to task completion: the 
orchestrator-side trace has no child spans, the Airflow-side trace has no 
parent.
   
   ### Use case/motivation
   
   Filing this from the LinkedIn Data Infrastructure Airflow / Orchestrator 
team. Scheduler and worker spans flow through our OTel pipeline today (AIP-59 
coverage); CLI-issued operations don't. When something unexpected happens — a 
Dag run materialises in an unexpected state, a manual trigger from a CI job 
produces no follow-up runs, a task-test invocation from a developer's terminal 
misbehaves — we can't trace it back through OTel to the originating CLI 
command. Operators end up grepping logs by hand to reconstruct the sequence.
   
   The general shape is "every reasonable user of OTel hits the same wall when 
CLI commands sit in the middle of the trace chain." Wrapping the four CLI entry 
points in spans that honour W3C TraceContext closes that gap so:
   
   - An external orchestrator with a parent trace can invoke `airflow dags 
trigger ...` and see its trace ID stitch to Airflow's downstream task spans.
   - A CI pipeline invoking `airflow tasks test ...` for verification gets 
first-class span emission anchored to the pipeline's trace.
   - Debug harnesses that already export traces (custom test runners, 
breakpoint shells) can attach to the same trace without manual correlation.
   
   The default no-op behaviour stays unchanged when OTel isn't configured.
   
   ### Related issues
   
   Implementation in #66789. Builds on AIP-59 (existing tracer setup in 
`airflow_shared.observability.traces`).
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to