nickstenning commented on PR #61897: URL: https://github.com/apache/airflow/pull/61897#issuecomment-3943884028
> I've already added a context_carrier column in the dag_run Perfect, then the only hard bit is already done :) > Let's assume that we create a span using a context manager, which is what this PR does [...] There's nothing in principle wrong with the approach taken in this PR, it's just that the root span _ends_ immediately. Or, to be more precise, its `end_time` is very shortly after it begins, because it's a "trigger dag run" or "create dag run " span rather than a "dag run" span. Nothing goes missing, but the visual effect is different from a span that has an `end_time` that corresponds to the end of the dag run. @dstandish posted an example: <img width="1612" height="542" alt="image" src="https://github.com/user-attachments/assets/5850ec3d-71f9-4e31-a87d-6a14b8e9e908"; /> Here the solid blocks represent span duration and the extended lines to the right for `trigger_dag_run.hello`, `create_dag_run`, and `task.triggered.hi` indicate spans that have children that ended after them. > So, we either recreate the span with a synthetic one or somehow understand that the parent process changed and create a new one and make it the new parent. Changing the identity of the parent span after spans in different processes have already emitted is essentially impossible unless you don't emit any telemetry at all until the whole thing has finished. So yes, on balance, I think the thing that will give users what they expect without requiring all kinds of technical gymnastics on our part is to synthesise a `dag_run` span when the dag run completes, in whichever scheduler is handling completion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
