nickstenning commented on PR #61897: URL: https://github.com/apache/airflow/pull/61897#issuecomment-3940476896
> @nickstenning so i guess you're saying that you would make the tasks a child of a synthetic span that gets created when the dag run finishes. Yes, exactly. > for that to work, i think that may require that, until and unless the synthetic span is created, the child spans would be disconnected from the parent span. Technically you are correct, but that's also true of most spans in most traces. Spans aren't emitted until they end, so the root span is the last to arrive under normal circumstances. The only weird thing about this is how we're generating the root spans, not how they look from the point of view of the receiving system. > i really don't love the complexity of the bookkeeping. but we could certainly try it. It is certainly a little more complicated, but I think it may be worth it to make our traces look a bit more "normal." > the other thing is, when you try to think of when you would start the span and end it (and where to insert it in the parent-child hierarchy), that's maybe not so obvious. like current start / end date for dag run is when scheduler sets to running vs some terminal state. but maybe it makes more sense to have it be from trigger to last task done. I _think_ the answer is that the dag run span should start whenever the work for that dag run starts, and finish whenever it reaches a terminal state. Similarly for a task instances within a dag run. If there are retries, I would expect them to show up as separate spans within the dag run but perhaps my mental model isn't quite right here. This is certainly where my knowledge is weaker than yours. Maybe let's talk about it next week? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
