xBis7 commented on PR #43941: URL: https://github.com/apache/airflow/pull/43941#issuecomment-2551301114
Hi @ashb, * I've addressed all your comments, * I created a new migration file, * I added some unit tests for the scheduler changes * I fixed 2 bugs that I came across while making the changes. Please take a look when you get a chance and let me know how it looks. I'll push some commits for moving the dagrun/scheduler span changes to new methods, to make the code more readable. > I think this will work, though the main question I have is: if we can re-create the span for an unhealthy scheduler, couldn't we do that for healthy schedulers too? That way we wouldn't need to store active_spans in memory at all, and nor would we need to store span status in the DB? This will get us to the old implementation where we can't create spans from under tasks. If a task has already been run and it created spans, then these spans will be lost when we recreate the task span. This is happening for a few reasons * the already exported spans are referencing the old span as a parent * we are not setting the traceid and spanid deterministically but we are letting the otel sdk to handle that * the task sub spans are getting the parent with context propagation The only way to get the spans back, will be to rerun the task. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
