dstandish commented on PR #62154: URL: https://github.com/apache/airflow/pull/62154#issuecomment-3929278371
> We can't use opentelemetry.* imports throughout the codebase and let me explain why. Airflow is a complicated project composed of a bunch of independent processes and subprocesses. You don't have a central in-memory constructor that initializes a bunch of objects and shares them across all processes. When I'm saying processes, I'm referring to a scheduler, a worker, a dag processor, etc. Each one of these, needs to configure the SDK and get its own instance of a tracer. That's why we need a common file with the SDK initialization logic that can be reused. Addiitionally, the current implementation with the metaclass makes sure that the SDK is initialized only once per process. Apart from the initialization, airflow is very hard to auto-instrument. Many operations are depending on db updates. E.g. once that is changed in the db, then take this action. For that reason, we have to handle the span context change manually and start-end spans manually. @xBis7 These issues are pretty much unrelated to how we get the tracer. @nickstenning is saying, "let's just call trace.get_tracer" instead of doing all of this custom stuff. I think that is very wise and I am not sure why we can't just do that. I think the real argument here is whether we Airflow needs to come up with its own tracing interface just so that users could hypothetically use a different tracing option. To me, that does not seem like a super strong case. Why do we want to get involved in defining our own tracing interface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
