nickstenning commented on PR #62154: URL: https://github.com/apache/airflow/pull/62154#issuecomment-3926355196
> That's why we need a common file with the SDK initialization logic that can be reused. It's certainly true that we need common logic for _SDK_ initialization, but I don't think it's a given that that should happen through calls to the tracing _API_ (I'm using these terms in the sense of `opentelemetry-api` and `opentelemetry-sdk`). It seems to be that it would be reasonable to have common SDK initialization logic which is called at bootstrap time for each process. Separately, there's the question of what the instrumentation API looks like for instrumented code. It certainly sounds like there are some more complicated use cases where trace context will have to be retrieved from an external system, spans started and ended explicitly, etc., but none of this is unfamiliar territory for `opentelemetry-api`, and all the API methods support this natively: `Tracer.start_span` takes an optional `Context`, for example. I'm not arguing that we shouldn't have some helper packages to make instrumenting Airflow easier, but I think the current implementation is somewhat mixing up and confusing the instrumentation API with the SDK. I think that should be avoidable and I'd recommend avoiding it. > The debug spans are providing details about internal operations and they are only useful to developers who have better tools at their disposal like an IDE debugger or a flamegraph profiler. @nickstenning If I understand correctly, you are also agreeing on this. I'm not super familiar with what is instrumented with debug spans today. I suspect you are probably right. There may still be good arguments for a configurable level of detail in the traces emitted by Airflow. If we want to do that I'd still suggest going the head-sampling route. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
