malthe commented on issue #16706: URL: https://github.com/apache/airflow/issues/16706#issuecomment-908309524
I think perhaps a starting point is to contribute "opentelemetry-instrumentation-X" packages (e.g. [opentelemetry-instrumentation-psycopg2](https://pypi.org/project/opentelemetry-instrumentation-psycopg2/)). It should be possible then to [add a span around task execution](https://opentelemetry.io/docs/java/manual_instrumentation/#create-nested-spans) to logically group an arbitrary and unknown set of events and/or metrics produced by would-be instrumented libraries used during the execution of a single task. The next step might then be to add a metrics/events tab to the task instance page and display events/metrics emitted during the execution – for example, if exporting telemetry to Prometheus, then we can use the [HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/#expression-queries) to pull this data dynamically. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
