howardyoo commented on issue #41822:
URL: https://github.com/apache/airflow/issues/41822#issuecomment-2689407123
Airflow recently release support for Otel traces, which emits trace in otel,
if you haven’t tried. It does not replace the metrics but adds additional
details on how dags are running, so might be an option.Sent from my iPhoneOn
Feb 27, 2025, at 3:36 PM, Jens Scheffler ***@***.***> wrote:
rather than monitoring when the task has started and finished using metrics,
a better approach could be to monitor
them using traces
@howardyoo mhm, switching all metrics to traces would be a huge re-write in
the core and be a breaking change towards StatsD. I think this is not a easy
fix and would change the strategy completely. Reading the comments I rather
fear that we have a general conceptual problem is metrics are "just not
working". And for Airflow we need to assume we have a distributed environment.
Relying on singletons is... how are other applications implementing this?
Airflow can not be the first using Otel?—Reply to this email directly, view it
on GitHub, or unsubscribe.You are receiving this because you were
mentioned.Message ID: ***@***.***>
jscheffl left a comment (apache/airflow#41822)
rather than monitoring when the task has started and finished using metrics,
a better approach could be to monitor
them using traces
@howardyoo mhm, switching all metrics to traces would be a huge re-write in
the core and be a breaking change towards StatsD. I think this is not a easy
fix and would change the strategy completely. Reading the comments I rather
fear that we have a general conceptual problem is metrics are "just not
working". And for Airflow we need to assume we have a distributed environment.
Relying on singletons is... how are other applications implementing this?
Airflow can not be the first using Otel?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are
receiving this because you were mentioned.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
