syun64 commented on issue #11463: URL: https://github.com/apache/airflow/issues/11463#issuecomment-1377515940
Thank you for the quick response @potiuk ! I'm starting to really appreciate the friendliness and responsiveness of this community, and it really shows with every single response I get! I do agree that the idea of Open Telemetry implementation makes a whole lot of sense, especially from the perspective of maintaining Airflow in a stable manner by abiding to a well-defined tagging standard. Unfortunately, I'm not sure if we will be able to adopt Airflow-OpenTelemetry Metrics out of the box as easily as we think. The main hurdles for some folks in big companies like us will just be that we rely on managed infrastructures that only accept certain protocols into their metric storage. In our case, we are only able to accept Statsd/influxdb-statsd UDP protocols which is then aggregated and propagated to our MetricTank down the pipeline. I'm not an expert on the exact factors that ultimately drove our managed infrastructure to hold off on adopting OTLP as a supported input protocol at the company-wide level, but I'm guessing it would have to do with the fact that gRPC/HTTP introduces more network traffic compared to UDP - to a factor that might have a significant performance impact if adopted at scale. Another idea I had to try to adopt Airflow-OpenTelemetry with our existing infrastructure was to use a StatsdExporter - but this seems to be [blocked by OpenTelemetry community](https://github.com/open-telemetry/opentelemetry-specification/issues/374) for reasons we all very well know. Given these limitations, I think it would make more sense to introduce callbacks that emit Statsd metrics in the dag definitions themselves if we aren't able to introduce the tagging related changes above... Adopting Airflow-OpenTelemetry just doesn't seem possible for us just yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
