GitHub user 4whomtbts added a comment to the discussion: Memory Leak in OpenTelemetry Tracing — Request to Upgrade to opentelemetry 1.35.0+
Our airflow version is 2.10.5 and we are struggling with same issue. but in our case, upgrading opentelemetry version to 1.35.0+ didn't help we upgraded opentelemetry version as follows: ``` airflow@sorec-airflow-dag-processor-<pod hash>:/opt/airflow$ pip list | grep opentelemetry opentelemetry-api 1.35.0 opentelemetry-exporter-otlp 1.35.0 opentelemetry-exporter-otlp-proto-common 1.35.0 opentelemetry-exporter-otlp-proto-grpc 1.35.0 opentelemetry-exporter-otlp-proto-http 1.35.0 opentelemetry-exporter-prometheus 0.56b0 opentelemetry-proto 1.35.0 opentelemetry-sdk 1.35.0 opentelemetry-semantic-conventions 0.56b0 ``` ``` part of our airflow.cfg [traces] otel_application = airflow-dev otel_host = <out airflow host> otel_on = True otel_port = 80 otel_ssl_active = False otel_task_log_event = False ``` when we set 'traces' block's otel_on option to 'True', **it still causes memory leak regardless of opentelemetry version (whether it is 1.27.0 or 1.35.0)**. but with otel_on=False setting, it doesn't cause memory leak. Here is our memray flamegraph (observed over 12 hours) <img width="1915" height="798" alt="image" src="https://github.com/user-attachments/assets/38fa93d7-5344-45d0-895f-0a0836c04fb0"; /> It seems clear that an opentelemetry (especially, tracing) causes memory leak. GitHub link: https://github.com/apache/airflow/discussions/53771#discussioncomment-14683437 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
