junghoo-de commented on PR #71855: URL: https://github.com/apache/airflow/pull/71855#issuecomment-5904087326
Confirming this on Airflow 3.3.2 in production (CeleryExecutor + KubernetesExecutor, 3 scheduler replicas), exporting OTLP with cumulative temporality to Google Cloud Monitoring (Telemetry API). - Each scheduler process ends up with more than one live `MeterProvider` under the same `service.instance.id`. In 3.3.2 the two `stats.initialize()` calls are `executors/base_executor.py:213` and `jobs/scheduler_job_runner.py:1643`; the replaced provider keeps exporting (`shutdown_on_exit=False`). - Observable effect: the scheduler's `serde.load_serializers` count series receives ~80 points per 20 minutes instead of 40 at a 30s export interval, with alternating ~5s/25s gaps (two writers with a fixed phase offset). The backend rejects part of these points as duplicate time series or for exceeding its maximum sampling period. - Series first created after the scheduler loop starts (e.g. `dagrun.duration.*`) had a single writer and stayed accurate — 18/18 DAGs matched the `DagRun Finished` scheduler log lines over a 24-minute window. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
