SamWheating opened a new issue, #24266: URL: https://github.com/apache/airflow/issues/24266
### Apache Airflow version 2.3.2 (latest released) ### What happened I recently disabled the printing of the file parsing stats by setting `scheduler.print_stats_interval` to `0` in the airflow config, because it was causing a lot of noise in the scheduler logs. However, this set of some monitors we have set up on the file processor, since we were no longer receiving `dag_processing.last_run.seconds_ago.<dagrun>` metrics. On closer inspection, it appears that these metrics are emitted at the same time that the file parsing stats are logged, so disabling the logs disabled the metrics: https://github.com/apache/airflow/blob/1507ca48d7c211799129ce7956c11f4c45fee5bc/airflow/dag_processing/manager.py#L734-L739 https://github.com/apache/airflow/blob/1507ca48d7c211799129ce7956c11f4c45fee5bc/airflow/dag_processing/manager.py#L781-L787 ### What you think should happen instead I should be able to disable the logging of stats without also disabling the metrics. ### How to reproduce _No response_ ### Operating System Debian GNU/Linux 10 (buster) ### Versions of Apache Airflow Providers _No response_ ### Deployment Other 3rd-party Helm chart ### Deployment details _No response_ ### Anything else Let me know if you have any preferences or suggestions for a possible fix. I was thinking about adding an explicit `scheduler.print_file_processor_stats` boolean to decouple the printing of the stats from the computing of metrics. This way a user could enable or disable the printing of the metrics separately from each other (as the metrics can already be disabled via the `metrics.statsd_allow_list` configuration. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
