Fokko closed pull request #4027: [AIRFLOW-3177] Change scheduler_heartbeat from gauge to counter URL: https://github.com/apache/incubator-airflow/pull/4027
This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/UPDATING.md b/UPDATING.md index 74337f3fe8..5e1402576b 100644 --- a/UPDATING.md +++ b/UPDATING.md @@ -52,6 +52,10 @@ To delete a user: airflow users --delete --username jondoe ``` +### StatsD Metrics + +The `scheduler_heartbeat` metric has been changed from a gauge to a counter. Each loop of the scheduler will increment the counter by 1. This provides a higher degree of visibility and allows for better integration with Prometheus using the [StatsD Exporter](https://github.com/prometheus/statsd_exporter). Scheduler upness can be determined by graphing and alerting using a rate. If the scheduler goes down, the rate will drop to 0. + ### Custom auth backends interface change We have updated the version of flask-login we depend upon, and as a result any diff --git a/airflow/jobs.py b/airflow/jobs.py index b224f75545..3922939a86 100644 --- a/airflow/jobs.py +++ b/airflow/jobs.py @@ -1895,7 +1895,7 @@ def process_file(self, file_path, pickle_dags=False, session=None): @provide_session def heartbeat_callback(self, session=None): - Stats.gauge('scheduler_heartbeat', 1, 1) + Stats.incr('scheduler_heartbeat', 1, 1) class BackfillJob(BaseJob): ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
