ashb commented on a change in pull request #3997: [AIRFLOW-3153] send dag
last_run to statsd
URL: https://github.com/apache/incubator-airflow/pull/3997#discussion_r222699391
##########
File path: airflow/jobs.py
##########
@@ -1513,6 +1518,16 @@ def _log_file_processing_stats(self,
if processor_start_time else None)
last_run = processor_manager.get_last_finish_time(file_path)
+ file_name = file_path[len(dags_folder) + 1:]
+ dag_name = os.path.splitext(file_name)[0].replace(os.sep, '.')
+ if last_runtime is not None:
+ Stats.gauge('last_runtime.{}'.format(dag_name), last_runtime)
+ if last_run is not None:
+ unixtime = last_run.strftime("%s")
+ seconds_ago = (timezone.utcnow() - last_run).total_seconds()
+ Stats.gauge('last_run.unixtime.{}'.format(dag_name), unixtime)
+ Stats.gauge('last_run.seconds_ago.{}'.format(dag_name),
seconds_ago)
Review comment:
How often does this stat get updated? (I'm not sure from reviewing the PR
where we are in the scheduler?) If we were using Promethus I would be tempted
to say that just `last_run.unixtime` stat would be the only one we should have,
but I honestly' don't remember how Statsd works anymore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
