ianlofs commented on issue #14364: URL: https://github.com/apache/airflow/issues/14364#issuecomment-793172918
@ashb Ah yeah for sure! So ideally everyone who created a DAG would know to setup `catchup=True` for DAGs where this is necessary. However this is not always the case in practice when you have many teams utilizing Airflow that you are trying to support. It would be nice to be able to use latency metrics to know when a DAG that should be starting isn't starting (this could be because a previous DAG run is still executing). However it also could be some other, unreported failure or error. If there was a steady stream of latency metrics for DAGs the absence of those metrics could be used to alert on. So instead of only emitting a latency metric every time a a new DagRun is created emitting a latency metric for each DAG every time the scheduler parses the DAGs. I guess in the case of a new DagRun not being created this would be the most recent latency. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
