Pebble32 opened a new pull request, #75474: URL: https://github.com/apache/airflow/pull/75474
A queued run whose Dag cannot be loaded (for example a custom timetable calling `Variable.get` on a missing Variable) is skipped on every scheduler loop and stays queued forever. Add `[scheduler] max_dag_load_failures_per_queued_run` (default 0, disabled) to mark such a run failed after that many consecutive load failures, then apply the existing `max_consecutive_failed_dag_runs` check so the broken Dag can be paused automatically. The count is kept in memory per scheduler, so it resets on restart, and with several schedulers a run can take up to that many failures per scheduler before it is failed. Follow-up to #74499. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
