ashb commented on a change in pull request #11732:
URL: https://github.com/apache/airflow/pull/11732#discussion_r510068292
##########
File path: airflow/jobs/scheduler_job.py
##########
@@ -1640,11 +1652,13 @@ def _schedule_dag_run(self, dag_run: DagRun,
currently_active_runs: int, session
return 0
if dag.max_active_runs:
- if currently_active_runs >= dag.max_active_runs:
+ if len(currently_active_runs) >= dag.max_active_runs and \
+ dag_run.execution_date not in currently_active_runs:
self.log.info(
Review comment:
With this change it should happen a lot lot less anyway, but yes, or we
fix it properly:
> This approach works around the problem for now, but a better longer term
fix for this would be to introduce a "queued" state for DagRuns, and then when
manually creating dag runs (or clearing) set it to queued, and only have the
scheduler set DagRuns to running, nothing else -- this would mean we wouldn't
need to examine active runs in the TI part of the scheduler loop, only in
DagRun creation part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
