The GitHub Actions job "Tests" on airflow.git has succeeded.
Run started by GitHub user jedcunningham (triggered by jedcunningham).

Head commit for run:
f6dece48ad777e1258ecce91b4c5295155433a44 / Jed Cunningham 
<[email protected]>
Make DagRun state updates for paused DAGs faster

If you have a large number of paused DAGs with running DagRuns in your instance,
the SchedulerJob function that tries to update the state for those DagRuns
can be really slow (slow enough to cause the scheduler to be killed by
k8s livenessprobes, for example).

This PR does a few things:

- Only check paused DAGs that have running DagRuns
- Only check DagRuns that have TI's updated since the last
  DagRun.last_scheduling_decision
- Only get the serialized DAG if we are ready to try updating the DagRun
  state

This prevents Airflow from trying to update every "paused" DagRun over
and over again, and doesn't do expensive work until we know it makes
sense to try updating the state.

Report URL: https://github.com/apache/airflow/actions/runs/3483798801

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to