This is an automated email from the ASF dual-hosted git repository.
vatsrahul1001 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 3f03b18920f Document that use_job_schedule also stops asset-triggered
Dag runs (#73952)
3f03b18920f is described below
commit 3f03b18920f3a8aadb601e979c7e18018b89c99d
Author: Lucas Pimentel <[email protected]>
AuthorDate: Sun Oct 18 03:33:12 2026 -0300
Document that use_job_schedule also stops asset-triggered Dag runs (#73952)
* Add a test for asset-triggered runs with use_job_schedule off
The option is documented as turning off the scheduler's use of cron
intervals, with manually triggered runs still running. Asset events are
not cron intervals, so a queued event should still produce a run when
the option is off. Today it does not: the event stays queued forever.
* Create asset-triggered Dag runs when use_job_schedule is off
The option is documented as turning off the scheduler's use of cron
intervals, but its check wrapped all automatic run creation, so asset
events and partitioned-asset events stopped producing runs too and
stayed queued.
With the option off, a Dag that is due by its schedule never gets a
run, so it stays due. Were such Dags still selected, they would take
the per-loop slots and keep their row locks on every loop, and on
PostgreSQL, which sorts the asset-triggered Dags' empty
next_dagrun_create_after last, they could keep those Dags out of every
batch. They are left out of the query instead.
The best-practices page offered the option as a way to stop all new
Dag runs during maintenance, which is no longer true for
asset-triggered runs.
* Document use_job_schedule in the config reference and pin its SQL filter
The config.yml description still said the option turns off cron
intervals and that only manually triggered Dags keep running. It now
says the scheduler stops creating runs from timetables while manual
and asset-triggered runs are still created.
The include_scheduled predicate is the only enforcement of the option,
so build it before the query and say so in the comment. The test now
limits the batch to one Dag so a filter applied after the query would
fail it on Postgres, where the asset Dag's NULL sorts last.
* Document that use_job_schedule=False also stops asset-triggered Dag runs
Reviewers want the option to keep stopping every run the scheduler
creates by itself, because best-practices.rst recommends it for
maintenance windows and the draining work follows the same rule. The
config description predates assets and only mentioned cron intervals,
so it described the behaviour wrongly. Revert the scheduler change and
its tests, and fix the description instead.
---
airflow-core/src/airflow/config_templates/config.yml | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/airflow-core/src/airflow/config_templates/config.yml
b/airflow-core/src/airflow/config_templates/config.yml
index 586153619ec..586553e4f09 100644
--- a/airflow-core/src/airflow/config_templates/config.yml
+++ b/airflow-core/src/airflow/config_templates/config.yml
@@ -2949,8 +2949,10 @@ scheduler:
default: "1000"
use_job_schedule:
description: |
- Turn off scheduler use of cron intervals by setting this to ``False``.
- DAGs submitted manually in the web UI or with trigger_dag will still
run.
+ Set to ``False`` to stop the scheduler creating Dag runs on its own,
whether a Dag is due
+ by its timetable or has received asset events. Manually triggered Dag
runs and backfills
+ are still created. Asset events received while this is off stay queued
and create their
+ runs once it is turned back on.
version_added: 1.10.2
type: boolean
example: ~
