kaxil opened a new pull request, #73151: URL: https://github.com/apache/airflow/pull/73151
Follow-up to #72407, which added the `is_draining` filter to this query but left `is_paused` unchecked. Pausing a Dag stops the scheduler creating runs for it on every path except Dags scheduled on partitioned assets, where it keeps creating them. They cannot start, because the queued-to-running transition does filter on `is_paused`, so they sit in `QUEUED` until someone unpauses and the whole backlog starts at once. The automatic pause after `max_consecutive_failed_dag_runs` hits the same way, without anyone choosing it. It also leaves draining incomplete: `_finalize_draining_dags` clears `is_draining` and sets `is_paused` together, so every `AssetPartitionDagRun` the draining filter was holding is released the moment the drain completes, against the Dag just drained. **Why at creation rather than the queued gate**, which already blocks the run: creating it sets `created_dag_run_id` on the `AssetPartitionDagRun`, permanently spending a pending partition run on a Dag run that cannot proceed. Skipping creation keeps the APDR available for when the Dag is active again. The predicate sits in the `WHERE` clause so paused Dags stop consuming the oldest-first per-tick budget ahead of Dags that can run. **Three of the five `dags_needing_dagruns` predicates, deliberately.** `has_import_errors` is only ever set and cleared alongside `is_stale`, which is already filtered here. `exceeds_max_non_backfill` is a real divergence, but it only moves where a backlog waits, since `max_active_runs` is re-enforced at `QUEUED` to `RUNNING`. Reachable since the partitioned-asset path landed in #64571, so 3.3.0 onward. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
