vatsrahul1001 opened a new pull request, #75010: URL: https://github.com/apache/airflow/pull/75010
## What `_find_task_instances_without_heartbeats` now orders by oldest heartbeat and caps the batch at `max_tis_per_query`; the remainder are purged on subsequent scheduler loops. ## Why The heartbeat-timeout purge finds **every** task instance past its timeout and fails them in a **single transaction**. `_purge_task_instances_without_heartbeats` does per-TI work for each one — revalidates committed state (`refresh_from_db`), loads the serialized DAG (`get_dag_for_run`) for `fail_fast`/callback context, writes a `Log` row, and runs `handle_failure` — while the scan holds `FOR UPDATE` (skip_locked) on every matched row. On a mass heartbeat-loss event (a node dying with hundreds–thousands of RUNNING task instances), that unbounded set turns one scheduler loop into a long, widely-locking transaction that stalls the loop and blocks writers. This mirrors the "batched LIMIT + process per loop" rule the project applies to other scheduler cleanup paths. The per-TI revalidation/serialized-DAG load are deliberate correctness guards (added with the attempts-as-rows model) and are kept as-is; this change only bounds how many are handled per loop. `max_tis_per_query` is reused as the per-loop sizing knob (`0` keeps the prior unbounded behaviour, consistent with its use in scheduling). Oldest-heartbeat-first keeps the longest-dead instances prioritised and the LIMIT deterministic. Added `test_find_task_instances_without_heartbeats_is_bounded_by_max_tis_per_query`. No newsfragment — internal scheduler batching, no user-facing behaviour change. (Open to a dedicated config instead of reusing `max_tis_per_query` if preferred.) --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 4.8) Generated-by: Claude Code (Opus 4.8) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
