ephraimbuddy opened a new pull request, #61983: URL: https://github.com/apache/airflow/pull/61983
When many task instances enter UP_FOR_RESCHEDULE state, the query to fetch the latest reschedule date becomes slow due to a missing composite index. This causes the scheduler to miss heartbeats. Previously only sensors used reschedule mode, but since fddf4a72406fda039dea813c0ff6e58dd0cd2891, non-sensor tasks can also be rescheduled, significantly increasing the number of rows per task instance in the task_reschedule table. Add a composite (ti_id, id DESC) index to the task_reschedule table, replacing the single-column (ti_id) index. The reschedule query: https://github.com/apache/airflow/blob/baa8c72036920f67d8b7479685dadc2c73da0739/airflow-core/src/airflow/ti_deps/deps/ready_to_reschedule.py#L73 Other places this can benefit: 1. https://github.com/apache/airflow/blob/baa8c72036920f67d8b7479685dadc2c73da0739/airflow-core/src/airflow/models/taskinstance.py#L1171 2. https://github.com/apache/airflow/blob/baa8c72036920f67d8b7479685dadc2c73da0739/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_reschedules.py#L40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
