ephraimbuddy opened a new pull request #16289: URL: https://github.com/apache/airflow/pull/16289
Currently, the chances of tasks being killed by the LocalTaskJob heartbeat is high. This is because, when mini scheduler is enabled, then after marking a task successful/failed in Taskinstance.py, we start running the mini scheduler. Whenever the mini scheduling takes time and meets the next job heartbeat, the heartbeat detects that this task has succeeded with no return code because LocalTaskJob.handle_task_exit was not called after the task succeeded. Hence, the heartbeat thinks that this task was externally marked as failed/successful. This change resolves this by moving the mini scheduler to LocalTaskJob at the handle_task_exit method ensuring that the task will no longer be met by the next heartbeat --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
