ephraimbuddy opened a new pull request #15929: URL: https://github.com/apache/airflow/pull/15929
When a task fails in executor while still queued in scheduler, the executor reports this failure but scheduler doesn't change the task state resulting in the task being queued until the scheduler is restarted. This commit fixes it by ensuring that when a task is reported to have failed in the executor, the task is failed in scheduler This can be reproduced with CeleryExecutor in breeze 1. Start breeze without redis integration 2. start the webserver and scheduler 3. Run a dag e.g example_bash_operator.py 4. The CeleryExecutor will raise exceptions and fails the task 4. Notice that the tasks remain queued --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
