yuqian90 commented on a change in pull request #16681:
URL: https://github.com/apache/airflow/pull/16681#discussion_r660698247
##########
File path: airflow/models/taskinstance.py
##########
@@ -1503,6 +1503,10 @@ def handle_failure_with_callback(
def is_eligible_to_retry(self):
"""Is task instance is eligible for retry"""
+ if self.state == State.CLEARED_WHEN_RUNNING:
+ # If a task was cleared when running, it is always eligible for
retry
Review comment:
Thanks @ashb. I agree `RESTARTING` is a more concise and easier to
understand name. So I'm changing to that.
Regarding "it would go back via the scheduler loop", I think that's actually
a better behaviour. TaskInstances that are cleared should go back to the
scheduler loop instead of being restarted by the worker immediately. One
example scenario is a DAG like this:
```
A >> B
```
If A is in success state, and B is a long running task that's currently
running. And some external conditions requires a user to re-run both A and B.
He can do this by clearing A. This sets the state of A to NONE and state of B
to RESTARTING. B is then sent a SIGTERM and goes into UP_FOR_RETRY. While
that's happening, the scheduler starts to run A first. Once A finishes, B can
then run.
However, if we allow the worker to restart B immediately after it is cleared
when running, it would have violated the dependency graph because A might not
done yet when the worker restarts B.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
