Bisk1 commented on PR #33746:
URL: https://github.com/apache/airflow/pull/33746#issuecomment-1694530704
@potiuk I understand the concerns about historical state, I'll try to answer
your questions and make a case why is it safe to ignore it.
- a task in `SHUTDOWN` state that was removed from the `State` enum will
still be visible in UI - the only difference is the box color in grid (it'll be
white because it's the default) and 500 error in `taskInstances/{task_id}` REST
endpoint for this task because schema validation failure, but it doesn't
prevent all UI features from loading and browsing this task's details, logs,
etc., so I'd say nothing really breaks from the UI user perspective
- SHUTDOWN was not a terminal state but a temporary one that was supposed to
quickly transition to something else - only an Airflow bug could have caused a
situation where a task got stuck in this state in DB for longer and is still
there after Airflow restart, so even if historical state like that exists in
some Airflow instances, it is hardly a useful information for Airflow users -
it merely provides evidence of some bug in an older Airflow version
- the last Airflow version that could set TaskInstance to SHUTDOWN was 2.1.4
(released 2021-09-18) so having recent instances of tasks in this state could
only happen if an Airflow instance wasn't updated for at least 2 years - it
certainly limits the number of users affected by a lot
- there is a precedent for removing states here:
https://github.com/apache/airflow/pull/25507/files#diff-960201decc4b79dda0f0850a378ad5cc1caf83dd154593d8a4025af7913e82dfL52
someone could also argue that that change could affect historical state of
tasks but apparently there were no complaints
Last but not least, I made this change because the current description of
states makes it difficult to understand the code. The description doesn't
reflect reality, so I've spend a couple of days debugging Airflow trying to
pinpoint my issue because of tech debt like this. Cleaning it up would be
beneficial for future maintainers who would have easier time contributing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
