nicolasge opened a new issue, #42459: URL: https://github.com/apache/airflow/issues/42459
### Description Add a new status called iced to [taskinstance](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#task-instances) When the task instance status is set to ICED via taskinstance.set_state('iced'), Airflow will immediately stop the task instance and mark it as "ICED", and the downstream tasks will continue to run as if this task is "success". This feature can be rollout phase by phase: Phase 1: Add this new status in metaDB and UI, and when do scheduling treat it like the combination of "skip" and "success", the task will be ignored but the downstream thinks it has been successfully completed Phase 2: If this task has dataset outlets, it will trigger the dataset update as if the task has been successfully completed Phase 3: Users can leverage Airflow UI, and put a task into ICED status This concept is from Autosys which is another job scheduling system. ### Use case/motivation Let's say we have a DAG with three tasks: A >> B >> C. During the day2day work, there might be some occurred in task B which is caused by external, ie the error is not caused by a bug of the code. In such cases, we want to "skip" this task, so during the schedule the new flow looks like this: A >> C. Please note the "skip" here is different with the Airflow built-in skip, this is something we'd like to call it ICED. If a task is ICED, all the downstream tasks will continue to run even they set the trigger rules to 'all success'. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
