pierrejeambrun commented on code in PR #44423:
URL: https://github.com/apache/airflow/pull/44423#discussion_r1867505483
##########
airflow/www/views.py:
##########
@@ -5298,42 +5298,50 @@ def _clear_task_instances(
"""
Clear task instances, optionally including their downstream
dependencies.
- :param task_instances: list of TIs to clear
- :param clear_downstream: should downstream task instances be cleared
as well?
-
- :return: a tuple with:
- - count of cleared task instances actually selected by the user
- - count of downstream task instances that were additionally cleared
+ :param task_instances: List of Task Instances to clear
+ :param clear_downstream: Whether downstream task instances should also
be cleared
+ :return: A tuple:
+ - Number of cleared task instances selected by the user
+ - Number of downstream task instances additionally cleared
"""
cleared_tis_count = 0
cleared_downstream_tis_count = 0
- # Group TIs by dag id in order to call `get_dag` only once per dag
+ # Group Task Instances by DAG ID to minimize calls to get_dag
tis_grouped_by_dag_id = itertools.groupby(task_instances, lambda ti:
ti.dag_id)
for dag_id, dag_tis in tis_grouped_by_dag_id:
dag = get_airflow_app().dag_bag.get_dag(dag_id)
+ if not dag:
+ raise AirflowException(f"DAG with id '{dag_id}' does not
exist. Cannot clear tasks.")
+
Review Comment:
I don't think this will behave correctly.
Each item of the loop will proceed in success until one of the `dag` is
missing and then the process will stop.
Earlier dag will have succeeded and the transaction is committed. Then it is
hard for the user to know what dag left to clear, what's not.
I would take this check first on all dags:
```
dag = get_airflow_app().dag_bag.get_dag(dag_id)
```
Accumulate all the missing dag and tasks. And in case we have any, return
something like:
```
AirflowException(f"DAGs with ids {dag_ids} doe not exist. Related tasks
{task_ids} cannot be cleared.")
```
`dag_ids` is ids of all the missing dags. `task_ids` is the ids of all tasks
related to these missing dags.
(Either everything succeed, or nothing does, the user has information to
remove the tasks belonging to missing dags and proceed with the clear)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]