potiuk opened a new issue #21867: URL: https://github.com/apache/airflow/issues/21867
### Discussed in https://github.com/apache/airflow/discussions/21333 <div type='discussions-op-text'> <sup>Originally posted by **sartyukhov** February 4, 2022</sup> ### Description Hello! Previously, a SubDag was used to organize tasks into groups. Now you've introduced a TaskGroups to the world . It's nice and very clever. But it has a one big disadvantage over the SubDag - it cant be repeated. ### Use case/motivation For example: In a project I have two task (A >> B): A - collect data (PythonOperator) B - update material view in postgres (PostgresOperator) 'A' could collect only part of data and mark itself as failed (there is no "half-failed" status as I know). But task 'B' should run regardless of A`s result (trigger_rule="all_done" for example) to update matview with part of data. In an ~ hour I would like to repeat that process (A >> B). With SubDag I could do that: - initiate SubDag with parameter _retries=10_ - add DummyTask 'C' with trigger_rule="all_success" - change flow to A >> B >> C and A >> C and that's it, C marks dag as failed and trigger it to retry. But TaskGroup does not have retry parameter. I also can't retry whole DAG, because it's big. I also don't want to update material view inside task 'A' because in that way I can't do [A0, A1..An] >> B (update material view just once for several collects). I hope it's possible. Or maybe it could be done some other way. Thanks in advance. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) </div> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
