zuk commented on a change in pull request #12741:
URL: https://github.com/apache/airflow/pull/12741#discussion_r562715639
##########
File path: docs/apache-airflow/concepts.rst
##########
@@ -1022,6 +1023,28 @@ This animated gif shows the UI interactions. TaskGroups
are expanded or collapse
.. image:: img/task_group.gif
+While TaskGroups and SubDAGs are both used to create repeating patterns,
depending on your use case, one may be better
+than the other. The SubDagOperator launches a DAG as a separate entity from
the original graph. This design pattern
+offers flexibility to create SubDAGs with different schedulers and executors
at the cost of greater complexity and
+maintenance burden. TaskGroups creates a UI grouping concept on the same
original DAG which simplifies logic and
+maintenance for less flexibility.
+
+Here is a table that summarizes their differences:
++----------------------+----------------------+
+| Task Group | SubDAG |
++======================+======================+
+| Repeating patterns | Repeating patterns |
+| live on the same DAG | run as separate DAGs|
++----------------------+----------------------+
+| Follows schedule of | Creates a new |
+| DAG | schedule |
++----------------------+----------------------+
+| Has same executor as | Can specify an |
+| DAG | executor |
++----------------------+----------------------+
+| Honors all pool | Does not honor pool |
+| configurations | configurations |
++----------------------+----------------------+
Review comment:
This is very helpful, but one thing still not clear to me from these
docs: Since SubDAGs can have different schedules, how does a SubDAG's execution
trigger or block executions of tasks further down in the parent DAG?
For example, if a SubDAG is scheduled to execute every 1 hour, and the
parent DAG is scheduled every 20 minutes, will the SubDAG be executed every 20
minutes?
Or if the schedules were reversed (SubDAG every 20 minutes and parent DAG
every hour), how do the SubDAG's multiple executions in that hour factor into
the parent DAG's eventual execution?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
