dimberman opened a new pull request #15034: URL: https://github.com/apache/airflow/pull/15034
From https://github.com/astronomer/airflow/pull/new/taskgroup_decorator Added task_group decorator which can be used to create taskgroup from python callable. Inside python callable tasks can be grouped by calling tasks (python callable ) created with task decorator. - taskgroup decorator takes both optional and keyword argument which are passed to Constructor of TaskGroup class. - TaskGroup can be created with one of following syntax ``` python @task_group @task_group() @task_group(group_id='group_name') ``` - TaskGroup class constructor takes one mandatory argument group_id, if not given in decorator it sets group_id to python callable name. Following is a simple example demonstrating use of taskgroup decorator grouping multiple tasks. ``` python @task def task_1(value): return f'[ Task1 {value} ]' @task def task_2(value): print(f'[ Task2 {value} ]') @task_group def section_1(value): return task_2(task_1(value)) ``` task_group decorator utilizes existing TaskGroup context manager currently used for creating TaskGroup, which means we can create nested taskgroup by created nested callable. Following is an example demonstrating use of nested taskgroup. ``` python @task def task_start(): return '[Task_start]' @task def task_end(): print(f'[ Task_End ]') @task def task_1(value): return f'[ Task1 {value} ]' @task def task_2(value): print(f'[ Task2 {value} ]') @task def task_3(value): return f'[ Task3 {value} ]' @task def task_4(value): print(f'[ Task4 {value} ]') @task_group def section_1(value): @task_group def section_2(value2): return task_4(task_3(value2)) op1 = task_2(task_1(value)) return section_2(op1) ``` Dedicated test cases for taskgroup decorator is created in file /tests/utils/test_task_group_decorator.py Recent changes - Added logic to append suffix to duplicate group_id. closes: #11870 <!-- Thank you for contributing! Please make sure that your code changes are covered with tests. And in case of new features or big changes remember to adjust the documentation. Feel free to ping committers for the review! In case of existing issue, reference it using one of the following: closes: #ISSUE related: #ISSUE How to write a good git commit message: http://chris.beams.io/posts/git-commit/ --> --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
