casassg commented on a change in pull request #8962:
URL: https://github.com/apache/airflow/pull/8962#discussion_r441017597
##########
File path: docs/concepts.rst
##########
@@ -173,6 +213,61 @@ Each task is a node in our DAG, and there is a dependency
from task_1 to task_2:
We can say that task_1 is *upstream* of task_2, and conversely task_2 is
*downstream* of task_1.
When a DAG Run is created, task_1 will start running and task_2 waits for
task_1 to complete successfully before it may start.
+.. _concepts:task_decorator:
+
+Python task decorator
+---------------------
+
+Airflow ``task`` decorator converts any Python decorated function to a Python
Airflow operator.
+The decorated function can be called once to set the arguments and key
arguments for operator execution.
+
+
+.. code:: python
+
+ with DAG('my_dag', start_date=datetime(2020, 5, 15)) as dag:
+
+ @dag.task
+ def hello_world():
+ print('hello world!')
+
+
+ # Also...
+
+ from airflow.decorators import task
+
+ @task
+ def hello_name(name: str):
+ print(f'hello {name}!')
+
+ hello_name('Airflow users')
+
+Operator decorator captures returned values and sends them to the :ref:`XCom
backend <concepts:xcom>`. By default, returned
+value is saved as a single XCom value. You can set ``multiple_outputs`` key
argument to ``True`` to unroll dictionaries,
+lists or tuples into seprate XCom values. This can be used with regular
operators to create
+:ref:`functional DAGs <concepts:functional_dags>`.
+
+Calling a decorated function returns an ``XComArg`` instance. You can use it
to set templated fields on downstream
+operators.
+
+You can call a decorated function twice in a DAG. The decorated function will
automatically generate unique a ``task_id``
+for each generated operator.
+
+.. code:: python
+
+ with DAG('my_dag', start_date=datetime(2020, 5, 15)) as dag:
+
+ @dag.task
+ def update_user(user_id: int):
+ ...
+
+ for user_id in user_ids:
Review comment:
Adding a note `Avoid generating this list dynamically to keep dag
topology stable between DAG runs`
If we have dag serialization and dag versioning, that should be fine in the
future though right? 👀
##########
File path: docs/concepts.rst
##########
@@ -116,6 +116,46 @@ DAGs can be used as context managers to automatically
assign new operators to th
op.dag is dag # True
+.. _concepts:functional_dags:
+
+Functional DAGs
Review comment:
That's fair. The Functional DAGs comes from the AIP title itself. I do
understand the confusion, but not sure if literate is better.
Ideally though, having XComArg and all this, it should allow DAGs to be
written without as much side effects. You will be able to paramatrize an
operator such that it is indeed more functional (takes A and B, outputs C)
instead of current approach where you mostly generate artifacts that depend on
Airflow context and such they are side effects.
Not sure if the above makes sense. In my mind, it makes it more functional
(less prone to side effects) as you make operators resemble more functions, and
XComArgs map to function parameters and function outputs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
