josh-fell commented on code in PR #24452:
URL: https://github.com/apache/airflow/pull/24452#discussion_r897311128
##########
docs/apache-airflow/dag-run.rst:
##########
@@ -265,8 +265,32 @@ Example of a parameterized DAG:
dag=dag,
)
+Example of a parameterized DAG that reads the configuration from the context:
+
+.. code-block:: python
+
+ import pendulum
+
+ from airflow import DAG
+ from airflow.operators.python import PythonOperator
+
+ dag = DAG(
+ "example_parameterized_dag",
+ schedule_interval=None,
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
+ catchup=False,
+ )
+
+ def print_conf(**context):
+ conf1 = context['dag_run'].conf['conf1']
+ print(conf1)
+
+ parameterized_task = PythonOperator(
+ task_id="parameterized_task",
+ python_callable=print_conf,
+ dag=dag,
+ )
-**Note**: The parameters from ``dag_run.conf`` can only be used in a template
field of an operator.
Review Comment:
I think this note is still worthwhile but could use an addition of more
context rather than deleting entirely. If you choose to access `dag_run.conf`
in a Jinja expression then it can only be done in a template field of an
operator. WDYT?
##########
docs/apache-airflow/dag-run.rst:
##########
@@ -265,8 +265,32 @@ Example of a parameterized DAG:
dag=dag,
)
+Example of a parameterized DAG that reads the configuration from the context:
+
+.. code-block:: python
+
+ import pendulum
+
+ from airflow import DAG
+ from airflow.operators.python import PythonOperator
+
+ dag = DAG(
+ "example_parameterized_dag",
+ schedule_interval=None,
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
+ catchup=False,
+ )
+
+ def print_conf(**context):
+ conf1 = context['dag_run'].conf['conf1']
+ print(conf1)
+
+ parameterized_task = PythonOperator(
+ task_id="parameterized_task",
+ python_callable=print_conf,
+ dag=dag,
+ )
Review Comment:
```suggestion
import pendulum
from airflow import DAG
dag = DAG("example_parameterized_dag",
schedule_interval=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
)
@dag.task(task_id="parameterized_task")
def print_conf(dag_run=None):
conf1 = dag_run.conf["conf1"]
print(conf1)
print_conf()
```
We've moved away from using `PythonOperator` with preference for the
TaskFlow API. I don't think the docs overall reflect this (...yet) but the
example DAGs were all updated with this in mind. Also with the TaskFlow API you
can [access context variables
directly](https://airflow.apache.org/docs/apache-airflow/stable/tutorial_taskflow_api.html#accessing-context-variables-in-decorated-tasks)
without needing `kwargs`.
##########
docs/apache-airflow/dag-run.rst:
##########
@@ -265,8 +265,32 @@ Example of a parameterized DAG:
dag=dag,
)
+Example of a parameterized DAG that reads the configuration from the context:
+
+.. code-block:: python
+
+ import pendulum
+
+ from airflow import DAG
+ from airflow.operators.python import PythonOperator
+
+ dag = DAG(
+ "example_parameterized_dag",
+ schedule_interval=None,
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
+ catchup=False,
+ )
+
+ def print_conf(**context):
+ conf1 = context['dag_run'].conf['conf1']
+ print(conf1)
+
+ parameterized_task = PythonOperator(
+ task_id="parameterized_task",
+ python_callable=print_conf,
+ dag=dag,
+ )
Review Comment:
```suggestion
import pendulum
from airflow import DAG
dag = DAG("example_parameterized_dag",
schedule_interval=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
)
@dag.task(task_id="parameterized_task")
def print_conf(dag_run=None):
conf1 = dag_run.conf["conf1"]
print(conf1)
print_conf()
```
We've moved away from using `PythonOperator` with preference for the
TaskFlow API. I don't think the docs overall reflect this (...yet) but the
example DAGs were all updated with this in mind. Also with the TaskFlow API you
can [access context variables
directly](https://airflow.apache.org/docs/apache-airflow/stable/tutorial_taskflow_api.html#accessing-context-variables-in-decorated-tasks)
without needing `kwargs`. I suppose this is could be argued as a style
preference but helpful to have another example in the docs IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
