This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push: new e57d0c913c Doc: Add concrete examples for accessing context variables from TaskFlow tasks (#33296) e57d0c913c is described below commit e57d0c913cb6525c05561473f0e53129fa68055d Author: Ryan Hatter <25823361+rnh...@users.noreply.github.com> AuthorDate: Tue Aug 15 10:29:27 2023 -0400 Doc: Add concrete examples for accessing context variables from TaskFlow tasks (#33296) --- docs/apache-airflow/core-concepts/taskflow.rst | 11 +++++++-- docs/apache-airflow/templates-ref.rst | 12 ++++++++++ docs/shared/template-examples/taskflow-kwargs.rst | 27 +++++++++++++++++++++++ docs/shared/template-examples/taskflow.rst | 24 ++++++++++++++++++++ 4 files changed, 72 insertions(+), 2 deletions(-) diff --git a/docs/apache-airflow/core-concepts/taskflow.rst b/docs/apache-airflow/core-concepts/taskflow.rst index 8dcd52acd1..abe9da25cb 100644 --- a/docs/apache-airflow/core-concepts/taskflow.rst +++ b/docs/apache-airflow/core-concepts/taskflow.rst @@ -66,9 +66,16 @@ If you want to learn more about using TaskFlow, you should consult :doc:`the Tas Context ------- -When running your callable, Airflow will pass a set of keyword arguments that can be used in your function. This set of kwargs correspond exactly to the :ref:`context variables<templates:variables>` you can use in your Jinja templates. +You can access Airflow :ref:`context variables <templates:variables>` by adding them as keyword arguments as shown in the following example: + +.. include:: ../../shared/template-examples/taskflow.rst + +Alternatively, you may add ``**kwargs`` to the signature of your task and all Airflow context variables will be accessible in the ``kwargs`` dict: + +.. include:: ../../shared/template-examples/taskflow-kwargs.rst + +For a full list of context variables, see :ref:`context variables <templates:variables>`. -For this to work, you need to define ``**kwargs`` in your function header, or you can add directly the keyword arguments you would like to get such as ``ti=None`` to have the task instance passed. Logging ------- diff --git a/docs/apache-airflow/templates-ref.rst b/docs/apache-airflow/templates-ref.rst index bfb3cad9eb..7670264627 100644 --- a/docs/apache-airflow/templates-ref.rst +++ b/docs/apache-airflow/templates-ref.rst @@ -81,6 +81,18 @@ Variable Type Description The DAG run's logical date, and values derived from it, such as ``ds`` and ``ts``, **should not** be considered unique in a DAG. Use ``run_id`` instead. +Accessing Airflow context variables from TaskFlow tasks +------------------------------------------------------- + +While ``@task`` decorated tasks don't support rendering jinja templates passed as arguments, +all of the variables listed above can be accessed directly from tasks. The following code block +is an example of accessing a ``task_instance`` object from its task: + +.. include:: ../shared/template-examples/taskflow.rst + +Deprecated variables +------------------------------------------------------- + The following variables are deprecated. They are kept for backward compatibility, but you should convert existing code to use other variables instead. diff --git a/docs/shared/template-examples/taskflow-kwargs.rst b/docs/shared/template-examples/taskflow-kwargs.rst new file mode 100644 index 0000000000..7a4c27c566 --- /dev/null +++ b/docs/shared/template-examples/taskflow-kwargs.rst @@ -0,0 +1,27 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + .. code-block:: python + + @task + def print_ti_info(**kwargs): + ti = kwargs["task_instance"] + print(f"Run ID: {ti.run_id}") # Run ID: scheduled__2023-08-09T00:00:00+00:00 + print(f"Duration: {ti.duration}") # Duration: 0.972019 + + dr = kwargs["dag_run"] + print(f"DAG Run queued at: {dr.queued_at}") # 2023-08-10 00:00:01+02:20 diff --git a/docs/shared/template-examples/taskflow.rst b/docs/shared/template-examples/taskflow.rst new file mode 100644 index 0000000000..a9d46e1c63 --- /dev/null +++ b/docs/shared/template-examples/taskflow.rst @@ -0,0 +1,24 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + .. code-block:: python + + @task + def print_ti_info(task_instance=None, dag_run=None): + print(f"Run ID: {task_instance.run_id}") # Run ID: scheduled__2023-08-09T00:00:00+00:00 + print(f"Duration: {task_instance.duration}") # Duration: 0.972019 + print(f"DAG Run queued at: {dag_run.queued_at}") # 2023-08-10 00:00:01+02:20