This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 96a5a63b0f Update cluster-policies.rst docs (#27362)
96a5a63b0f is described below
commit 96a5a63b0f9f5e47a9c64e77d124b67cbdd56abd
Author: Bowrna <[email protected]>
AuthorDate: Mon Nov 7 07:55:19 2022 +0530
Update cluster-policies.rst docs (#27362)
This is based on an explanation about cluster policy from @potiuk in the
slack channel. Having it in docs will make this feature more understandable.
---
docs/apache-airflow/concepts/cluster-policies.rst | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/apache-airflow/concepts/cluster-policies.rst
b/docs/apache-airflow/concepts/cluster-policies.rst
index a8cee72003..242924a3f3 100644
--- a/docs/apache-airflow/concepts/cluster-policies.rst
+++ b/docs/apache-airflow/concepts/cluster-policies.rst
@@ -26,9 +26,9 @@ If you want to check or mutate DAGs or Tasks on a
cluster-wide level, then a Clu
There are three types of cluster policy:
-* ``dag_policy``: Takes a :class:`~airflow.models.dag.DAG` parameter called
``dag``. Runs at load time.
-* ``task_policy``: Takes a :class:`~airflow.models.baseoperator.BaseOperator`
parameter called ``task``. Runs at load time.
-* ``task_instance_mutation_hook``: Takes a
:class:`~airflow.models.taskinstance.TaskInstance` parameter called
``task_instance``. Called right before task execution.
+* ``dag_policy``: Takes a :class:`~airflow.models.dag.DAG` parameter called
``dag``. Runs at load time of the DAG from DagBag
:class:`~airflow.models.dagbag.DagBag`.
+* ``task_policy``: Takes a :class:`~airflow.models.baseoperator.BaseOperator`
parameter called ``task``. The policy gets executed when the task is created
during parsing of the task from DagBag at load time. This means that the whole
task definition can be altered in the task policy. It does not relate to a
specific task running in a DagRun. The ``task_policy`` defined is applied to
all the task instances that will be executed in the future.
+* ``task_instance_mutation_hook``: Takes a
:class:`~airflow.models.taskinstance.TaskInstance` parameter called
``task_instance``. The ``task_instance_mutation`` applies not to a task but to
the instance of a task that relates to a particular DagRun. It is executed in a
"worker", not in the dag file processor, just before the task instance is
executed. The policy is only applied to the currently executed run (i.e.
instance) of that task.
The DAG and Task cluster policies can raise the
:class:`~airflow.exceptions.AirflowClusterPolicyViolation` exception to
indicate that the dag/task they were passed is not compliant and should not be
loaded.
