This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v3-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 9767c80a1339f60634dad69c37d2d3245904beae Author: D. Ferruzzi <[email protected]> AuthorDate: Wed Sep 17 16:29:51 2025 -0700 SLA-to-Deadlines migration guide and minor Deadlines-related docs updates (#55743) (cherry picked from commit 48237f16bac89cda6a0c1dc000a504644145c5af) --- airflow-core/docs/howto/deadline-alerts.rst | 7 ++ airflow-core/docs/howto/index.rst | 1 + airflow-core/docs/howto/sla-to-deadlines.rst | 89 ++++++++++++++++++++++ .../docs/installation/upgrading_to_airflow3.rst | 2 +- task-sdk/src/airflow/sdk/definitions/dag.py | 3 +- 5 files changed, 100 insertions(+), 2 deletions(-) diff --git a/airflow-core/docs/howto/deadline-alerts.rst b/airflow-core/docs/howto/deadline-alerts.rst index edfcc3b2e77..33d420e1214 100644 --- a/airflow-core/docs/howto/deadline-alerts.rst +++ b/airflow-core/docs/howto/deadline-alerts.rst @@ -29,6 +29,11 @@ Deadline Alerts allow you to set time thresholds for your Dag runs and automatic thresholds are exceeded. You can set up Deadline Alerts by choosing a built-in reference point, setting an interval, and defining a response using either Airflow's Notifiers or a custom callback function. +Migrating from SLA +------------------ + +For help migrating from SLA to Deadlines, see the :doc:`migration guide </howto/sla-to-deadlines>` + Creating a Deadline Alert ------------------------- @@ -80,6 +85,8 @@ The timeline for this example would look like this: Scheduled Queued Started Deadline 00:00 00:03 00:05 00:18 +.. _built-in-deadline-references: + Using Built-in References ------------------------- diff --git a/airflow-core/docs/howto/index.rst b/airflow-core/docs/howto/index.rst index 28dd34a8058..9abc9caa76e 100644 --- a/airflow-core/docs/howto/index.rst +++ b/airflow-core/docs/howto/index.rst @@ -34,6 +34,7 @@ configuring an Airflow environment. add-dag-tags add-owner-links notifications + sla-to-deadlines deadline-alerts set-config set-up-database diff --git a/airflow-core/docs/howto/sla-to-deadlines.rst b/airflow-core/docs/howto/sla-to-deadlines.rst new file mode 100644 index 00000000000..dd1e4159260 --- /dev/null +++ b/airflow-core/docs/howto/sla-to-deadlines.rst @@ -0,0 +1,89 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Migrating from SLA to Deadline Alerts +===================================== + +Two Different Paradigms +----------------------- + +While the goal of the **SLA** and **Deadline Alerts** features are very similar, they use two very different approaches. +This guide will lay out the major differences and help you decide on the best approach for your use case. + +To begin with, we'll start by explaining the two approaches then go into how to find the right Deadline for your use case. + +SLA +^^^ + +When the dag run **finishes**, check the current time. If the time is greater than (logical_date + sla) then +execute ``sla_miss_callback``. If the Dag run never finishes, the SLA is never checked. + +Deadline Alerts +^^^^^^^^^^^^^^^ + +When a Dag run **starts**, calculate and store (:ref:`DeadlineReference <built-in-deadline-references>` + interval). +The scheduler loop then checks periodically (default 5 seconds, set by ``scheduler_heartbeat_sec``) if any of those +times have passed then execute ``callback(**kwargs)``. + +The most direct migration path would be to use the ``DeadlineReference.DAGRUN_LOGICAL_DATE`` reference, but note that +the major change is that the Deadline's callback will execute "immediately" (within ``scheduler_heartbeat_sec`` of the +calculated expiration time) and not wait until the Dag finishes first. + +Equivalent Example Dags +----------------------- + +Below is a Dag using a 1-hour SLA, followed by an equivalent Dag using Deadline Alerts. + +SLA Example +^^^^^^^^^^^ + +.. code-block:: python + + with DAG( + "minimal_sla_example", + default_args={"sla": timedelta(hours=1)}, + sla_miss_callback=SlackWebhookNotifier( + text="SLA missed for {{ dag_run.dag_id }}", + ), + ): + BashOperator(task_id="long_task", bash_command="sleep 3600") + +Deadline Alerts Example +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: python + + with DAG( + "minimal_deadline_example", + deadline=DeadlineAlert( + reference=DeadlineReference.DAGRUN_LOGICAL_DATE, + interval=timedelta(hours=1), + callback=AsyncCallback( + SlackWebhookNotifier, + kwargs={ + text: "Deadline missed for {{ dag_run.dag_id }}", + }, + ), + ), + ): + BashOperator(task_id="long_task", bash_command="sleep 3600") + + +Further Reading +--------------- + +For more details on the Deadline Alerts feature, see the :doc:`how-to guide </howto/deadline-alerts>`. diff --git a/airflow-core/docs/installation/upgrading_to_airflow3.rst b/airflow-core/docs/installation/upgrading_to_airflow3.rst index ecd7090fd07..4a9f8a8209c 100644 --- a/airflow-core/docs/installation/upgrading_to_airflow3.rst +++ b/airflow-core/docs/installation/upgrading_to_airflow3.rst @@ -259,7 +259,7 @@ These include: - **SubDAGs**: Replaced by TaskGroups, Assets, and Data Aware Scheduling. - **Sequential Executor**: Replaced by LocalExecutor, which can be used with SQLite for local development use cases. - **CeleryKubernetesExecutor and LocalKubernetesExecutor**: Replaced by `Multiple Executor Configuration <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/index.html#using-multiple-executors-concurrently>`_ -- **SLAs**: Deprecated and removed; Will be replaced by forthcoming `Deadline Alerts <https://cwiki.apache.org/confluence/x/tglIEw>`_. +- **SLAs**: Deprecated and removed; replaced with :doc:`Deadline Alerts </howto/deadline-alerts>`. - **Subdir**: Used as an argument on many CLI commands, ``--subdir`` or ``-S`` has been superseded by :doc:`Dag bundles </administration-and-deployment/dag-bundles>`. - **REST API** (``/api/v1``) replaced: Use the modern FastAPI-based stable ``/api/v2`` instead; see :doc:`Airflow API v2 </stable-rest-api-ref>` for details. - **Some Airflow context variables**: The following keys are no longer available in a :ref:`task instance's context <templates:variables>`. If not replaced, will cause Dag errors: diff --git a/task-sdk/src/airflow/sdk/definitions/dag.py b/task-sdk/src/airflow/sdk/definitions/dag.py index 6523dead760..5c4b139eb75 100644 --- a/task-sdk/src/airflow/sdk/definitions/dag.py +++ b/task-sdk/src/airflow/sdk/definitions/dag.py @@ -335,7 +335,8 @@ class DAG: beyond this the scheduler will disable the DAG :param dagrun_timeout: Specify the duration a DagRun should be allowed to run before it times out or fails. Task instances that are running when a DagRun is timed out will be marked as skipped. - :param sla_miss_callback: DEPRECATED - The SLA feature is removed in Airflow 3.0, to be replaced with a new implementation in 3.1 + :param sla_miss_callback: DEPRECATED - The SLA feature is removed in Airflow 3.0, to be replaced with DeadlineAlerts in 3.1 + :param deadline: An optional DeadlineAlert for the Dag. :param catchup: Perform scheduler catchup (or only run latest)? Defaults to False :param on_failure_callback: A function or list of functions to be called when a DagRun of this dag fails. A context dictionary is passed as a single parameter to this function.
