This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 0db0ff14da added difference between Deferrable and Non-Deferrable
Operators (#31840)
0db0ff14da is described below
commit 0db0ff14da449dc3dbfe9577ccdb12db946b9647
Author: Rohan Anand <[email protected]>
AuthorDate: Sat Jun 24 22:10:26 2023 +0530
added difference between Deferrable and Non-Deferrable Operators (#31840)
---------
Co-authored-by: Tzu-ping Chung <[email protected]>
---
.../authoring-and-scheduling/deferring.rst | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/docs/apache-airflow/authoring-and-scheduling/deferring.rst
b/docs/apache-airflow/authoring-and-scheduling/deferring.rst
index a20e3dafe6..4755566670 100644
--- a/docs/apache-airflow/authoring-and-scheduling/deferring.rst
+++ b/docs/apache-airflow/authoring-and-scheduling/deferring.rst
@@ -166,3 +166,25 @@ Airflow tries to only run triggers in one place at once,
and maintains a heartbe
This means it's possible, but unlikely, for triggers to run in multiple places
at once; this is designed into the Trigger contract, however, and entirely
expected. Airflow will de-duplicate events fired when a trigger is running in
multiple places simultaneously, so this process should be transparent to your
Operators.
Note that every extra ``triggerer`` you run will result in an extra persistent
connection to your database.
+
+
+Difference between Mode='reschedule' and Deferrable=True in Sensors
+-------------------------------------------------------------------
+
+In Airflow, Sensors wait for specific conditions to be met before proceeding
with downstream tasks. Sensors have two options for managing idle periods:
mode='reschedule' and deferrable=True. As mode='reschedule' is a parameter
specific to the BaseSensorOperator in Airflow, which allows the sensor to
reschedule itself if the condition is not met, whereas, 'deferrable=True' is a
convention used by some operators to indicate that the task can be retried (or
deferred) later, but it is not a [...]
+
++--------------------------------------------------------+--------------------------------------------------------+
+| mode='reschedule' |
deferrable=True |
++========================================================+========================================================+
+| Continuously reschedules itself until condition is met | Pauses execution
when idle, resumes when condition |
+| | changes
|
++--------------------------------------------------------+--------------------------------------------------------+
+| Resource Usage is Higher (repeated execution) | Resource Usage is
Lower (pauses when idle, frees |
+| | up worker slots)
|
++--------------------------------------------------------+--------------------------------------------------------+
+| Conditions expected to change over time | Waiting for
external events or resources |
+| (e.g. file creation) | (e.g. API
response) |
++--------------------------------------------------------+--------------------------------------------------------+
+| Built-in functionality for rescheduling | Requires custom
logic to defer task and handle |
+| | external changes
|
++--------------------------------------------------------+--------------------------------------------------------+
