dstandish commented on code in PR #33620:
URL: https://github.com/apache/airflow/pull/33620#discussion_r1322086094


##########
docs/apache-airflow/authoring-and-scheduling/deferring.rst:
##########
@@ -18,45 +18,48 @@
 Deferrable Operators & Triggers
 ===============================
 
-Standard :doc:`Operators </core-concepts/operators>` and :doc:`Sensors 
<../core-concepts/sensors>` take up a full *worker slot* for the entire time 
they are running, even if they are idle; for example, if you only have 100 
worker slots available to run Tasks, and you have 100 DAGs waiting on a Sensor 
that's currently running but idle, then you *cannot run anything else* - even 
though your entire Airflow cluster is essentially idle. ``reschedule`` mode for 
Sensors solves some of this, allowing Sensors to only run at fixed intervals, 
but it is inflexible and only allows using time as the reason to resume, not 
anything else.
+Standard :doc:`Operators </core-concepts/operators>` and :doc:`Sensors 
<../core-concepts/sensors>` take up a full *worker slot* for the entire time 
they are running, even if they are idle. For example, if you only have 100 
worker slots available to run tasks, and you have 100 DAGs waiting on a sensor 
that's currently running but idle, then you *cannot run anything else* - even 
though your entire Airflow cluster is essentially idle. ``reschedule`` mode for 
sensors solves some of this, by allowing sensors to only run at fixed 
intervals, but it is inflexible and only allows using time as the reason to 
resume, not other criteria.
 
-This is where *Deferrable Operators* come in. A deferrable operator is one 
that is written with the ability to suspend itself and free up the worker when 
it knows it has to wait, and hand off the job of resuming it to something 
called a *Trigger*. As a result, while it is suspended (deferred), it is not 
taking up a worker slot and your cluster will have a lot less resources wasted 
on idle Operators or Sensors. Note that by default deferred tasks will not use 
up pool slots, if you would like them to, you can change this by editing the 
pool in question.
+This is where *Deferrable Operators* can be used. When it has nothing to do 
but wait, a deferrable operator can suspend itself and free up the worker for 
other processes. When an operator defers, execution moves to the triggerer, 
where the trigger specified by the operator will run, and the trigger can do 
the polling or waiting required by the operator. Then, when the trigger 
finishes polling or waiting, it sends a signal for the operator to resume its 
execution. During the deferred phase of execution, since work has been 
offloaded to the triggerer, the task no longer occupies a worker slot, and you 
have more free workload capacity. By default, deferred tasks don't use pool 
slots. If you would like them to, you can change this by editing the pool in 
question.

Review Comment:
   ```suggestion
   This is where *Deferrable Operators* can be used. When it has nothing to do 
but wait, an operator can suspend itself and free up the worker for other 
processes by *deferring*. When an operator defers, execution moves to the 
triggerer, where the trigger specified by the operator will run.  The trigger 
can do the polling or waiting required by the operator. Then, when the trigger 
finishes polling or waiting, it sends a signal for the operator to resume its 
execution. During the deferred phase of execution, since work has been 
offloaded to the triggerer, the task no longer occupies a worker slot, and you 
have more free workload capacity. By default, deferred tasks don't use pool 
slots. If you would like them to, you can change this by editing the pool in 
question.
   ```
   
   main point of this suggestion is essentially that... _any_ operator can 
_defer_.... what _defines_ a deferrable operator is the mere fact _that_ it 
defers.  So, the change from "a deferrable operator can defer" to "an operator 
can defer" seems a little more consistent.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to