rohan472000 commented on code in PR #31840: URL: https://github.com/apache/airflow/pull/31840#discussion_r1233215945
########## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ########## @@ -166,3 +166,13 @@ Airflow tries to only run triggers in one place at once, and maintains a heartbe This means it's possible, but unlikely, for triggers to run in multiple places at once; this is designed into the Trigger contract, however, and entirely expected. Airflow will de-duplicate events fired when a trigger is running in multiple places simultaneously, so this process should be transparent to your Operators. Note that every extra ``triggerer`` you run will result in an extra persistent connection to your database. + + +Difference between Mode='reschedule' and Deferrable=True in Sensors +------------------------------------------------------------------- + +In Airflow, Sensors wait for specific conditions to be met before proceeding with downstream tasks. Sensors have two options for managing idle periods: mode='reschedule' and deferrable=True. + +* Mode='reschedule': The sensor continuously reschedules itself until the condition is met or the maximum retries are reached, suitable for conditions expected to change over time.Resource consumption may be higher as the sensor is repeatedly executed. + +* Deferrable=True: The sensor pauses execution when idle, freeing up worker slots for other tasks, ideal for sensors waiting for external events or resources. Requires proper implementation of deferring logic and coordination with the event or resource. Review Comment: For your DAG, it depends on the nature of the condition you're waiting for. If the condition is expected to change over time, use mode='reschedule'. If you're waiting for an external event, use deferrable=True. Consider the trade-off between resource consumption and responsiveness. Choose the option that best aligns with your specific use case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
