romsharon98 commented on issue #41098: URL: https://github.com/apache/airflow/issues/41098#issuecomment-2264791386
I think this issue is more a feature then a bug. First let's clarify some definitions. (took from the [documentation](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/sensors.html#sensors)) Because sensors are primarily idle, they have two different modes of running so you can be a bit more efficient about using them: - `poke` (default): The Sensor takes up a worker slot for its entire runtime - `reschedule`: The Sensor takes up a worker slot only when it is checking, and sleeps for a set duration between checks Because you are using `reschedule` it repeat the whole execution and part of it is to clear the Xcom https://github.com/apache/airflow/blob/8a1b84dc54b403985e64a9cdb88e9731e83c8b39/airflow/models/taskinstance.py#L3045-L3047 If you will use `mode="poke"` the Xcom will be saved. (you can read about the tradeoff between those two [here](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/sensors.html#sensors). About `deferrable` as mention [here](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/deferring.html#difference-between-mode-reschedule-and-deferrable-true-in-sensors) it's NOT built-in parameter or mode in Airflow, when using deferrable the Xcom should save (according the comment above the code), If you want you can [implement a deferred sensor](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/deferring.html#writing-deferrable-operators). [here](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/deferring.html#difference-between-mode-reschedule-and-deferrable-true-in-sensors) you can read more about the difference between deferrable and reschedule. Let me know if it was helpful or if you have more questions before closing this issue 😃 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
