uranusjr commented on issue #38632: URL: https://github.com/apache/airflow/issues/38632#issuecomment-2030931931
DEFERRED is the easy one, it is basically the same as RUNNING from the scheduler’s perspective, except it is waiting for the trigger to continue. For a ti to entered the DEFERRED, it must first be in the RUNNING state (because deferral happens in the worker), so it should have set start_date, but not end_date. UP_FOR_RESCHEDULE is similar, except it’s done by a (synchronous) sensor in reschedule mode. It is also basically the same as RUNNING except waiting for the scheduler to pick it up again. Since signalling a reschedule is done in the worker, the ti should have set start_date. The situation with end_state is… complicated (this uncertainly is a part of the reason why sensors are disfavoured). _Personally_ I would prefer if it does not set end_date (nor re-set start_date when the scheduler restarts the ti), but it is what it is and we can’t change that without breaking compatibility (and when we can break compatibility in Airflow 3, we probably want to remove synchronous sensors altogether anyway). I think UP_FOR_RESCHEDULE should be considered a special state that is also finished and pre-scheduling and keep its current behaviour. RESTARTING is also similar (it is actually relative new) in that it’s set after the ti have started, so start_date should have been set. It should keep its start_date until the scheduler picks the ti back up, and end_date should remain null. The same applies for UP_FOR_RETRY. (I think your reference on UP_FOR_RETRY’s end_date is wrong? The code it links does not handle UP_FOR_RETRY, from what I can tell.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
