jedcunningham commented on code in PR #39336: URL: https://github.com/apache/airflow/pull/39336#discussion_r1594459245
########## newsfragments/39336.significant.rst: ########## @@ -0,0 +1,5 @@ +``try_number`` is no longer incremented during task execution + +Previously, the try_number was incremented at the beginning of task execution on the worker. This was problematic for many reasons. For one it meant that the try_number was incremented when it was not supposed to, namely when resuming from reschedule or deferral. And it also resulted in the try_number being "wrong" when the task had not yet started. The workarounds for these two issues caused a lot of confusion. What we do instead now, is we only increment the try_number when it is scheduled by the scheduler. So the try number for a task run is determined in advanced, and does not change in flight, and it is never decremented. So after the task runs, the observed try_number be remain the same as it was when the task was running; only when there is a "new try" will the try_number be incremented again. + +One consequence of this change is, if users were "manually" running tasks (e.g. by calling ``ti.run()`` directly, or command line ``airflow tasks run``), try_number will no longer be incremented. Airflow assumes that tasks are always run after being scheduled by the scheduler, so we do not regard this as a breaking change. Review Comment: ```suggestion Previously, the try number (``try_number``) was incremented at the beginning of task execution on the worker. This was problematic for many reasons. For one it meant that the try number was incremented when it was not supposed to, namely when resuming from reschedule or deferral. And it also resulted in the try number being "wrong" when the task had not yet started. The workarounds for these two issues caused a lot of confusion. Now, instead, the try number for a task run is determined at the time the task is scheduled, and does not change in flight, and it is never decremented. So after the task runs, the observed try number remains the same as it was when the task was running; only when there is a "new try" will the try number be incremented again. One consequence of this change is, if users were "manually" running tasks (e.g. by calling ``ti.run()`` directly, or command line ``airflow tasks run``), try number will no longer be incremented. Airflow assumes that tasks are always run after being scheduled by the scheduler, so we do not regard this as a breaking change. ``` We could convert a few ``try_number``s to "try number" if we want less inline code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
