dstandish commented on PR #32990: URL: https://github.com/apache/airflow/pull/32990#issuecomment-1688961260
So, what was the case before this change? IIUC: * execution_timeout applied to the overall execution of the task (e.g. duration of a single run of `execute`) * task deferral timeout is applied to the duration of a single deferral, and ignores overal execution_timeout * if task comes out of deferral then the execution_timeout is applied again to the task from the moment it resumes exeuction. so that for example if a task runs, defers, and resumes, and the execution timeout is 1 hr, then the task could run for 1 hr + deferral timeout + 1hr after resuming. do i have that right? it makes me a bit uncomfortable to remove the feature to be able to set a specific deferral timeout on a given deferral. not that it's necessarily all that useful, but it's there. now as to your change, let me see if i understand the new behavior correctly. * execution_timeout will be applied to the overall duration of the task instance, no matter how many defer-resume cycles there are. * if the execution_timeout is 1 hr, then now if phase 1 takes 10 minutes, the deferral timeout will be set to 50 minutes. then if task is deferred for 30 minutes before resuming, then when task resumes, effective execution timeout will be 30 minutes. and so on. * `timeout` will be ignored if passed to self.defer do i have all of that correct? or if i got something wrong, please correct. thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
