ChickenBenny commented on PR #74869: URL: https://github.com/apache/airflow/pull/74869#issuecomment-6259369970
Thanks Jason for taking a look! Quick update on where the PR is: killing a timed-out process is already handled in one place by the supervisor. The language SDK coordinators go through ActivitySubprocess too, so Go / TS / Java don't need any changes. The only thing that's still different is where the value comes from: Python still works it out itself and sends it to the supervisor. Here's roughly what I'm planning to change: 1. Drop the is_stub-only check and send execution_timeout for all tasks 2. Have the Python task runner read the value from TIRunContext instead of working it out again 3. Rename SetExecutionTimeout to StartExecutionTimeout and drop timeout_seconds A couple of things I'd like to check first: - When the clock starts: after this change, the supervisor starts the clock as soon as the task starts and restarts it on the Start message. So if a Python task's bundle loading + Dag parsing takes too long, it might get killed before execute() even runs (right now that time isn't counted). Is that OK? - Item 3 touches Andrushika's SetExecutionTimeout. Should that be a separate PR, or just keep it in this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
