ashb commented on issue #14337: URL: https://github.com/apache/airflow/issues/14337#issuecomment-782738720
I have been working on this slowly - my hypothesis is it's a race condition: when the runner is busy it is protected from scale in, it finishes, gets un-protected from scale in, AWS starts terminating it, but before the instance terminates it picks up a new job. Right in time to get hard killed. My in progress fix is to use a lifecycle hook to not get killed instantly. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
