PCTEJA commented on issue #77278: URL: https://github.com/apache/airflow/issues/77278#issuecomment-7000415204
Hi, I reproduced this locally against main (fa1aa9de). With each log upload taking 0.1s, a single TriggerStateChanges message with 400 finished triggers holds _handle_request for about 40s, because finalize_trigger_log uploads each log inline, one after another. That is past the 30s runner_health_check_threshold, so heartbeat() would treat the runner as deadlocked, skip the heartbeat, and the scheduler would reassign the triggers. My approach would be to move the finished-trigger log uploads in finalize_trigger_log off the supervisor main loop into a small bounded thread pool, so _handle_request returns promptly and the main loop keeps heartbeating. Uploads would still complete before the log file is closed. Could I work on this? Happy to be assigned if that works. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
