TheChyeahhh opened a new pull request, #67752: URL: https://github.com/apache/airflow/pull/67752
## Description Fix the shutdown sequence in `CloudwatchTaskHandler.close()` so that pending log events are flushed to CloudWatch **before** the watchtower handler is closed. ## Root Cause `watchtower.CloudWatchLogHandler.close()` sets `shutting_down=True`, which causes all subsequent `flush()` calls to be no-ops. The previous code called `self.handler.close()` **before** `self.io.upload()`, meaning the flush inside `upload()` → `CloudWatchRemoteLogIO.close()` was silently skipped and queued log events were never sent to CloudWatch Logs. This was particularly visible with the ECS Executor (`AwsEcsExecutor`) because the task process terminates immediately after task completion, giving no opportunity for a second flush cycle. Under `CeleryExecutor`, the worker process lives longer and could sometimes flush cached logs through other paths, masking the bug. ## Fix Swap the order in `CloudwatchTaskHandler.close()`: 1. Call `self.io.upload()` first — this flushes pending events to CloudWatch 2. Then call `self.handler.close()` — this sets `shutting_down=True` and signals the queue threads to exit ## Related Fixes #66475 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
