fpiped opened a new pull request, #73211: URL: https://github.com/apache/airflow/pull/73211
`AwsTaskLogFetcher.run` checks the stop flag at the top of its loop, then sleeps, then fetches. When `stop()` arrives while the thread is inside a fetch, or in the gap between a fetch finishing and the next check, the loop ends with no read after the stop, and the events the container wrote at the end of the task are never forwarded to the task log. Both callers stop the fetcher as soon as the task or job has ended, which is exactly when those events appear: `EcsRunTaskOperator.execute` (`finally: self.task_log_fetcher.stop()`) and `BatchClientHook.wait_for_job` (`finally: batch_log_fetcher.stop()`). This forwards the events once more after leaving the loop. The continuation token keeps the extra read from repeating events already seen, and `AwsLogsHook.get_log_events` returns as soon as the stream is exhausted, so the cost is one `get_log_events` call at thread exit. The body of the loop moved into `_forward_log_events` unchanged. `test_run_forwards_the_events_written_before_it_was_stopped` covers it: it fails on the current code (one fetch, the last event never logged) and passes here. The two existing `run` tests each get one more empty page in their `side_effect`, since `run` now always performs that final fetch. Verified locally with `apache-airflow 3.1.8` and the provider installed from this branch: ``` providers/amazon/tests/unit/amazon/aws/utils/test_task_log_fetcher.py 32 passed ruff check / ruff format --check on both files clean ``` `tests/unit/amazon/aws/operators/test_ecs.py`, `test_batch.py` and `hooks/test_batch_client.py` patch `AwsTaskLogFetcher` as a whole and never exercise `run`, so they are unaffected; CI runs them. closes: #73210 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) The behaviour was reproduced against the released provider before the change, the fix and its test were reviewed line by line, and the tests and static checks above were run locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
