shahar1 opened a new issue, #70465: URL: https://github.com/apache/airflow/issues/70465
### Under which category would you file this issue? Providers ### Apache Airflow version main (development) ### What happened and how to reproduce it? When `EcsRunTaskOperator` runs in deferrable mode with `awslogs_region` set to a different region than `region_name`, the CloudWatch logs forwarded **during** deferral are fetched from the wrong region. `EcsRunTaskOperator.execute` passes only `region_name=self.region_name` to `TaskDoneTrigger`: https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/ecs.py#L553-L564 and `TaskDoneTrigger.run` uses that single region for **both** the ECS waiter client and the CloudWatch `get_log_events` client: https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/triggers/ecs.py#L190-L223 So `_forward_logs` queries `awslogs_group`/`log_stream` in the task's region instead of the logs region. The stream is not found there, and the trigger logs "Tried to get logs from stream ... but it didn't exist (yet)" for the whole task duration; no task logs are forwarded while deferred. **Steps to reproduce:** 1. Create an ECS task definition whose `awslogs` driver ships logs to a CloudWatch log group in region B, while the task runs in region A. 2. Run an `EcsRunTaskOperator` with `deferrable=True`, `region_name="<region A>"`, `awslogs_region="<region B>"`, `awslogs_group` and `awslogs_stream_prefix` set. 3. Observe that no container logs are relayed during deferral, only the "didn't exist (yet)" warnings. ### What you think should happen instead? The trigger should read logs from `awslogs_region` (falling back to `region_name` when unset), matching the non-deferrable path. This is the remaining half of the log-region resolution fixed in #70464: that PR made the post-deferral fetch in `execute_complete` use `resolve_awslogs_region()`, but the in-deferral forwarding in `TaskDoneTrigger` still uses the ECS region. The likely fix is a separate log-region parameter on `TaskDoneTrigger` (the single `region_name` is used for two different clients), defaulted for backward compatibility with in-flight serialized triggers. ### Operating System Not Applicable ### Deployment Other ### Apache Airflow Provider(s) amazon ### Versions of Apache Airflow Providers apache-airflow-providers-amazon (main) ### Official Helm Chart version Not Applicable ### Anything else? Found while reviewing #70464. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) --- Drafted-by: Claude Code (Fable 5) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
