virendhar opened a new issue #19817: URL: https://github.com/apache/airflow/issues/19817
### Description ## Objective: Add functionality to Airflow ECSOperator to capture the task execution logs as they happen from AWS CloudWatch to the ECSOperator's task execution logs. ### Use case/motivation ## Present Challenge: When executing Fargate tasks via ECSOperator the logs are not readily available in Airflow task during and after execution. The final status of the container execution is reflected in Airflow ECS Operator task logs. As a result one should navigate the task logs in Airflow to get a handle of the container log path in AWS Cloudwatch and then access the log from AWS Cloudwatch. The current behavior introduces the following challenges, - The ongoing/final execution logs are not captured in Airflow task ( ECS Operator ) log. This typically involves couple of additional steps - which can be avoided. - The visibility is limited to the final status of the task execution ## Solution Mechanism to relay the logs from AWS CloudWatch during execution to Airflow logs as and when they are happening. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
