mrk-andreev opened a new issue, #45061: URL: https://github.com/apache/airflow/issues/45061
### Description During the discussion in [PR #43853](https://github.com/apache/airflow/pull/43853) with [@dstandish](https://github.com/dstandish), we noticed that there are two separate implementation flows for collecting logs from init_containers and containers that do similar things. The current implementation processes logs sequentially, which is correct for init_containers because they are executed one after another—each starting only after the previous one has successfully completed. However, for regular containers (not init_containers), this logic isn’t fully accurate since they can run in parallel. Related code are https://github.com/apache/airflow/blob/main/providers/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py#L617 I suggest to implement concurrent logs collection for containers. ### Use case/motivation This might cause issues during long-running executions, as we won’t see logs from other containers until one of them has completed. ### Related issues https://github.com/apache/airflow/pull/43853 ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
