karenbraganz opened a new issue, #48710: URL: https://github.com/apache/airflow/issues/48710
### Description When a task is in the running state, only the last 100 lines are streamed from the worker pod (with KubernetesExecutor). The entire logs appear once the task is completed. The 100 line limit is hardcoded [here](https://github.com/apache/airflow/blob/4606c446d728e2f2301bf1eb39389e8a380f1a78/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py#L139). We should allow this limit to be configurable or display all logs that have been generated so far instead of only the last 100 lines. ### Use case/motivation Users might need the entire log history or more context to ensure that long running tasks are executing as expected. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
