jason810496 opened a new pull request, #69300: URL: https://github.com/apache/airflow/pull/69300
related: depends on #69299 (merge that first); no conflict with `refactor/logging/configurable-running-pod-log-lines` (any order after core) ## Why `KubernetesExecutor.get_task_log` materializes the whole pod log in the API server. Implementing the new `BaseExecutor.get_streaming_task_log` yields lines lazily into the bounded `LogStreamAccumulator`: ~11.6x lower peak heap growth (+2093.9 MiB vs +179.9 MiB) serving a ~415 MB running-task log (full benchmark in #69299). ## What - Implement `get_streaming_task_log` on `KubernetesExecutor`; `get_task_log` stays and delegates to it, so older cores keep working. - Advertise `supports_streaming_logs` on `KubernetesExecutor`, `CeleryKubernetesExecutor`, and `LocalKubernetesExecutor` (the wrappers route kubernetes-queue tasks). - Update log handler tests to assert the streaming read path for running tasks. Note: the diff includes the #69299 commit until it merges; only the last commit is this PR. --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Fable 5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
