moiseenkov commented on PR #28336: URL: https://github.com/apache/airflow/pull/28336#issuecomment-1447894749
@dstandish , thank you for proposing a nicer approach. You mentioned earlier that we do k8s API calls every chunk. It is true for earlier versions of this PR, but currently the API response is being cached for 120 seconds. Please, take a look at the method `PodLogsConsumer.read_pod()` - it invokes `PodManager.read_pod()` only if the cache is empty or last call was more than `self.read_pod_cache_timeout` seconds ago (by default 120s). Thus there's no such problem now, however the `EventScheduler` approach would look nicer, and the code can always be refactored if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
