kjh0623 commented on PR #70058: URL: https://github.com/apache/airflow/pull/70058#issuecomment-5103773894
Yeah, I've been running this on our on-prem K8s cluster for a while now — it's actually already deployed in our production Airflow and has been working without issues. In our setup the DAG processor keeps the tracking repo fresh on the shared volume, so having every worker pod re-fetch on init was pure overhead. With refresh_on_initialize=False that ~15s per-pod cold start on task startup basically goes away, since it just reuses what's already on disk. The clone-if-missing and explicit refresh() paths still behave the same, so we haven't seen any staleness problems in practice. Happy to share more details about the environment if that'd help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
