potiuk commented on issue #27589: URL: https://github.com/apache/airflow/issues/27589#issuecomment-1330306476
Yes. I suggest to upgrade to the latest version and see if it still happens. I also think the overall memory usage is not enough. I read a bit about the subject and the problem is far from simple. Kubernetes not only runs your application but also runs monitoring, tweaks memory use for Pods via kubelet and additionally, when you run monitoring/Prometheus, the agent inside every pod migh impact the memory used by caching some stuff. You can read a lot about it here for example https://github.com/kubernetes-monitoring/kubernetes-mixin/issues/227 Even if you use grafana, grafana can cause the increase - depend on the version. Example here: https://github.com/grafana/loki/issues/5569 I think there is not much we can do in Airflow with WSS reporting showing those numbers unless someone can dig deeper and pin-point the memory usage to Airlfow, and not to other components (especially monitoring impacting the memory usage). I suggest to upgrade to latest versions of everything you have (k8s, grafana, prometheus, airflow) and try again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
