potiuk commented on issue #35938: URL: https://github.com/apache/airflow/issues/35938#issuecomment-1956742007
BTW. I am not an expert in reading those graphs and I have no idea what kinds of memory it is and what it means/ what does it mean but it does seem you have an interference of some kind. There are at least few level downs you can go. Your pods are running on vms that are running on physical machines and your k8s is running on all of this and decides to allocate resources etc. so maybe that is some kind of interference - betwen the PODS that are running on the same machine for example. Recently for example someone reported memory issues with airflow and turned out that their vms were set to never overcommit memory: https://github.com/apache/airflow/discussions/36891#discussion-6094121 - and that basically meant that when Pod was allocated on a machine, a memory needed were allocated even if it was not used and that did not show in the reports. Maybe you have similar issues but in order to do that you need to deep dive intoy understanding what happens in your systme, I am afraid - no airflow-specific knowledge is helpful here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
