potiuk commented on issue #27589: URL: https://github.com/apache/airflow/issues/27589#issuecomment-1316114343
Another option for pin-pointing is to selectively disable certain processes and compare the usage before/after. For example if you see a pod running with multiple processes in it 0 you can delete certain processes in some containers - changing an entrypoint command to run with "sleep 3600" will run - likely everything else that there is to run with something that for sure does not take memory - and you can see which process caused it. On top of that - again switching airfflow back to originl configuration and "vanilla" state might tell you for example that your configuration is your problem and applying configuraiotn (including logging handles, setting default values for host name check and many others might help with pin-pointing. It's almost certain Airlfow in the vanilla state has no leak. With the It would be far too easy to see - so it must be something on your side. The growth you should is pretty catastrophic and it would demand most of airflow installation to restart scheduler every day or so - which does not happen. I also suggest (if you get there to vanilla and the memory is stil growing) to test different airflow versions - maybe what you see is a mistake- and trying various versions might simply give more answers. And finally if you see it in several airflow versions, I would try to run other experiments - replacing scheduler with other components etc. etc. Unfortnately I cannot have access to your system to play with it but if I were you, this is what I'd do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
