potiuk commented on issue #14924: URL: https://github.com/apache/airflow/issues/14924#issuecomment-911852026
> I don't get why it would cache files it doesn't even need to look at (when I create dummy folder in the logs folder) Well you do create a cache at the moment you WRITE the file (and when it is flushed to disk). This is simply how linux Page Cache write. You can specifically prevent the file to be written to cache when you save files but this is low-level API and very few systems do it because it has no effect besides dropping the metrics. > Strange thing is that this memory increase is not seen with the web service With the webserver, you likely already reached the memory limits and the whole available memory is used for cache. This is pretty much that happens on any long-running system that creates or reads a lot of files. Maybe you can compare the limits you have there. > And finally, what you say is that it doesn't relate to the memory leak topic that's it ? or maybe the memory leak is a false flag for the freeze @itispankajsingh was concerned about I do not know that. I am just saying that IF you see `container_memory_cache` growing, this is pretty normal and expected and the `container_memory_working_set_bytes` is something that you should rather look at if you want to see if there is a memory leak. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
