Hi Joachim. Joachim Fahrner - 11.08.17, 17:43: > Linux uses all available more for caching of filesystems. When copying > large files to slow network filesystems (nfs, smb, sshfs, davfs) it > takes a long time until such allocated memory becomes free. When these > network filesystems saturate memory linux becomes very unresponsive. It > can take minutes to start applications. > > Is there a way to limit memory usage of network filesystems?
Not specifically for network filesystems, but for all filesystems. See - /proc/sys/vm/dirty_background_bytes (and ratio… but I´d set it in bytes) - sysctl - /etc/sysctl.d - Kernel documentation: sysctl/vm.txt - If you want to get rid of caches for specific files or run a workload in a way that tells the kernel not to cache, see package nocache Linus once recommended to set this to about what your storage can write out in about 2-3 seconds AFAIR. I concur. I´d set this to about no more than 4-5 seconds. Its enough for Linux to optimize write ordering for rotating disks. Linus said the usual tresholds of 10% was made in times when computers had way less memory. Unfortunately the default value was never replaced by some kind of autotuning. I use this for example: martin@merkaba:~> cat /etc/sysctl.d/vm.conf # Only cache what my SSDs can write a few seconds vm.dirty_background_bytes = 1073741824 vm.dirty_bytes = 2147483648 Especially when local storage reads are faster than NFS writes then its crucical to reduce this to a sane value. This can help especially well in combination with rsync which sync()´s on a file after writing it. I once found a 2x speedup between large and low dirty memory threshold. Also it tends to create bulky writeout before (piling up dirty memory, then having to write it in a sudden… pile it up again and so on). This is the short version (lol!). I teach this in my Linux performance analysis & tuning courses. Beware of old pages about this. They may contain misleading recommendations. Thanks, -- Martin _______________________________________________ Dng mailing list [email protected] https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
