On 23/04/2020 08:18, Sad Clouds wrote:
On Wed, 22 Apr 2020 20:46:08 +0200
tlaro...@polynum.com wrote:
(This can be also simply a memory leak in the program but since the
RAM is not exhausted...).
Without much detail about how much memory, swap, etc your system is
using, it's anyone's guess really. You say "RAM is not exhausted" but
that why would the system be swapping? Maybe you're not looking at it
at the right time, it is possible for a process to malloc() large
amount of memory and then quickly release it, by the time you look at
it, some processes got swapped out, but the memory got released to the
system.
I agree with all this but I'd add one more thing to the mix that might
help your system ride out this better.
If you have both memory intensive and filesystem intensive processes
running on a NetBSD system the kernel filesystem cache can end up
evicting programs running in the background that are inactive which then
take a LONG time to recover. For a system with a reasonable amount of
memory I found the vm.filemin and vm.filemax needed to be tweaked so
that filesystem cache was more likely to be tweaked than program code.
I set:
vm.filemax=10
vm.filemin=1
in /etc/sysctl.conf for a system with 16GB of RAM.
Before I made those changes pkgsrc builds of things like rust and
firefox would end up with over 60% of the RAM dedicated to FS cache
forcing the swap out of the jenkins server process that was running the
build! It would also do the same to live firefox/thunderbird processes
running on the same system and pretty much the only way back was a
reboot as these processes could never swap themselves back in in a sane
amount of time :(
I've also seen the same sort of odd memory consumption behaviour in
scripts that 'just' process text files. I once managed to write a perl
script that parsed firewall logs that ended up consuming all ram on the
system for a reasonable sized log. I never noticed it on FreeBSD and
NetBSD (I just put the disk activity down to the fact it was crunching a
large log file). However when I tried to run the same script on Linux it
would never complete as Linux wasn't able to deal with its memory hog
nature at the time (this was years ago on Linux kernel 2.0).
Mike
Mike
