Re: [Bug 1861359] Re: swap storms kills interactive use
On Sat, Feb 29, 2020 at 10:06:42AM -, Andrea Righi wrote: > Seth, can you try to see if you can reproduce the problem with the > latest unstable kernel (5.4.0-17.21)? https://launchpad.net/~canonical- > kernel-team/+archive/ubuntu/unstable > > I can't reproduce the problem with it. I have not verified yet, but I > suspect it might be related to this commit: > > d92ff805127f mm: move_pages: report the number of non-attempted pages I've installed this kernel, but haven't rebooted into it. I noticed while installing it, I got a zillion messages: depmod: ERROR: ../libkmod/libkmod.c:515 lookup_builtin_file() could not open builtin file '/lib/modules/5.4.0-17-generic/modules.builtin.bin' Both linux-modules-5.4.0-17-generic and linux-modules-extra-5.4.0-17-generic were installed, so I'm not sure what's gone wrong here. -- Steve Beattie http://NxNW.org/~steve/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861359 Title: swap storms kills interactive use To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1861359/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1861359] Re: swap storms kills interactive use
> What do you have in /proc/sys/vm/swappiness? Could you try to set that > to 0 (kernel prefers to drop file-backed pages instead of swapping out > anonymous pages) and see if the swap out activity is still happening? Unfortunately, this did not solve the problem. Setting swappiness to 0 and re-enabling my swap nearly immediately caused the whole computer to lag whenever firefox had focus -- and swap use jumped to 380M, despite having roughly five gigs free space. Turning swap back off again helped immensely. Thanks -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861359 Title: swap storms kills interactive use To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1861359/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1861359] Re: swap storms kills interactive use
On Mon, Feb 10, 2020 at 07:53:20AM -, Andrea Righi wrote: > OK, so we know that it's not related to the memory cgroup subsystem. But this is a good instinct. It does seem to happen when eg firefox or git is in heavy memory use, not the system as a whole. > Another reason of such unexpected swapping activity could be due to > memory compaction code that is triggering some direct memory reclaim and > forcing to swap out pages. > > What do you have in /proc/sys/vm/swappiness? Could you try to set that > to 0 (kernel prefers to drop file-backed pages instead of swapping out > anonymous pages) and see if the swap out activity is still happening? $ cat /proc/sys/vm/swappiness 60 I'll try 0. > Moreover, we can do some more advanced tracing of the direct memory reclaim > code, for example: > $ sudo kprobe-perf -s 'p:shrink_node' > This doesn't work: # kprobe-perf -s 'p:shrink_node' ERROR: func shrink_node not in /sys/kernel/debug/tracing/available_filter_functions. Either it doesn't exist, or, it might be unsafe to kprobe. Exiting. Use -F to override. # ls -l /sys/kernel/debug/tracing/available_filter_functions ls: cannot access '/sys/kernel/debug/tracing/available_filter_functions': No such file or directory # find /sys/kernel/debug/tracing /sys/kernel/debug/tracing /sys/kernel/debug/tracing/instances /sys/kernel/debug/tracing/trace_stat /sys/kernel/debug/tracing/per_cpu /sys/kernel/debug/tracing/per_cpu/cpu7 /sys/kernel/debug/tracing/per_cpu/cpu6 /sys/kernel/debug/tracing/per_cpu/cpu5 /sys/kernel/debug/tracing/per_cpu/cpu4 /sys/kernel/debug/tracing/per_cpu/cpu3 /sys/kernel/debug/tracing/per_cpu/cpu2 /sys/kernel/debug/tracing/per_cpu/cpu1 /sys/kernel/debug/tracing/per_cpu/cpu0 /sys/kernel/debug/tracing/options # Thanks -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861359 Title: swap storms kills interactive use To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1861359/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
