[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-05-04 Thread Stefan Bader
*** This bug is a duplicate of bug 1926808 *** https://bugs.launchpad.net/bugs/1926808 All 4 patches were already applied via "Bionic update: upstream stable patchset 2021-04-30" (bug #1926808): e1e50ec65274 mm: memcontrol: fix NR_WRITEBACK leak in memcg and system stats 73b14d6718b0 mm:

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-29 Thread Tim Gardner
Patches submitted: https://lists.ubuntu.com/archives/kernel- team/2021-April/119661.html ** Changed in: linux (Ubuntu Bionic) Assignee: (unassigned) => Tim Gardner (timg-tpi) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-29 Thread Tim Gardner
Andrew - thanks for the testing and results. I'll get these patches submitted for review. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1926081 Title: nr_writeback memory leak in kernel 4.15.0-137+

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-29 Thread Andrew Taylor
Installed and booted into the newly supplied kernel: root@jenkins-lon02-02-general-swarm-node-03:~# dpkg -l | grep linux | grep 144 ii linux-image-unsigned-4.15.0-144-generic 4.15.0-144.148~LP1926081.2 amd64Linux kernel image for version 4.15.0 on 64 bit x86 SMP ii

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-29 Thread Andrew Taylor
Tim, I'll cordon off the machine we've been using again and run some tests with this new kernel, hopefully I'll be able to do it today. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1926081 Title:

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-28 Thread Tim Gardner
Andrew - It was suggested that there are a couple more patches relevant to this issue: mm: fix oom_kill event handling mem_cgroup: make sure moving_account, move_lock_task and stat_cpu in the same cacheline mm: memcg: make sure memory.events is uptodate when waking pollers mm: memcontrol: fix

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-28 Thread Tim Gardner
** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Bionic) Status: New => In Progress ** Changed in: linux (Ubuntu Bionic) Importance: Undecided => Medium -- You received this bug notification because you are a member of

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-28 Thread Tim Gardner
Patches sent: https://lists.ubuntu.com/archives/kernel- team/2021-April/119627.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1926081 Title: nr_writeback memory leak in kernel 4.15.0-137+ To

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-28 Thread Tim Gardner
** Description changed: + SRU Justification - Ubuntu 18.04.5 4.15.0 LTS kernels at version 4.15.0-137 and above contain a memory leak due to the inclusion of patch from the upstream kernel, but not the fix for that patch which was released later. + [Impact] + + Ubuntu 18.04.5 4.15.0 LTS

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Andrew Taylor
Awesome, thank you! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1926081 Title: nr_writeback memory leak in kernel 4.15.0-137+ To manage notifications about this bug go to:

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Tim Gardner
Andrew - thanks for the testing feedback. We'll get these patches included in the next SRU cycle. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1926081 Title: nr_writeback memory leak in kernel

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Andrew Taylor
Good news on the custom -144 kernel you provided: root@jenkins-lon02-02-general-swarm-node-03:~# uname -r 4.15.0-144-generic root@jenkins-lon02-02-general-swarm-node-03:~# dpkg -l | grep 144 ii linux-image-unsigned-4.15.0-144-generic 4.15.0-144.148~LP1926081.1 amd64

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Andrew Taylor
Thank you both, I'll give that kernel a go now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1926081 Title: nr_writeback memory leak in kernel 4.15.0-137+ To manage notifications about this bug

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Guilherme G. Piccoli
That's awesome Andrew, thanks for being so proactive in the issue. For your first LP bug report, you're doing an amazing job! Tim posted above a kernel with both fixes, if you can try it, that'd be good. And I want to apologize, I said a wrong information above - the fixes ARE in linux-stable

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Tim Gardner
Andrew - please find a test kernel at https://kernel.ubuntu.com/~rtg/LP%231926081. wget https://kernel.ubuntu.com/~rtg/LP%231926081/amd64/linux-image-unsigned-4.15.0-144-generic_4.15.0-144.148~LP1926081.1_amd64.deb wget

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Andrew Taylor
So, good news. I pulled in the ubuntu kernel sources and applied the main patch I had previously identified (https://github.com/torvalds/linux/commit/c3cc39118c3610eb6ab4711bc624af7fc48a35fe). Looking at the other patch (https://github.com/torvalds/linux/commit/e27be240df53), that seems to be

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Andrew Taylor
Guilherme, thank you for your kind words :) I have been trying to reproduce this bug on several other systems that I have access to in our cloud account, but I have been unable to reproduce it on a VM (either with SAN or local SSD storage). The main set of servers where this has been seen by us

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-27 Thread Andrew Taylor
apport information ** Tags added: apport-collected ** Description changed: Ubuntu 18.04.5 4.15.0 LTS kernels at version 4.15.0-137 and above contain a memory leak due to the inclusion of patch from the upstream kernel, but not the fix for that patch which was released later. This

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-26 Thread Guilherme G. Piccoli
Hi Andrew, thanks a lot for the very thorough bug report! This far better than apport logs, you basically explained everything that is going on plus what is need to fix it, much appreciated! Based on upstream tree, I see the following 2 commits that carry Fixes tag to the offender you pointed:

[Bug 1926081] Re: nr_writeback memory leak in kernel 4.15.0-137+

2021-04-25 Thread Andrew Taylor
I did not initially run the `apport-collect` command as the servers on which I observed this bug have been upgraded to use the 5.4.0 kernel to mitigate the issue (as mentioned in the initial report), therefore the kernel related information may be misleading. I will endeavour to find a server