[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-03-16 Thread Colin Ian King
** Changed in: linux (Ubuntu) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once a week

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-03-16 Thread Launchpad Bug Tracker
This bug was fixed in the package linux - 4.15.0-91.92 --- linux (4.15.0-91.92) bionic; urgency=medium * bionic/linux: 4.15.0-91.92 -proposed tracker (LP: #1865109) * CVE-2020-2732 - KVM: x86: emulate RDPID - KVM: nVMX: Don't emulate instructions in guest mode - KVM:

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-03-12 Thread Colin Ian King
** No longer affects: zram-config (Ubuntu) ** No longer affects: zram-config (Ubuntu Bionic) ** Changed in: linux (Ubuntu) Status: Incomplete => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-02-21 Thread Colin Ian King
Soak tested the -proposed kernel for 2 hours with no hang occurring. Verified OK. ** Tags removed: verification-needed-bionic ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-02-17 Thread Ubuntu Kernel Bot
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- bionic' to 'verification-done-bionic'. If the problem still exists, change the tag

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-02-13 Thread Khaled El Mously
** Changed in: linux (Ubuntu Bionic) Status: Confirmed => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once a week

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-02-07 Thread Colin Ian King
Cornered this to zswap and not an issue with mm or I/O. Figured out that 3 hours soak testing on each bisect step is the only reliably way to do a bisect. Bisected between 4.20 and 5.0 finally cornered the issue and hence the commits required to fix this. ** Description changed: + == SRU

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-30 Thread Colin Ian King
Running w/o swapfile and zswap and just stress-ng brk and stack stressors with NO file I/O can also lock the system. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-30 Thread Colin Ian King
Couple more notes: 1. Disable file based swap on /swapfile - can reproduce issue 2. Use partition based swap on 2nd disk - can reproduce issue -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-28 Thread Colin Ian King
Captured the hard lock on the following (gdb) stepi 0x8c4e29e5 in ?? () => 0x8c4e29e5: eb ec jmp0x8c4e29d3 (gdb) stepi 0x8c4e29d3 in ?? () => 0x8c4e29d3: 8b 07 mov(%rdi),%eax (gdb) stepi 0x8c4e29d5 in ?? () => 0x8c4e29d5: 85

Re: [Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-16 Thread Luis Rodriguez
It sounds like what I was getting. On Thu, Jan 16, 2020 at 11:05 PM Colin Ian King <1799...@bugs.launchpad.net> wrote: > After quite a bit of experimentation I found that I can reproduce the bug > if I have zram *and* also swap on the filesystem enabled while exercising > the brk stressors and

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-16 Thread Colin Ian King
After quite a bit of experimentation I found that I can reproduce the bug if I have zram *and* also swap on the filesystem enabled while exercising the brk stressors and aiol (to cause lots of I/O). Eventually the system grinds to a halt, we lose interactivity and we eventually get lockups as

Re: [Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-10 Thread Luis Rodriguez
Hi.. I had to remove zram config from my production servers long ago. ... since then I don't have the issue. I was using LXD containers a lot on the hosts with different kind of usage,, But I don't have any other setup at the moment On Fri, Jan 10, 2020 at 12:11 AM Colin Ian King

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-09 Thread Colin Ian King
Can reproduce this with stress-ng exercising high memory pressure scenario using: stress-ng --brk 0 -v --aiol 0 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-09 Thread Colin Ian King
Can reproduce this with stress-ng exercising high memory pressure scenario using: stress-ng --brk 0 -v --aiol 0 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-09 Thread Colin Ian King
I'm assuming the defaults are being used for the moment, this means 50% of total memory being used in total distributed across the number of CPUs, as defined in /usr/bin/init-zram-swapping -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-09 Thread Colin Ian King
It would be useful to know if one has made any specific zram config changes, and if so, what your current config is just to help with the debugging of this issue. ** Changed in: linux (Ubuntu) Status: Confirmed => Incomplete ** Changed in: zram-config (Ubuntu) Status: Confirmed =>

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2020-01-07 Thread Colin Ian King
** Changed in: linux (Ubuntu) Assignee: (unassigned) => Colin Ian King (colin-king) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2019-11-01 Thread David Roberts
** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed ** Changed in: linux (Ubuntu Bionic) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2019-10-30 Thread David Roberts
Other reports of similar issues with zram on kernel 4.15: - https://bbs.archlinux.org/viewtopic.php?id=234951 - https://lore.kernel.org/lkml/0516ae2d-b0fd-92c5-aa92-112ba7bd3...@contabo.de/T/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2019-10-30 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: zram-config (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2019-10-30 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: zram-config (Ubuntu Bionic) Status: New => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2019-10-30 Thread David Roberts
** Also affects: zram-config (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once a week

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2019-07-24 Thread Brad Figg
** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once a week Status in linux package in Ubuntu: Incomplete Status in

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-12-18 Thread Luis Rodriguez
OK.. it is been quite a while with no locks I had it once after the zram config pacakge was removed,, but no other locks since then. kernel version is 4.15.0-33 to 38 in different servers.. I am going to update the servers to latest version reboot, and wait for a little longer. then I am going

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-19 Thread Joseph Salisbury
** Tags removed: kernel-key ** Tags added: kernel-da-key -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once a week Status in linux package

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-05 Thread Stefan Bader
Darn, would have been too good if it had only happened with zram. :( Sounds a bit like quickly catching all CPUs in a spin-dead-lock. And I am not sure right now which path to use for debugging. Try turning on lock debug in the kernel, though that often changes timing in ways that prevent the

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-03 Thread Luis Rodriguez
Got a hot locked with no zram-config installed.. Same behaviour, no log information, can't even type in the console, no ssh, no ping. ALso all the LXD containers don't ping either -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-01 Thread Luis Rodriguez
Correct.. I ould like to give it some more time to see if it doesn't happen. So far so good, no lockups. I hadnt have to restart any server in a week and a half. I'll try to prepare the same setup on another server with zram-config to see if it happens again on that particular server -- You

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-01 Thread Stefan Bader
Darn, wanted to reply earlier. So maybe at least for Luis who sounds to have multiple servers in a test environment, it would be possible to run two otherwise identical servers and only remove zram-config on one. Then one locking up and the other not would be quite good proof. -- You received

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-01 Thread Stéphane Graber
Oh, I am also using zram-config on the affected machine. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title: 4.15 kernel hard lockup about once a week Status in linux package

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-11-01 Thread Luis Rodriguez
In my case it hasn't happen again.. Although I removed package zram- config from the host servers ( I think this is the only difference in software from 16.04 to 18.04 that I added. I would like to either discard or confirm that that it has an effect on the issue -- You received this bug

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-31 Thread Stéphane Graber
Just happened again, though the machine wouldn't reboot at all afterwards, leading to the hosting provider going for a motherboard replacement, so I guess better luck next week with debugging this. -- You received this bug notification because you are a member of Kernel Packages, which is

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-24 Thread Stéphane Graber
The server doesn't respond to pings when locked up. I do have IPMI and console redirection going for my server and have enabled all sysrq now though it's unclear whether I can send those through the BMC yet (as just typing them would obviously send them to my laptop...). I've setup debug console

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-24 Thread Stefan Bader
To add a bit more detail (maybe unrelated but with so little evidence everything helps), when thos lockups happen, is the server at least pingable? Some other idea would be, as long as those servers are accessible enough to see whether sysrq combinations are still handled. Though I fear at least

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-24 Thread Luis Rodriguez
Hello, I sumbitted the report on LXD since that is the only thing I have installed on the server that is actively running as Stéphane mentioned on https://github.com/lxc/lxd/issues/5197 I also thought it maybe hardware issue, but since upgrading to 18.04 in May I have experienced this on a

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-23 Thread Stéphane Graber
Note that I've deleted the wifisyslog and currentdmesg as they're not relevant (current boot) and included information that I'd rather not have exposed publicly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-23 Thread Stéphane Graber
Well, kinda, this is a production server running a lot of publicly visible services, so I can run test kernels on it so long as they don't regress system security. There's also the unfortunate problem that it takes over a week for me to see the problem in most cases and that my last known good

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-23 Thread Stéphane Graber
Oh and whatever kernel I boot needs to have support for ZFS 0.7 or I won't be able to read my drives. ** Tags added: apport-collected ** Description changed: My main server has been running into hard lockups about once a week ever since I switched to the 4.15 Ubuntu 18.04 kernel. When

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-23 Thread Joseph Salisbury
Would you be able to test some kernels? A bisect can be done if we can identify what kernel version introduced this issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1799497 Title:

[Kernel-packages] [Bug 1799497] Re: 4.15 kernel hard lockup about once a week

2018-10-23 Thread Joseph Salisbury
** Changed in: linux (Ubuntu) Importance: Undecided => High ** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Bionic) Importance: Undecided => High ** Tags added: bionic kernel-key -- You received this bug notification