This change was made by a bot.

** Changed in: linux (Ubuntu)
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1982014

Title:
  kernel panics/hangs after upgrading to linux-
  image-5.4.0-122-generic:amd64

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  We had a series of unattended-upgrades:
  ```
  Start-Date: 2022-07-13  06:19:32
  Commandline: /usr/bin/unattended-upgrade
  Install: linux-modules-extra-5.4.0-122-generic:amd64 (5.4.0-122.138, 
automatic), linux-modules-5.4.0-122-generic:amd64 (5.4.0-122.138, automatic), 
linux-headers-5.4.0-122:amd64 (5.4.0-122.138, automatic), 
linux-headers-5.4.0-122-generic:amd64 (5.4.0-122.138, automatic), 
linux-image-5.4.0-122-generic:amd64 (5.4.0-122.138, automatic)
  Upgrade: linux-headers-generic:amd64 (5.4.0.121.122, 5.4.0.122.123), 
linux-image-generic:amd64 (5.4.0.121.122, 5.4.0.122.123), linux-generic:amd64 
(5.4.0.121.122, 5.4.0.122.123)
  End-Date: 2022-07-13  06:20:56

  Start-Date: 2022-07-13  06:21:00
  Commandline: /usr/bin/unattended-upgrade
  Remove: linux-headers-5.4.0-120-generic:amd64 (5.4.0-120.136), 
linux-headers-5.4.0-120:amd64 (5.4.0-120.136)
  End-Date: 2022-07-13  06:21:05

  Start-Date: 2022-07-13  06:21:09
  Commandline: /usr/bin/unattended-upgrade
  Remove: linux-modules-extra-5.4.0-120-generic:amd64 (5.4.0-120.136), 
linux-modules-5.4.0-120-generic:amd64 (5.4.0-120.136), 
linux-image-5.4.0-120-generic:amd64 (5.4.0-120.136)
  End-Date: 2022-07-13  06:21:28
  ```

  The same day, I performed a manual upgrade:
  ```
  Start-Date: 2022-07-13  14:20:07
  Commandline: apt upgrade
  Upgrade: pdns-backend-pgsql:amd64 (4.6.2-1pdns.focal, 4.6.3-1pdns.focal), 
pdns-server:amd64 (4.6.2-1pdns.focal, 4.6.3-1pdns.focal), git-man:amd64 
(1:2.25.1-1ubuntu3.4, 1:2.25.1-1ubuntu3.5), git:amd64 (1:2.25.1-1ubuntu3.4, 
1:2.25.1-1ubuntu3.5), linux-firmware:amd64 (1.187.31, 1.187.32), 
pdns-backend-bind:amd64 (4.6.2-1pdns.focal, 4.6.3-1pdns.focal)
  End-Date: 2022-07-13  14:22:01
  ```

  followed by a reboot.

  Within a short period, the computer (bare metal, living in a data
  center) would hang (not respond to console, ping, ssh, ...) or panic.

  After a couple of rounds of asking the IT staff to find the box and
  kick it (sometimes they saw an actual report of a panic, and sometimes
  it just didn't show anything), I reverted `linux-firmware:amd64` to
  `1.187.31` and rebooted. This did not fix the problem.

  Eventually we realized that the earlier thing had upgraded the kernel
  separately. Reverting that to the previous (5.4.0-121.137) kernel
  resulted in a reliable happy box.

  ```
  ii  binutils-x86-64-linux-gnu             2.34-6ubuntu1.3                     
         amd64        GNU binary utilities, for x86-64-linux-gnu target
  ii  linux-base                            4.5ubuntu3.7                        
         all          Linux image base package
  ii  linux-firmware                        1.187.31                            
         all          Firmware for Linux kernel drivers
  ii  linux-headers-5.4.0-121               5.4.0-121.137                       
         all          Header files related to Linux kernel version 5.4.0
  ii  linux-headers-5.4.0-121-generic       5.4.0-121.137                       
         amd64        Linux kernel headers for version 5.4.0 on 64 bit x86 SMP
  rc  linux-image-5.4.0-100-generic         5.4.0-100.113                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-104-generic         5.4.0-104.118                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-105-generic         5.4.0-105.119                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-107-generic         5.4.0-107.121                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-109-generic         5.4.0-109.123                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-110-generic         5.4.0-110.124                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-113-generic         5.4.0-113.127                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-117-generic         5.4.0-117.132                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-120-generic         5.4.0-120.136                       
         amd64        Signed kernel image generic
  ii  linux-image-5.4.0-121-generic         5.4.0-121.137                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-122-generic         5.4.0-122.138                       
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-94-generic          5.4.0-94.106                        
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-96-generic          5.4.0-96.109                        
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-97-generic          5.4.0-97.110                        
         amd64        Signed kernel image generic
  rc  linux-image-5.4.0-99-generic          5.4.0-99.112                        
         amd64        Signed kernel image generic
  ii  linux-libc-dev:amd64                  5.4.0-122.138                       
         amd64        Linux Kernel Headers for development
  rc  linux-modules-5.4.0-100-generic       5.4.0-100.113                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-104-generic       5.4.0-104.118                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-105-generic       5.4.0-105.119                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-107-generic       5.4.0-107.121                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-109-generic       5.4.0-109.123                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-110-generic       5.4.0-110.124                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-113-generic       5.4.0-113.127                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-117-generic       5.4.0-117.132                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-120-generic       5.4.0-120.136                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  ii  linux-modules-5.4.0-121-generic       5.4.0-121.137                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-122-generic       5.4.0-122.138                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-62-generic        5.4.0-62.70                         
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-94-generic        5.4.0-94.106                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-96-generic        5.4.0-96.109                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-97-generic        5.4.0-97.110                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-5.4.0-99-generic        5.4.0-99.112                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-100-generic 5.4.0-100.113                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-104-generic 5.4.0-104.118                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-105-generic 5.4.0-105.119                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-107-generic 5.4.0-107.121                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-109-generic 5.4.0-109.123                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-110-generic 5.4.0-110.124                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-113-generic 5.4.0-113.127                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-117-generic 5.4.0-117.132                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-120-generic 5.4.0-120.136                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  ii  linux-modules-extra-5.4.0-121-generic 5.4.0-121.137                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-122-generic 5.4.0-122.138                       
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-94-generic  5.4.0-94.106                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-96-generic  5.4.0-96.109                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-97-generic  5.4.0-97.110                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  rc  linux-modules-extra-5.4.0-99-generic  5.4.0-99.112                        
         amd64        Linux kernel extra modules for version 5.4.0 on 64 bit 
x86 SMP
  ```

  ```
  lsb_release -rd
  Description:  Ubuntu 20.04.4 LTS
  Release:      20.04
  ```

  3) What you expected to happen
  1. Servers that have been reliable for years should not spontaneously 
panic/hang (neither in general, nor after an upgrade, especially not an 
automated one)
  2. Ideally as on Windows, a minidump should be safely generated to a volume 
so that I can provide it to support (or quickly debug it w/o losing many days 
and sleepless nights).

  4) What happened instead
  The computer would panic or hang and necessitate an actual human to go find 
the computer and kick it. During which time, I had to adjust routing so that 
another server could handle this server's job responsibilities. (And I had to 
have others do the same, as it happened repeatedly.)

  ProblemType: Bug
  DistroRelease: Ubuntu 20.04
  Package: linux-image-5.4.0-121-generic 5.4.0-121.137
  ProcVersionSignature: Ubuntu 5.4.0-121.137-generic 5.4.189
  Uname: Linux 5.4.0-121-generic x86_64
  NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
  AlsaVersion: Advanced Linux Sound Architecture Driver Version 
k5.4.0-121-generic.
  ApportVersion: 2.20.11-0ubuntu27.24
  Architecture: amd64
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', 
'/dev/snd/controlC0', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D2c', 
'/dev/snd/pcmC0D1p', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/seq', 
'/dev/snd/timer'] failed with exit code 1:
  Card0.Amixer.info:
   Card hw:0 'MID'/'HDA Intel MID at 0xfbeec000 irq 33'
     Mixer name : 'Realtek ALC889'
     Components : 'HDA:10ec0889,15d9060d,00100004'
     Controls      : 55
     Simple ctrls  : 24
  CasperMD5CheckResult: skip
  Date: Mon Jul 18 13:16:53 2022
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  Lsusb:
   Bus 002 Device 003: ID 14dd:0002 Raritan Computer, Inc. 
   Bus 002 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
   Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
   Bus 001 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
   Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  MachineType: Supermicro C7SIM-Q
  ProcFB: 0 i915drmfb
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-121-generic 
root=UUID=25b80abe-c15a-4298-9a39-b83ed0d7a02f ro net.ifnames=0 biosdevname=0 
quiet splash vt.handoff=7
  RelatedPackageVersions:
   linux-restricted-modules-5.4.0-121-generic N/A
   linux-backports-modules-5.4.0-121-generic  N/A
   linux-firmware                             1.187.31
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  SourcePackage: linux
  UpgradeStatus: Upgraded to focal on 2020-10-21 (635 days ago)
  dmi.bios.date: 08/16/10
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: 1.0b
  dmi.board.asset.tag: To Be Filled By O.E.M.
  dmi.board.name: C7SIM-Q
  dmi.board.vendor: Supermicro
  dmi.board.version: 0123456789
  dmi.chassis.asset.tag: To Be Filled By O.E.M.
  dmi.chassis.type: 24
  dmi.chassis.vendor: Supermicro
  dmi.chassis.version: 0123456789
  dmi.modalias: 
dmi:bvnAmericanMegatrendsInc.:bvr1.0b:bd08/16/10:svnSupermicro:pnC7SIM-Q:pvr0123456789:rvnSupermicro:rnC7SIM-Q:rvr0123456789:cvnSupermicro:ct24:cvr0123456789:
  dmi.product.family: To Be Filled By O.E.M.
  dmi.product.name: C7SIM-Q
  dmi.product.sku: To Be Filled By O.E.M.
  dmi.product.version: 0123456789
  dmi.sys.vendor: Supermicro

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1982014/+subscriptions


-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to