apport information ** Tags added: apport-collected
** Description changed: We've run into some issues where upgrading the kernel from a 4.10 series to a 4.13 series on Ubuntu 16.04 hosts that make heavy use of inotify causes panics and lockups in the kernel in inotify-related code. Our particular use case seemed to hit these at a rate of one every 30 minutes or so when serving up production traffic. Unfortunately, I have been unable to replicate the issue so far with a simulated load-testing environment. When the issue occurs, we get dmesg entries like "BUG: soft lockup - CPU#0 stuck for 22s!" or "General protection fault: 0000 [#1] SMP PTI". In the soft lockup case, the host is still up but all I/O operations stall indefinitely (e.g. typing "sync" into the console will hang forever). In the protection fault case, the system reboots. I've attached dmesg output from the two cases to this bugreport. We have noticed the issue with the following kernels: - linux-image-4.13.0-1013-gcp - linux-image-4.13.0-1015-gcp - linux-image-4.13.0-36-generic We did _not_ have the issue with - linux-image-4.10.0-32-generic I've submitted this bug report from a system which should be configured identically to our production hosts that were having issue (the affected hosts were immediately rolled back to 4.10). This bug appears to have been fixed upstream as of 4.17-rc3 in this commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d90a10e2444ba5a351fa695917258ff4c5709fa5 I would guess that perhaps this patch should be backported into both the 4.13 HWE and GCP Ubuntu kernel series? Thanks, KJ ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.13.0-1013-gcp 4.13.0-1013.17 ProcVersionSignature: Ubuntu 4.13.0-1013.17-gcp 4.13.16 Uname: Linux 4.13.0-1013-gcp x86_64 ApportVersion: 2.20.1-0ubuntu2.16 Architecture: amd64 Date: Mon May 14 07:58:29 2018 ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: linux-gcp UpgradeStatus: No upgrade log present (probably fresh install) + --- + AlsaDevices: + total 0 + crw-rw---- 1 root audio 116, 1 May 10 07:57 seq + crw-rw---- 1 root audio 116, 33 May 10 07:57 timer + AplayDevices: Error: [Errno 2] No such file or directory + ApportVersion: 2.20.1-0ubuntu2.16 + Architecture: amd64 + ArecordDevices: Error: [Errno 2] No such file or directory + AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: + CRDA: N/A + DistroRelease: Ubuntu 16.04 + IwConfig: Error: [Errno 2] No such file or directory + Lsusb: Error: command ['lsusb'] failed with exit code 1: + MachineType: Google Google Compute Engine + Package: linux (not installed) + PciMultimedia: + + ProcEnviron: + TERM=xterm-256color + PATH=(custom, no user) + LANG=en_US.UTF-8 + SHELL=/bin/bash + ProcFB: + + ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.10.0-32-generic root=UUID=73ea38ed-7fcd-4871-8afa-17d36f4e4bfc ro scsi_mod.use_blk_mq=Y console=ttyS0 + ProcVersionSignature: Ubuntu 4.10.0-32.36~16.04.1-generic 4.10.17 + RelatedPackageVersions: + linux-restricted-modules-4.10.0-32-generic N/A + linux-backports-modules-4.10.0-32-generic N/A + linux-firmware N/A + RfKill: Error: [Errno 2] No such file or directory + Tags: xenial uec-images xenial uec-images + Uname: Linux 4.10.0-32-generic x86_64 + UnreportableReason: The report belongs to a package that is not installed. + UpgradeStatus: No upgrade log present (probably fresh install) + UserGroups: + + WifiSyslog: + + _MarkForUpload: False + dmi.bios.date: 01/01/2011 + dmi.bios.vendor: Google + dmi.bios.version: Google + dmi.board.asset.tag: 98BEC19B-1DEB-1A9F-1146-C6E4D8577ADB + dmi.board.name: Google Compute Engine + dmi.board.vendor: Google + dmi.chassis.type: 1 + dmi.chassis.vendor: Google + dmi.modalias: dmi:bvnGoogle:bvrGoogle:bd01/01/2011:svnGoogle:pnGoogleComputeEngine:pvr:rvnGoogle:rnGoogleComputeEngine:rvr:cvnGoogle:ct1:cvr: + dmi.product.name: Google Compute Engine + dmi.sys.vendor: Google ** Attachment added: "CurrentDmesg.txt" https://bugs.launchpad.net/bugs/1771075/+attachment/5139132/+files/CurrentDmesg.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1771075 Title: General Protection fault in inotify (fixed upstream) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1771075/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
