[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Changed in: linux (Ubuntu) Assignee: Dan Streetman (ddstreet) => (unassigned) ** Changed in: linux (Ubuntu Xenial) Assignee: Dan Streetman (ddstreet) => (unassigned) ** Changed in: linux (Ubuntu Yakkety) Assignee: Dan Streetman (ddstreet) => (unassigned) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Confirmed Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Still not solved. Bug happens on Fresh install of Kubuntu 21.04 with reccomended HDD distribution on modern Lapotp (Ryzen5, 5500U). And on older Laptop (~2014) with Ubuntu 20.04 & 21.04, Kubuntu 21.04 and Lubuntu 20.04 & 20.10 & 21.04. (experienced often, sometimes every few hours) Kswapd0 process rapidly increases in CPU then uses all resources (no result after hours) necessating reboot. Killing a high Ram process (in the few seconds till all CPU is taken) (p.e. Firefox) stops the Kswapd0 process. Am willing to provide additional data / reformat old laptop for tests etc. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Confirmed Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Still not solved. Linux 5.0.0-29-generic #31-Ubuntu SMP Thu Sep 12 13:05:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux This system has i5-9600K and 16 GB RAM, no swap. This happens everytime I run a VirtualbBox VM. The VM has got 4 GB RAM. I updated my VMs to current default settings for Windows and reduced CPU count to 1. This solved the issue until I ran video compression on the host system concurrently. So the problem still happens when CPU usage is high and a VM is running. This time I succeeded in rescuing the system avoiding a reboot. I switched to console mode and waited a lot of time (like an hour) until I was able to kill VirtualBox. What I saw was: kswapd0 was running crazy. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Confirmed Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Confirmed Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Recently got a new laptop with a fresh install of Ubuntu 18.04 and now updated to kernel 4.19.21-041921-generic. It still happens even with 16 GB of memory and 4GB more in swap on SSD. What is more dangerous is now that laptop has hyperthreading it goes absolutely crazy and heats up insanely. At that point, the system is so frozen no commands can fix it as one never gets to write them. Restarting the machine with a button is the only option. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Confirmed Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Bug watch added: Linux Kernel Bug Tracker #110501 https://bugzilla.kernel.org/show_bug.cgi?id=110501 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Confirmed Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Launchpad has imported 55 comments from the remote bug at https://bugzilla.kernel.org/show_bug.cgi?id=65201. If you reply to an imported comment from within Launchpad, your comment will be sent to the remote bug automatically. Read more about Launchpad's inter-bugtracker facilities at https://help.launchpad.net/InterBugTracking. On 2013-11-19T19:40:40+00:00 nleo wrote: kswapd0 randomly load one core of CPU by 100% Linux localhost 3.12.0-1-ARCH #1 SMP PREEMPT Wed Nov 6 09:06:27 CET 2013 x86_64 GNU/Linux No swap enabled Befor on same laptop was installed Ubuntu 12.04 and kernel 3.2 32-bit pae, and there is no such problem. [root@localhost ~]# free -mh total used free sharedbuffers cached Mem: 3.8G 2.4G 1.3G 0B 150M 508M -/+ buffers/cache: 1.8G 2.0G Swap: 0B 0B 0B [root@localhost ~]# cat /proc/meminfo MemTotal:3935792 kB MemFree: 1381360 kB Buffers: 154216 kB Cached: 533096 kB SwapCached:0 kB Active: 1958896 kB Inactive: 438004 kB Active(anon):1740916 kB Inactive(anon): 136292 kB Active(file): 217980 kB Inactive(file): 301712 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 2064 kB Writeback: 0 kB AnonPages: 1709628 kB Mapped: 196696 kB Shmem:167620 kB Slab: 81516 kB SReclaimable: 61312 kB SUnreclaim:20204 kB KernelStack:1696 kB PageTables:13088 kB NFS_Unstable: 0 kB Bounce:0 kB WritebackTmp: 0 kB CommitLimit: 1967896 kB Committed_AS:3498576 kB VmallocTotal: 34359738367 kB VmallocUsed: 361304 kB VmallocChunk: 34359300731 kB HardwareCorrupted: 0 kB AnonHugePages:157696 kB HugePages_Total: 0 HugePages_Free:0 HugePages_Rsvd:0 HugePages_Surp:0 Hugepagesize: 2048 kB DirectMap4k: 18476 kB DirectMap2M: 4059136 kB And I can't kill it. I heared that it's not good idea, but just for lulz) Reply at: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/comments/0 On 2013-11-20T23:32:02+00:00 atomlin wrote: (In reply to nleo from comment #0) > kswapd0 randomly load one core of CPU by 100% You cannot issue a SIGKILL to 'kswapd' since it is a kernel thread. > CommitLimit: 1967896 kB > Committed_AS:3498576 kB ^^^ Seem to be over committing memory. Reply at: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/comments/1 On 2013-11-22T00:57:01+00:00 akpm wrote: (switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Tue, 19 Nov 2013 19:40:40 + bugzilla-dae...@bugzilla.kernel.org wrote: > https://bugzilla.kernel.org/show_bug.cgi?id=65201 > > Bug ID: 65201 >Summary: kswapd0 randomly high cpu load >Product: Memory Management >Version: 2.5 > Kernel Version: 3.12 > Hardware: x86-64 > OS: Linux > Tree: Mainline > Status: NEW > Severity: normal > Priority: P1 > Component: Other > Assignee: a...@linux-foundation.org > Reporter: n...@nm.ru > Regression: No > > kswapd0 randomly load one core of CPU by 100% > > Linux localhost 3.12.0-1-ARCH #1 SMP PREEMPT Wed Nov 6 09:06:27 CET 2013 > x86_64 > GNU/Linux > > No swap enabled > > Befor on same laptop was installed Ubuntu 12.04 and kernel 3.2 32-bit pae, > and > there is no such problem. > > [root@localhost ~]# free -mh > total used free sharedbuffers cached > Mem: 3.8G 2.4G 1.3G 0B 150M 508M > -/+ buffers/cache: 1.8G 2.0G > Swap: 0B 0B 0B hm, I wonder what kswapd is up to. Could you please make it happen again and then dmesg -n 7 dmesg -c echo m > /proc/sysrq-trigger echo t > /proc/sysrq-trigger dmesg -s 100 > foo then send us foo? > > [root@localhost ~]# cat /proc/meminfo > MemTotal:3935792 kB > MemFree: 1381360 kB > Buffers: 154216 kB > Cached: 533096 kB > SwapCached:0 kB > Active: 1958896 kB > Inactive: 438004 kB > Active(anon):1740916 kB > Inactive(anon): 136292 kB > Active(file): 217980 kB > Inactive(file): 301712 kB > Unevictable: 0 kB > Mlocked: 0 kB > SwapTotal: 0 kB > SwapFree: 0 kB > Dirty: 2064 kB > Writeback: 0 kB > AnonPages: 1709628 kB > Mapped:
Re: [Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Sorry ! Le jeu. 26 oct. 2017 à 14:55, Dan Streetmana écrit : > Mathieu, > > I'm not mailing you, this is a public bug report, you subscribed > yourself to this bug: > > 2017-01-26 10:14:29 Mathieu Perona bug added > subscriber Mathieu Perona > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1518457 > > Title: > kswapd0 100% CPU usage > > Status in Linux: > Unknown > Status in linux package in Ubuntu: > Fix Released > Status in linux source package in Xenial: > Fix Released > Status in linux source package in Yakkety: > Fix Released > > Bug description: > As per bug 721896 and various others: > > I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). > Occasionally (about once a day), kswapd0 falls into a busy loop and > spins on 100% CPU usage indefinitely. This can be provoked by > copying/writing large files (e.g. dding a 256MB file), but it happens > occasionally otherwise. System memory usage (not including > buffers/caches) currently sits at 36%, which is typical[1]. Initially > I had no swap space configured; I've since tried enabling a 256MB swap > file, but the problem continues to occur and no swap space is used. > The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. > > Happy to provide further information/take further debugging actions. > > > [1] Full output from `free`: >total used free sharedbuffers cached > Mem: 1014936 483448 531488 28556 9756 112700 > -/+ buffers/cache: 360992 653944 > Swap: 262140 0 262140 > > ProblemType: Bug > DistroRelease: Ubuntu 15.10 > Package: linux-image-4.2.0-18-generic 4.2.0-18.22 > ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 > Uname: Linux 4.2.0-18-generic x86_64 > AlsaDevices: >total 0 >crw-rw 1 root audio 116, 1 Nov 19 19:40 seq >crw-rw 1 root audio 116, 33 Nov 19 19:40 timer > AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' > ApportVersion: 2.19.1-0ubuntu5 > Architecture: amd64 > ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' > AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', > '/dev/snd/timer'] failed with exit code 1: > CRDA: N/A > Date: Fri Nov 20 20:44:30 2015 > Ec2AMI: ami-1c552a76 > Ec2AMIManifest: (unknown) > Ec2AvailabilityZone: us-east-1d > Ec2InstanceType: t2.micro > Ec2Kernel: unavailable > Ec2Ramdisk: unavailable > IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' > Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to > initialize libusb: -99 > MachineType: Xen HVM domU > PciMultimedia: > > ProcEnviron: >TERM=screen >PATH=(custom, no user) >LANG=en_US.UTF-8 >SHELL=/bin/bash > ProcFB: 0 xen > ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic > root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 > console=ttyS0 net.ifnames=0 > RelatedPackageVersions: >linux-restricted-modules-4.2.0-18-generic N/A >linux-backports-modules-4.2.0-18-generic N/A >linux-firmwareN/A > RfKill: Error: [Errno 2] No such file or directory: 'rfkill' > SourcePackage: linux > UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' > UpgradeStatus: No upgrade log present (probably fresh install) > dmi.bios.date: 05/06/2015 > dmi.bios.vendor: Xen > dmi.bios.version: 4.2.amazon > dmi.chassis.type: 1 > dmi.chassis.vendor: Xen > dmi.modalias: > dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: > dmi.product.name: HVM domU > dmi.product.version: 4.2.amazon > dmi.sys.vendor: Xen > > To manage notifications about this bug go to: > https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions > -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used.
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Mathieu, I'm not mailing you, this is a public bug report, you subscribed yourself to this bug: 2017-01-26 10:14:29 Mathieu Perona bug added subscriber Mathieu Perona -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
Re: [Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Dear Dan, I am not sure you are mailing the right person. Anyway, the issue still arises when copying large files from a FAT drive to an EXT one. kswapd uses all available CPU power (but leaves much of the RAM unused) and copy throughput declines. Anyway, I solved the problem by transferring my external drive to an EXT4 filesystem, at the expense of interoperability. Best Le jeu. 26 oct. 2017 à 13:26, Dan Streetmana écrit : > this bug is fix released. please open a new bug. > > note that you may just have been out of memory. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1518457 > > Title: > kswapd0 100% CPU usage > > Status in Linux: > Unknown > Status in linux package in Ubuntu: > Fix Released > Status in linux source package in Xenial: > Fix Released > Status in linux source package in Yakkety: > Fix Released > > Bug description: > As per bug 721896 and various others: > > I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). > Occasionally (about once a day), kswapd0 falls into a busy loop and > spins on 100% CPU usage indefinitely. This can be provoked by > copying/writing large files (e.g. dding a 256MB file), but it happens > occasionally otherwise. System memory usage (not including > buffers/caches) currently sits at 36%, which is typical[1]. Initially > I had no swap space configured; I've since tried enabling a 256MB swap > file, but the problem continues to occur and no swap space is used. > The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. > > Happy to provide further information/take further debugging actions. > > > [1] Full output from `free`: >total used free sharedbuffers cached > Mem: 1014936 483448 531488 28556 9756 112700 > -/+ buffers/cache: 360992 653944 > Swap: 262140 0 262140 > > ProblemType: Bug > DistroRelease: Ubuntu 15.10 > Package: linux-image-4.2.0-18-generic 4.2.0-18.22 > ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 > Uname: Linux 4.2.0-18-generic x86_64 > AlsaDevices: >total 0 >crw-rw 1 root audio 116, 1 Nov 19 19:40 seq >crw-rw 1 root audio 116, 33 Nov 19 19:40 timer > AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' > ApportVersion: 2.19.1-0ubuntu5 > Architecture: amd64 > ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' > AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', > '/dev/snd/timer'] failed with exit code 1: > CRDA: N/A > Date: Fri Nov 20 20:44:30 2015 > Ec2AMI: ami-1c552a76 > Ec2AMIManifest: (unknown) > Ec2AvailabilityZone: us-east-1d > Ec2InstanceType: t2.micro > Ec2Kernel: unavailable > Ec2Ramdisk: unavailable > IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' > Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to > initialize libusb: -99 > MachineType: Xen HVM domU > PciMultimedia: > > ProcEnviron: >TERM=screen >PATH=(custom, no user) >LANG=en_US.UTF-8 >SHELL=/bin/bash > ProcFB: 0 xen > ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic > root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 > console=ttyS0 net.ifnames=0 > RelatedPackageVersions: >linux-restricted-modules-4.2.0-18-generic N/A >linux-backports-modules-4.2.0-18-generic N/A >linux-firmwareN/A > RfKill: Error: [Errno 2] No such file or directory: 'rfkill' > SourcePackage: linux > UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' > UpgradeStatus: No upgrade log present (probably fresh install) > dmi.bios.date: 05/06/2015 > dmi.bios.vendor: Xen > dmi.bios.version: 4.2.amazon > dmi.chassis.type: 1 > dmi.chassis.vendor: Xen > dmi.modalias: > dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: > dmi.product.name: HVM domU > dmi.product.version: 4.2.amazon > dmi.sys.vendor: Xen > > To manage notifications about this bug go to: > https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions > -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
this bug is fix released. please open a new bug. note that you may just have been out of memory. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
8 months on and one of my systems is exhibiting what would appear to be the same bug... it was running Ubuntu 17.04 happily and just got upgraded to 17.10 with today's updates. (Yeah... upgrade, not reinstall... not enough time to re-download all the Insync files!) These were the first lines of `top` as the computer started going into slowdown: PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 54 root 20 0 0 0 0 R 47.2 0.0 2:46.26 kswapd0 1678 tracey20 0 3917924 139976 4 D 1.2 3.6 1:01.26 insync kswapd0 went higher then it locked up. I have a small swapfile, although the system is running on an SSD. root@tracey-:~# swapon --show NAME TYPE SIZE USED PRIO /swapfile file 2G 498.5M -1 I have vm.swappiness=1 in /etc/sysctl.conf This is memory when the system is running normally: root@tracey-:~# free -h totalusedfree shared buff/cache available Mem: 3.7G2.2G339M998M1.2G392M Swap: 2.0G498M1.5G Suggestions please? Frustrated of Lyme Regis -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
As a comment to those expecting that disabling swap will prevent kswapd from doing anything, that's incorrect. kswapd also is responsible for clearing out the page cache, so even with swap disabled you'll still see it doing work, especially during heavy IO that uses the page cache. For example, copying large files from fast drives. That's completely normal. What this bug is about, is kswapd trying over and over to do its normal work, but making no progress, so it continues to use 100% cpu while the rest of the system is doing nothing. That bug should be fixed now. If anyone continues to see *this* bug - kswapd using 100% cpu while your system is doing nothing and kswapd never recovers - you can report it here, but you should also open a new bug and reference it in your comment (this bug is fixed and closed). However, if all you see is kswapd using cpu while you're doing things - especially file IO - that isn't this bug, and it probably isn't actually a bug at all. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
As a complement to my previous comment: both computers have a significant amount of RAM (4 and 8 Go respectively) and the swap partition is shown as not used by the system monitor. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
As of 2017-01-26, I experience this bug on two 16.10 boxes. This kswap0d behavior is 100% reproductible when I copy large files (more than c. 500 Mo) from or to a NTFS hard drive. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This bug is fixed released already, any new problems should be opened in a new bug. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Running 4.8.13-100.fc23.i686+PAE, no desktop, swapon and swapoff, swappiness 0, 60, and 100. kswapd0 usage high while reading from /dev/sda (not mounted, internal SSD with 500+MB/s read). After stop reading, kswapd0 usage is gone. No Problem when reading from USB-HDD. Problem with high-speed reading? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Running Ubuntu 16.10 Yakkety. Old 4.4.0-45-generic kernel works nicely. All the new 4.8.0 kernels eventually go berzerk with kswapd0, just today the newest 4.8.0-32-generic. Normal usage, just a couple of terminals, Chromium, Nautilus windows and Evince PDF documents open. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
> I can see it every now and then with Yakkety and linux 4.8.0-27. > Can you advice any action? what do you mean by "every now and then"? you mean kswapd runs at 100% for a short time, occasionally? that's normal. > kswapd0 no longer takes 100% CPU, but it still emerges to the top of the list > in top as it still > manages to take more then the other processes this sounds like kswapd is just doing its job now. if you stop all your applications, does kswapd still take up cpu time indefinitely? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Skylinux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This bug was fixed in the package linux - 4.8.0-30.32 --- linux (4.8.0-30.32) yakkety; urgency=low * CVE-2016-8655 (LP: #1646318) - packet: fix race condition in packet_set_ring -- Brad FiggThu, 01 Dec 2016 08:02:53 -0800 ** Changed in: linux (Ubuntu) Status: Invalid => Fix Released ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-8655 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Skylinux: Unknown Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Following this thread I had the same issue, running stock 16.04 (xenial) with kernel 4.4.0-38. I have upgraded the kernel to 4.8.10-040810 and the end result is the same but symptoms are a bit different: note: 1GB of RAM, no swap at all (old kernel) with 50% of RAM in buffers / cache, kswapd0 took all the CPU it could (~75%). iostat was pretty much at 0% utilisation. server ground to a halt immediately when free ram expired without ever "eating into" the buffers / cache. (new kernel) again, with 50% of RAM in buffers / cache, kswapd0 no longer takes 100% CPU, but it still emerges to the top of the list in top as it still manages to take more then the other processes. For a difference, now ~75% cpu processes are *WA*iting. A second difference is that now, after free ram is consumed, for a while buffers / cache are also being reduced in favour of the RAM hungry app. However, even this goes only down to about 40 - 43% or RAM being used by buffers / cache. Hopefully this is making at least some sense. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
> Dan Streetman (ddstreet) wrote on 2016-10-01: #127 > > The patch series that fixes this is included in yakkety > (if anyone reproduces this on a yakkety kernel, please let me know), I can see it every now and then with Yakkety and linux 4.8.0-27. Can you advice any action? $ uname -a Linux luc-MacBook 4.8.0-27-generic #29-Ubuntu SMP Thu Oct 20 21:01:44 UTC 2016 i686 i686 i686 GNU/Linux $ lsb_release -a Description:Ubuntu 16.10 Codename: yakkety -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This bug was fixed in the package linux - 4.8.0-27.29 --- linux (4.8.0-27.29) yakkety; urgency=low [ Seth Forshee ] * Release Tracking Bug - LP: #1635377 * proc_keys_show crash when reading /proc/keys (LP: #1634496) - SAUCE: KEYS: ensure xbuf is large enough to fix buffer overflow in proc_keys_show (LP: #1634496) * Revert "If zone is so small that watermarks are the same, stop zone balance" in yakkety (LP: #1632894) - Revert "UBUNTU: SAUCE: (no-up) If zone is so small that watermarks are the same, stop zone balance." * lts-yakkety 4.8 cannot mount lvm raid1 (LP: #1631298) - SAUCE: (no-up) dm raid: fix compat_features validation * kswapd0 100% CPU usage (LP: #1518457) - SAUCE: (no-up) If zone is so small that watermarks are the same, stop zone balance. * [Trusty->Yakkety] powerpc/64: Fix incorrect return value from __copy_tofrom_user (LP: #1632462) - SAUCE: (no-up) powerpc/64: Fix incorrect return value from __copy_tofrom_user * Ubuntu 16.10: Oops panic in move_page_tables/page_remove_rmap after running memory_stress_ng. (LP: #1628976) - SAUCE: (no-up) powerpc/pseries: Fix stack corruption in htpe code * Paths not failed properly when unmapping virtual FC ports in VIOS (using ibmvfc) (LP: #1632116) - scsi: ibmvfc: Fix I/O hang when port is not mapped * [Ubuntu16.10]KV4.8: kernel livepatch config options are not set (LP: #1626983) - [Config] Enable live patching on powerpc/ppc64el * CONFIG_AUFS_XATTR is not set (LP: #1557776) - [Config] CONFIG_AUFS_XATTR=y * Yakkety update to 4.8.1 stable release (LP: #1632445) - arm64: debug: avoid resetting stepping state machine when TIF_SINGLESTEP - Using BUG_ON() as an assert() is _never_ acceptable - usb: misc: legousbtower: Fix NULL pointer deference - Staging: fbtft: Fix bug in fbtft-core - usb: usbip: vudc: fix left shift overflow - USB: serial: cp210x: Add ID for a Juniper console - Revert "usbtmc: convert to devm_kzalloc" - ALSA: hda - Adding one more ALC255 pin definition for headset problem - ALSA: hda - Fix headset mic detection problem for several Dell laptops - ALSA: hda - Add the top speaker pin config for HP Spectre x360 - Linux 4.8.1 * PSL data cache should be flushed before resetting CAPI adapter (LP: #1632049) - cxl: Flush PSL cache before resetting the adapter * thunder nic: avoid link delays due to RX_PACKET_DIS (LP: #1630038) - net: thunderx: Don't set RX_PACKET_DIS while initializing * crypto/vmx/p8_ghash memory corruption (LP: #1630970) - crypto: ghash-generic - move common definitions to a new header file - crypto: vmx - Fix memory corruption caused by p8_ghash - crypto: vmx - Ensure ghash-generic is enabled * arm64: SPCR console not autodetected (LP: #1630311) - of/serial: move earlycon early_param handling to serial - [Config] CONFIG_ACPI_SPCR_TABLE=y - ACPI: parse SPCR and enable matching console - ARM64: ACPI: enable ACPI_SPCR_TABLE - serial: pl011: add console matching function * include/linux/security.h header syntax error with !CONFIG_SECURITYFS (LP: #1630990) - SAUCE: (no-up) include/linux/security.h -- fix syntax error with CONFIG_SECURITYFS=n * sha1-powerpc returning wrong results (LP: #1629977) - crypto: sha1-powerpc - little-endian support -- Seth ForsheeThu, 20 Oct 2016 14:09:37 -0500 ** Changed in: linux (Ubuntu Yakkety) Status: Invalid => Fix Released ** Changed in: linux (Ubuntu Yakkety) Status: Invalid => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This bug was fixed in the package linux - 4.8.0-27.29 --- linux (4.8.0-27.29) yakkety; urgency=low [ Seth Forshee ] * Release Tracking Bug - LP: #1635377 * proc_keys_show crash when reading /proc/keys (LP: #1634496) - SAUCE: KEYS: ensure xbuf is large enough to fix buffer overflow in proc_keys_show (LP: #1634496) * Revert "If zone is so small that watermarks are the same, stop zone balance" in yakkety (LP: #1632894) - Revert "UBUNTU: SAUCE: (no-up) If zone is so small that watermarks are the same, stop zone balance." * lts-yakkety 4.8 cannot mount lvm raid1 (LP: #1631298) - SAUCE: (no-up) dm raid: fix compat_features validation * kswapd0 100% CPU usage (LP: #1518457) - SAUCE: (no-up) If zone is so small that watermarks are the same, stop zone balance. * [Trusty->Yakkety] powerpc/64: Fix incorrect return value from __copy_tofrom_user (LP: #1632462) - SAUCE: (no-up) powerpc/64: Fix incorrect return value from __copy_tofrom_user * Ubuntu 16.10: Oops panic in move_page_tables/page_remove_rmap after running memory_stress_ng. (LP: #1628976) - SAUCE: (no-up) powerpc/pseries: Fix stack corruption in htpe code * Paths not failed properly when unmapping virtual FC ports in VIOS (using ibmvfc) (LP: #1632116) - scsi: ibmvfc: Fix I/O hang when port is not mapped * [Ubuntu16.10]KV4.8: kernel livepatch config options are not set (LP: #1626983) - [Config] Enable live patching on powerpc/ppc64el * CONFIG_AUFS_XATTR is not set (LP: #1557776) - [Config] CONFIG_AUFS_XATTR=y * Yakkety update to 4.8.1 stable release (LP: #1632445) - arm64: debug: avoid resetting stepping state machine when TIF_SINGLESTEP - Using BUG_ON() as an assert() is _never_ acceptable - usb: misc: legousbtower: Fix NULL pointer deference - Staging: fbtft: Fix bug in fbtft-core - usb: usbip: vudc: fix left shift overflow - USB: serial: cp210x: Add ID for a Juniper console - Revert "usbtmc: convert to devm_kzalloc" - ALSA: hda - Adding one more ALC255 pin definition for headset problem - ALSA: hda - Fix headset mic detection problem for several Dell laptops - ALSA: hda - Add the top speaker pin config for HP Spectre x360 - Linux 4.8.1 * PSL data cache should be flushed before resetting CAPI adapter (LP: #1632049) - cxl: Flush PSL cache before resetting the adapter * thunder nic: avoid link delays due to RX_PACKET_DIS (LP: #1630038) - net: thunderx: Don't set RX_PACKET_DIS while initializing * crypto/vmx/p8_ghash memory corruption (LP: #1630970) - crypto: ghash-generic - move common definitions to a new header file - crypto: vmx - Fix memory corruption caused by p8_ghash - crypto: vmx - Ensure ghash-generic is enabled * arm64: SPCR console not autodetected (LP: #1630311) - of/serial: move earlycon early_param handling to serial - [Config] CONFIG_ACPI_SPCR_TABLE=y - ACPI: parse SPCR and enable matching console - ARM64: ACPI: enable ACPI_SPCR_TABLE - serial: pl011: add console matching function * include/linux/security.h header syntax error with !CONFIG_SECURITYFS (LP: #1630990) - SAUCE: (no-up) include/linux/security.h -- fix syntax error with CONFIG_SECURITYFS=n * sha1-powerpc returning wrong results (LP: #1629977) - crypto: sha1-powerpc - little-endian support -- Seth ForsheeThu, 20 Oct 2016 14:09:37 -0500 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package:
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
After upgrading to 4.4.0-45 from 4.4.0-21 the issue seems to have gone away. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I have verified that the bug is fixed on 4.8.0-26.28 (yakkety), 4.4.0-43.63 (xenial) and 4.4.0-45.66 (xenial). Doing the same on 4.4.0-42.62 (xenial) reproduced the bug. All tests done on EC2 t2.small. ** Tags removed: verification-needed-yakkety ** Tags added: verification-done-yakkety -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
If the verification apply also on 16.04, it does fix the issue. We had a server that triggered the bug at least once a day (I suspect unattended-upgrade run every morning to trigger it). Since the upgrade - 2 days and half ago - the server had no issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
The fix is already released in 16.04, make sure you have updated to linux-image 4.4.0-43.63 or later. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Will we see this fix make it to 16.04 LTS? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- yakkety' to 'verification-done-yakkety'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-yakkety -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This bug was fixed in the package linux - 4.4.0-43.63 --- linux (4.4.0-43.63) xenial; urgency=low [ Seth Forshee ] * Release Tracking Bug - LP: #1632375 * kswapd0 100% CPU usage (LP: #1518457) - SAUCE: (no-up) If zone is so small that watermarks are the same, stop zone balance. -- Seth ForsheeTue, 11 Oct 2016 07:54:56 -0500 ** Changed in: linux (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Changed in: linux (Ubuntu Yakkety) Status: Fix Committed => Invalid -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Invalid Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Changed in: linux (Ubuntu Yakkety) Status: Confirmed => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Also affects: linux (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Xenial) Status: New => Fix Committed ** Changed in: linux (Ubuntu Xenial) Importance: Undecided => High ** Changed in: linux (Ubuntu Xenial) Assignee: (unassigned) => Dan Streetman (ddstreet) ** Also affects: linux (Ubuntu Yakkety) Importance: High Assignee: Dan Streetman (ddstreet) Status: Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
On review of the patch series, it's simply too large and complex to backport for this situation; it makes, and depends on, a rather large amount of change to the mm subsystem, and there are easier and smaller ways to work around this bug in the xenial kernel. Specifically, a comparison of the Xen balloon driver vs. the virtio balloon driver shows an important difference; while the Xen balloon driver hot-adds memory as soon as it initializes, the virtio driver does not hot-add memory; it only adjusts its size to adjust the amount of free memory. Most importantly, the Xen balloon driver initially hot- adds memory but does not make any (except a very small amount) available for system use. I'm looking at the Xen balloon driver to see how it can be changed to fix this bug. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
The patch series that fixes this is included in yakkety (if anyone reproduces this on a yakkety kernel, please let me know), so this only needs fixing in xenial. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
The problem is a bit complex. The Xen hypervisor uses memory ballooning, to control how many memory pages the guest can use. The kernel enumerates its e820 memory at boot, and since it's only 1G in this case, it all gets placed into the DMA32 zone. Then later during boot when the Xen balloon driver is initialized, it dynamically adds the balloon memory. The kernel always places hot-added memory into the Normal zone however, so the system winds up with the balloon memory, and only the balloon memory, in the Normal zone. Since the balloon driver starts at, or very close to, its memory target, only a very small number of pages are made available, which results in a Normal memory zone that's tiny - only 9 managed pages on the instance I tested. You can read the /proc/zoneinfo file to find the number of managed pages in the Normal zone. Then when the system encounters memory pressure (i.e. very little free memory left), it wakes up the kswapd daemon to start freeing memory. The kswapd daemon then tries to "balance" memory by "balancing" each zone - DMA, DMA32, and Normal zones. However, it's essentially impossible for it to free pages from the Normal zone, because there are so few pages that whenever one is freed, the next page allocation takes it (because pages are usually allocated from the Normal zone first), and kswapd winds up in a continuous cycle of trying to free pages from the Normal zone forever. This is also why disabling the udev memory hotadd (see comment 69) works around the problem - it prevents the Xen balloon driver from adding/enabling any of the pages in the Normal zone, so kswapd never has to bother trying to balance it, and thus there's no problem. This appears to be fixed by Mel Gorman's 34-commit patch series that changes kswapd memory balancing to "per node" instead of "per zone": https://marc.info/?l=linux-mm=146797052519026 That's a rather large patchset to backport to the xenial kernel, but I'll give it a try. ** Changed in: linux (Ubuntu) Assignee: (unassigned) => Dan Streetman (ddstreet) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I have the same issue on 16.04 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Thanks #123 (as easy as?) works for me as a workaround. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
>From man 7 udev: The udev rules are read from the files located in the system rules directory /lib/udev/rules.d, the volatile runtime directory /run/udev/rules.d and the local administration directory /etc/udev/rules.d. All rules files are collectively sorted and processed in lexical order, regardless of the directories in which they live. However, files with identical filenames replace each other. Files in /etc have the highest priority, files in /run take precedence over files with the same name in /lib. This can be used to override a system- supplied rules file with a local file if needed; a symlink in /etc with the same name as a rules file in /lib, pointing to /dev/null, disables the rules file entirely. Rule files must have the extension .rules; other extensions are ignored. As such, the best way to work around this is: sudo ln -s /dev/null /etc/udev/rules.d/40-vm-hotadd.rules Unlike deleting or modifying the original file, this will persist across upgrades without requiring manual conflict resolution. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
Re: [Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Did you try this from Rasmussen up above? sudo touch /etc/udev/rules.d/40-vm-hotadd.rules reboot That fixed it for me on EC2 On Saturday, 3 September 2016, Andy Robertsonwrote: > All of my t2.micro & nano instances are affected by this in AWS EC2 > after upgrading to Ubuntu16. > > doing "echo 1 > /proc/sys/vm/drop_caches" (also tried echo 3) works for > a short period of time, but it comes back within a few minutes. > > I moved a couple instances over to f1.micro on GCP / GCE (1 vCPU, 0.6 GB > memory) and the problem seems to have gone away. I can't do this with > all of my instances yet though so a fix in AWS would be nice. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1518457 > > Title: > kswapd0 100% CPU usage > > Status in Linux: > Unknown > Status in linux package in Ubuntu: > Confirmed > > Bug description: > As per bug 721896 and various others: > > I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). > Occasionally (about once a day), kswapd0 falls into a busy loop and > spins on 100% CPU usage indefinitely. This can be provoked by > copying/writing large files (e.g. dding a 256MB file), but it happens > occasionally otherwise. System memory usage (not including > buffers/caches) currently sits at 36%, which is typical[1]. Initially > I had no swap space configured; I've since tried enabling a 256MB swap > file, but the problem continues to occur and no swap space is used. > The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. > > Happy to provide further information/take further debugging actions. > > > [1] Full output from `free`: >total used free sharedbuffers cached > Mem: 1014936 483448 531488 28556 9756 112700 > -/+ buffers/cache: 360992 653944 > Swap: 262140 0 262140 > > ProblemType: Bug > DistroRelease: Ubuntu 15.10 > Package: linux-image-4.2.0-18-generic 4.2.0-18.22 > ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 > Uname: Linux 4.2.0-18-generic x86_64 > AlsaDevices: >total 0 >crw-rw 1 root audio 116, 1 Nov 19 19:40 seq >crw-rw 1 root audio 116, 33 Nov 19 19:40 timer > AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' > ApportVersion: 2.19.1-0ubuntu5 > Architecture: amd64 > ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' > AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', > '/dev/snd/timer'] failed with exit code 1: > CRDA: N/A > Date: Fri Nov 20 20:44:30 2015 > Ec2AMI: ami-1c552a76 > Ec2AMIManifest: (unknown) > Ec2AvailabilityZone: us-east-1d > Ec2InstanceType: t2.micro > Ec2Kernel: unavailable > Ec2Ramdisk: unavailable > IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' > Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to > initialize libusb: -99 > MachineType: Xen HVM domU > PciMultimedia: > > ProcEnviron: >TERM=screen >PATH=(custom, no user) >LANG=en_US.UTF-8 >SHELL=/bin/bash > ProcFB: 0 xen > ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic > root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 > console=ttyS0 net.ifnames=0 > RelatedPackageVersions: >linux-restricted-modules-4.2.0-18-generic N/A >linux-backports-modules-4.2.0-18-generic N/A >linux-firmwareN/A > RfKill: Error: [Errno 2] No such file or directory: 'rfkill' > SourcePackage: linux > UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' > UpgradeStatus: No upgrade log present (probably fresh install) > dmi.bios.date: 05/06/2015 > dmi.bios.vendor: Xen > dmi.bios.version: 4.2.amazon > dmi.chassis.type: 1 > dmi.chassis.vendor: Xen > dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/ > 06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: > dmi.product.name: HVM domU > dmi.product.version: 4.2.amazon > dmi.sys.vendor: Xen > > To manage notifications about this bug go to: > https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions > -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/comments/69 Seems to have resolved all this issues on AWS t2.micro instances. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
All of my t2.micro & nano instances are affected by this in AWS EC2 after upgrading to Ubuntu16. doing "echo 1 > /proc/sys/vm/drop_caches" (also tried echo 3) works for a short period of time, but it comes back within a few minutes. I moved a couple instances over to f1.micro on GCP / GCE (1 vCPU, 0.6 GB memory) and the problem seems to have gone away. I can't do this with all of my instances yet though so a fix in AWS would be nice. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I wrote a tiny batch script to reliably reproduce the bug. It mounts a tmpfs filesystem and writes a file that fills 98% of the currently available memory. You can also pass it a custom percentage, like: ./fillmem.sh 95 <95% is hit and miss on a newly launched instance. 98% (the default) has inmediately spun kswapd to 100% on all of my tests. -- @andrewtappert: steal time means the T2 instance has run out of CPU credits. They launch with just enough credits to burst the CPU to 100% for 30 minutes. I'm always getting SYS time on kswapd, until I hit the T2 credits limit. ** Attachment added: "fillmem.sh" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+attachment/4715197/+files/fillmem.sh -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
The original description says "kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely". But I think, while the effect may be similar, the actual behavior is a bit different. I think what is happening is that kswapd is accessing pages of memory that are causing the hypervisor (rather than the kernel) to do extra work. If you look at the overall CPU utilization of the instance, you'll see high "st" (steal) time. This can also be provoked manually, for example by trying to read via /proc/kcore from the extra memory region that has been identified in discussion above (for example, try to do a full memory dump with the Volatility getkcore tool). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
To add - these servers are all built on the official Ubuntu Amazon EC2 AMIs of the 'ebs-ssd' variety. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
We have been seeing this issue intermittently on a set of servers that were running Ubuntu 15.10 and then 16.04. After overriding that udev vm hotadd rule as suggested above a couple weeks ago, the issue has yet to return (not a conclusive result, but so far so good). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I have run into this issue when using the goofys s3 fuse filesystem (https://github.com/kahing/goofys) on a t2.small instances when copying large files (which causes many memory buffers to be allocated). I think anything that stresses the memory subsystem will be able to trigger it. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I've tried this on t1.micro (PV) and t2.micro (HVM) instances in eu-west-1. To reproduce, I used the following two commands: sudo apt install docker.io sudo docker run -p 80:8080 cptactionhank/atlassian-jira The startup should work, but navigate to http://instanceaddress/ and choose "I'll set it up myself" and "Built In" database. 10-20 seconds after you click "Next", you should see the memory being exhausted and kswapd0 use half of the CPU time. Test results: ubuntu/images/ebs-ssd/ubuntu-xenial-16.04-amd64-server-20160721 (PV): kswapd0 OK ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20160721 (HVM): kswapd0 high CPU usage It's worth noting that the memory blocks varies between distro and PV/HVM: Amazon Linux HVM: /sys/devices/system/memory/memory[0-7] RHEL 7.2 HVM: /sys/devices/system/memory/memory[0-7] Ubuntu 16.04 HVM: /sys/devices/system/memory/memory[0-8] Ubuntu 16.04 PV: /sys/devices/system/memory/memory[0-4] Why Ubuntu on HVM has an extra memory block is a mystery. It seems to be offline by default, but enabled by the udev hotadd rule. And EC2 doesn't support hotadd. As Joern Heissler suggested, why not remove the hotadd rule from the official images as a workaround? Although the underlying problem probably is related to why the additional memory block is there at all. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Is this known to affect paravirtualized instances, or is it restricted to hvm? Can anyone tell me what conditions I need to create this in a fresh instance? I'll spin up a PV t2.nano and see if I can reproduce it there. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Can this workaround please be added to the official EC2 images? EC2 cannot hot swap CPU or memory from what I know. Having this workaround built in would mean not having to reboot every newly launched instance. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
The suggested workaround works for me also on AWS instances with 15.10: sudo touch /etc/udev/rules.d/40-vm-hotadd.rules reboot Been stable now for over a week. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Is there a fix for this that doesn't require a reboot / something I could add to an ec2 instance's user_data? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
Re: [Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Thank you so much Rasmus! Your solution worked for me: sudo touch /etc/udev/rules.d/40-vm-hotadd.rules reboot I could always trigger the CPU usage bug with this: stress --cpu 8 --io 4 --vm 7 --vm-bytes 128M --vm-hang 3 --timeout 60s If running it once didn't work, a second run would do it. Now I've run it over and over and kswap hits 1-2% during the stress test and drops back down as soon as it ends. Like Rasmus says, it's an AWS instance which runs on Xen. On 17 June 2016 at 07:37, Rasmus Larsenwrote: > I had this issue too on AWS. > > In my case, it was the udev rule for vm-hotadd and the fix as mentioned > previously basically came down to "touch /etc/udev/rules.d/40-vm- > hotadd.rules" which effectively disables the /lib/udev/rules.d/40-vm- > hotadd.rules file (after a reboot). > > The udev rule basically seems to only be active for Xen or Hyper-V and > while it seems the Hyper-V stuff was also present in previous versions, > the Xen stuff seems to be introduced in 15.10 or newer. > > So if you're seeing this issue on anything running on Xen, including > AWS, try: > > touch /etc/udev/rules.d/40-vm-hotadd.rules > reboot > > This is probably a bug in Xen or a bug in the kernel. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1518457 > > Title: > kswapd0 100% CPU usage > > Status in Linux: > Unknown > Status in linux package in Ubuntu: > Confirmed > > Bug description: > As per bug 721896 and various others: > > I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). > Occasionally (about once a day), kswapd0 falls into a busy loop and > spins on 100% CPU usage indefinitely. This can be provoked by > copying/writing large files (e.g. dding a 256MB file), but it happens > occasionally otherwise. System memory usage (not including > buffers/caches) currently sits at 36%, which is typical[1]. Initially > I had no swap space configured; I've since tried enabling a 256MB swap > file, but the problem continues to occur and no swap space is used. > The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. > > Happy to provide further information/take further debugging actions. > > > [1] Full output from `free`: >total used free sharedbuffers cached > Mem: 1014936 483448 531488 28556 9756 112700 > -/+ buffers/cache: 360992 653944 > Swap: 262140 0 262140 > > ProblemType: Bug > DistroRelease: Ubuntu 15.10 > Package: linux-image-4.2.0-18-generic 4.2.0-18.22 > ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 > Uname: Linux 4.2.0-18-generic x86_64 > AlsaDevices: >total 0 >crw-rw 1 root audio 116, 1 Nov 19 19:40 seq >crw-rw 1 root audio 116, 33 Nov 19 19:40 timer > AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' > ApportVersion: 2.19.1-0ubuntu5 > Architecture: amd64 > ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' > AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', > '/dev/snd/timer'] failed with exit code 1: > CRDA: N/A > Date: Fri Nov 20 20:44:30 2015 > Ec2AMI: ami-1c552a76 > Ec2AMIManifest: (unknown) > Ec2AvailabilityZone: us-east-1d > Ec2InstanceType: t2.micro > Ec2Kernel: unavailable > Ec2Ramdisk: unavailable > IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' > Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to > initialize libusb: -99 > MachineType: Xen HVM domU > PciMultimedia: > > ProcEnviron: >TERM=screen >PATH=(custom, no user) >LANG=en_US.UTF-8 >SHELL=/bin/bash > ProcFB: 0 xen > ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic > root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 > console=ttyS0 net.ifnames=0 > RelatedPackageVersions: >linux-restricted-modules-4.2.0-18-generic N/A >linux-backports-modules-4.2.0-18-generic N/A >linux-firmwareN/A > RfKill: Error: [Errno 2] No such file or directory: 'rfkill' > SourcePackage: linux > UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' > UpgradeStatus: No upgrade log present (probably fresh install) > dmi.bios.date: 05/06/2015 > dmi.bios.vendor: Xen > dmi.bios.version: 4.2.amazon > dmi.chassis.type: 1 > dmi.chassis.vendor: Xen > dmi.modalias: > dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: > dmi.product.name: HVM domU > dmi.product.version: 4.2.amazon > dmi.sys.vendor: Xen > > To manage notifications about this bug go to: > https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions > -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I had this issue too on AWS. In my case, it was the udev rule for vm-hotadd and the fix as mentioned previously basically came down to "touch /etc/udev/rules.d/40-vm- hotadd.rules" which effectively disables the /lib/udev/rules.d/40-vm- hotadd.rules file (after a reboot). The udev rule basically seems to only be active for Xen or Hyper-V and while it seems the Hyper-V stuff was also present in previous versions, the Xen stuff seems to be introduced in 15.10 or newer. So if you're seeing this issue on anything running on Xen, including AWS, try: touch /etc/udev/rules.d/40-vm-hotadd.rules reboot This is probably a bug in Xen or a bug in the kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I'm also facing the kswap0 99% CPU issue. Typically, I notice it start minutes after boot time, but restarting the MongoDB process on the server will temporarily fix it, however the process will usually pop up again randomly. I used to run test instances of MongoDB on AWS t2.micros with no issues but recently am being plagued by the kswap0 issue after I tried to upgrade the base AMI to Ubuntu 16.04. uname -a Linux hostname 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can confirm that this issue also exist on AWS c4.large (vCPU 2, mem 3,75 GiB). $ uname -a Linux server 4.4.0-22-generic #39-Ubuntu SMP Thu May 5 16:53:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux $ free -m totalusedfree shared buff/cache available Mem: 376213071204 6912512321 Swap: 2047 392008 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Is the ability to repro a hold-up on this issue? I can spend time to provide STRs on an EC2 instance using the latest official 16.04 image, but I don't want to go through the effort if repro isn't really a hold- up. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Whoops, it does seem to happen on 4.6.1-2, although the condition are different. It seems, does anyone have a sure fire repro case? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I've fixed this in Arch Linux by moving to the 4.6.1-2-ARCH kernel, when previously on the 4.5.? kernel. This problem also doesn't occur in either the 4.0.7-2 or the 3.16-2 kernels for me at least. I hear this is happening mostly on the 4.4.0-* kernel, which doesn't seem to be an option in the Arch Linux archive(https://archive.archlinux.org/packages/l/linux/), in fact none of the *.*.0-* kernels are, if you can report a kernel that's in that list I can test it. I have a feeling simply reinstalling any kernel would have fixed it, and I had some sort of configuration problem before. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
Re: [Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I've had the issue mostly when running with no swap. I added 1GB swap file and it fixed it on the machine with less memory usage, but the machine with the database (using more memory most of the time) still has the bug occur about once a day. Ubuntu 16.04 for both, not happening on 14.04. Sent from my iPhone > On Jun 4, 2016, at 5:11 PM, Jonathan Vargas <1518...@bugs.launchpad.net> > wrote: > > I face the same issue, running a t2.micro (1 GB) for a Nginx/Ruby > application with really low demand. The kswapd0 process takes the CPU to > 100% and the overall system performance is downgraded. > > Linux server 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC > 2016 x86_64 x86_64 x86_64 GNU/Linux > > To reproduce it, I just create a 1GB file in the filesystem using "dd": > > if=/dev/zero of=temp bs=1k count=1024k > > I created a swap file to solve the issue but it continues. After > dropping the page cache with: > > echo 3 > /proc/sys/vm/drop_caches > > the issue is solved temporarily. > > > ** Attachment added: "Selection_320.png" > > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+attachment/4677030/+files/Selection_320.png > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1518457 > > Title: > kswapd0 100% CPU usage > > Status in Linux: > Unknown > Status in linux package in Ubuntu: > Confirmed > > Bug description: > As per bug 721896 and various others: > > I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). > Occasionally (about once a day), kswapd0 falls into a busy loop and > spins on 100% CPU usage indefinitely. This can be provoked by > copying/writing large files (e.g. dding a 256MB file), but it happens > occasionally otherwise. System memory usage (not including > buffers/caches) currently sits at 36%, which is typical[1]. Initially > I had no swap space configured; I've since tried enabling a 256MB swap > file, but the problem continues to occur and no swap space is used. > The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. > > Happy to provide further information/take further debugging actions. > > > [1] Full output from `free`: > total used free sharedbuffers cached > Mem: 1014936 483448 531488 28556 9756 112700 > -/+ buffers/cache: 360992 653944 > Swap: 262140 0 262140 > > ProblemType: Bug > DistroRelease: Ubuntu 15.10 > Package: linux-image-4.2.0-18-generic 4.2.0-18.22 > ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 > Uname: Linux 4.2.0-18-generic x86_64 > AlsaDevices: > total 0 > crw-rw 1 root audio 116, 1 Nov 19 19:40 seq > crw-rw 1 root audio 116, 33 Nov 19 19:40 timer > AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' > ApportVersion: 2.19.1-0ubuntu5 > Architecture: amd64 > ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' > AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', > '/dev/snd/timer'] failed with exit code 1: > CRDA: N/A > Date: Fri Nov 20 20:44:30 2015 > Ec2AMI: ami-1c552a76 > Ec2AMIManifest: (unknown) > Ec2AvailabilityZone: us-east-1d > Ec2InstanceType: t2.micro > Ec2Kernel: unavailable > Ec2Ramdisk: unavailable > IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' > Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to > initialize libusb: -99 > MachineType: Xen HVM domU > PciMultimedia: > > ProcEnviron: > TERM=screen > PATH=(custom, no user) > LANG=en_US.UTF-8 > SHELL=/bin/bash > ProcFB: 0 xen > ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic > root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 > net.ifnames=0 > RelatedPackageVersions: > linux-restricted-modules-4.2.0-18-generic N/A > linux-backports-modules-4.2.0-18-generic N/A > linux-firmwareN/A > RfKill: Error: [Errno 2] No such file or directory: 'rfkill' > SourcePackage: linux > UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' > UpgradeStatus: No upgrade log present (probably fresh install) > dmi.bios.date: 05/06/2015 > dmi.bios.vendor: Xen > dmi.bios.version: 4.2.amazon > dmi.chassis.type: 1 > dmi.chassis.vendor: Xen > dmi.modalias: > dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: > dmi.product.name: HVM domU > dmi.product.version: 4.2.amazon > dmi.sys.vendor: Xen > > To manage notifications about this bug go to: > https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I face the same issue, running a t2.micro (1 GB) for a Nginx/Ruby application with really low demand. The kswapd0 process takes the CPU to 100% and the overall system performance is downgraded. Linux server 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux To reproduce it, I just create a 1GB file in the filesystem using "dd": if=/dev/zero of=temp bs=1k count=1024k I created a swap file to solve the issue but it continues. After dropping the page cache with: echo 3 > /proc/sys/vm/drop_caches the issue is solved temporarily. ** Attachment added: "Selection_320.png" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+attachment/4677030/+files/Selection_320.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Actually, this is even a problem with no active swap partition. What is the actual problem here? Is kswapd0 just running for no reason? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Nope, setting it to performance doesn't seems to help. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
They seem to be set to "powersave", which is kinda ironic. I've set them to performance and will report back later. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
OOI, does this fix this? | for i in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do sudo sh -c "echo performance > $i"; done If so, was the scaling governor set to "ondemand" by any chance? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can confirm that setting vm.min_free_kbytes=67584 on my 2gb chromebook does not work for me (https://i.imgur.com/I7vEE5C.png) After restarting and running heavy processes, kswapd0 still uses 100% cpu until I reboot or run this follow very unfortunate script. #!/bin/bash if [ "$EUID" -ne 0 ]; then echo "Rerunning as root" sudo $0 exit fi swapoff -a sleep 1 echo 3 > /proc/sys/vm/drop_caches sleep 1 swapon /dev/sda2 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Bug watch added: Linux Kernel Bug Tracker #65201 http://bugzilla.kernel.org/show_bug.cgi?id=65201 ** Also affects: linux via http://bugzilla.kernel.org/show_bug.cgi?id=65201 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in Linux: Unknown Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can confirm after after 6 days uptime: my hotfix (echo "vm.min_free_kbytes=67584" > /etc/sysctl.d/60-workaround-kswapd- allcpu.conf) still works. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can the following hotfix works for me for several days on a 2GB Ram system: echo "vm.min_free_kbytes=67584" > /etc/sysctl.d/60-workaround-kswapd- allcpu.conf after that: reboot. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I last commented when I was on 15.10, but was just hit by this on 16.04. Again, EC2 t2.micro instance. $ uname -a Linux ip-172-31-45-223 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux $ top top - 21:32:32 up 6 days, 20 min, 1 user, load average: 1.01, 1.02, 0.96 Tasks: 135 total, 2 running, 133 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us,100.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 1014560 total, 426728 free, 215352 used, 372480 buff/cache KiB Swap:0 total,0 free,0 used. 733288 avail Mem PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 30 root 20 0 0 0 0 R 99.7 0.0 44:42.28 kswapd0 5434 ubuntu20 0 40512 3620 3032 R 0.3 0.4 0:00.03 top 1 root 20 0 37796 5880 4008 S 0.0 0.6 0:25.73 systemd -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Fresh install of 16.04 on EC2 Nano instance, and kswapd0 takes most of the CPU when running a job with high CPU and high IO. I've resorted to running the following command every minute as a cron task: # m hdom mon dow command *****echo 3 > /proc/sys/vm/drop_caches That works for 10 or 15 seconds, but after that kswapd0 comes back and hogs the CPU again. I'm using less than 20% of available memory, shouldn't need to swap. See attached screenshot of top at the 25-second mark after dropping caches. ** Attachment added: "Screenshot from 2016-05-19 15:25:29.png" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+attachment/4666557/+files/Screenshot%20from%202016-05-19%2015%3A25%3A29.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Same issue here, uname -a 4.4.0-22-generic #39-Ubuntu SMP Thu May 5 16:53:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux free -m totalusedfree shared buff/cache available Mem: 1836 681608 12 1591722 Swap: 1949 01949 (i'll edit this in a few minute, with top result (uptime is 0s) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I'm also seeing this on 16.04 on an AWS nano (~0.5GB RAM). More than happy to help troubleshoot this if people let me know what info is needed. Here's some basic info from my machine: $ uname -a 4.4.0-22-generic #39-Ubuntu SMP Thu May 5 16:53:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux $ free -m totalusedfree shared buff/cache available Mem:486 73 140 20 272 362 Swap: 0 0 0 $ top top - 01:44:48 up 3 days, 2:34, 1 user, load average: 1.00, 1.01, 1.05 Tasks: 131 total, 2 running, 129 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 90.9 sy, 0.0 ni, 9.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 498480 total, 143612 free,75852 used, 279016 buff/cache KiB Swap:0 total,0 free,0 used. 370732 avail Mem PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 29 root 20 0 0 0 0 R 99.9 0.0 4066:36 kswapd0 1 root 20 0 37908 6028 4048 S 0.0 1.2 0:08.79 systemd 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd - CUT THE REST - -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Confirm. Linux impulse-X55VD 4.6.0-040600rc6-generic #201605012031 SMP Mon May 2 00:33:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can see this bug on a system recently upgraded from Ubuntu 15.04 to 16.04. I did not see this behaviour before upgrading. The system is an Intel NUC Desktop with 8 GB RAM and kswapd0 completely locks up the system after a while. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
In the earliest linked report of this bug the deadlock occured also with 8GB RAM (and still the case in 15.10). Will test 16.04 later when it stabilizes. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can confirm this bug is only triggered when the machine got 2-3GB RAM or less. I got two identical machines running Ubuntu 16.04. One got 8Gb of RAM and on this machine kswapd0 doesn't deadlock at 100% CPU Usage. On my 2GB RAM machine the issue is worse than under Ubuntu 15.10 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I can also confirm the bug ist still present in Xubuntu 16.04 LTS desktop 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
confirmed on released version of 16.04 LTS desktop # uname -a Linux laptop 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
After reading 82 comments, I'm not sure if this is a kernel or udev (or other) bug. Any clues if anyone's working on this in relevant upstreams? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This is also affecting Ubuntu 15.10 Kernel 4.2.0-35-generic I have 700MB free RAM 400 MB Cached and kswapd0 is taking more than 99% of the CPU. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
This has also been affecting me on a t2.micro EC2 instance with Ubuntu 15.10, most recently occurred with: 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux For me it often triggers when installing updates (probably kernel ones in particular). Otherwise the server is not doing much so probably not much of a surprise it doesn't trigger on other occasions. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Alexander, please note that if a kernel process name contains "swap" as a substring, it does not immediately mean that this process exclusively does process memory swap-in/swap-out. kswapd is a kernel process name for an important piece of memory management subsystem: it frees pages by flushing them to disk or discarding when the system is low on memory, saving you the trouble of OOMs. This includes both buffer cache and process memory; however, disk buffers are freed first. Generally that means that on a system with disabled swap you _will_ see high cpu% in kswapd when you have almost no memory and buffers have not been synced to disk yet. This is not a bug (but you should consider installng more memory or having a stricter sync policy). However, this discussion is very off topic, since this bug is not an intended behavior. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I don't know, if the following comment be relevant to this particular bug, but here it is: * If there is no swap, the kernel should not try to swap. As simple as this. If there is no swap file or partition, and there is kswapd0 in top, if even it doesn't consume 100% of CPU, it is a bug. Sysadmins are entitled to forbid swapping altogether (because it's evil, as on a web server swapping is effectively deny of service), and the most natural way to do so is not to mount a swap partition. * If there is cached memory, there should be no swapping. Memory must be freed by flushing file buffers first. It is ridiculous that the kerner is generous to filesystems, caching their files, but not to the applications that really need memory, allowing either to swap or fork to fail. At the moment there is a misleading article somewhere in the Ubuntu knowledge base that claims that cached memory is not "eaten" and is as available as free. As long as the kernel is allowed to swap when there are file buffers, the latter are NOT free and available. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
The workaround fixes the issue in Xenial Xerus (development branch) as well. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I confirm that the bug is present on Amazon t2.nano instance with Ubuntu 15.10, kernel 4.2.0-30-generic, and can only be worked around by commenting out line 2 in /lib/udev/rules.d/40-vm-hotadd.rules. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I'm seeing this issue in Ubuntu Xenial Xerus (development branch). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
As part of my workaround/fix I downgraded udev and upgraded RAM from 3GB to 8GB on my homeserver. Now I am experiencing the following which can't be normal either: top - 13:50:48 up 1 day, 19:29, 1 user, load average: 0,11, 0,15, 0,14 tasks: 200 total, 1 running, 198 sleeping, 0 stopped, 1 zombie %Cpu(s): 6,1 be, 2,5 sy, 0,2 ni, 81,6 un, 9,1 wa, 0,0 hi, 0,4 si, 0,0 st KiB Mem: 8067248 total, 4768336 used, 3298912 free, 477524 buffers KiB Swap: 4194300 total, 4170948 used,23352 free. 2803672 cached Mem FilenameTypeSizeUsedPriority /var/cache/swap/swap0 file4194300 4170948 -1 Linux # 4.2.0-25-generic #30-Ubuntu SMP Mon Jan 18 12:31:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Any idea? I hope the system won't crash -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
One more note to anyone else trying to debug this: I can reproduce quite reliably by copying a 3GiB file from S3 onto a gp2 EBS volume using `aws s3 cp`. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
** Changed in: linux (Ubuntu) Status: In Progress => Confirmed ** Changed in: linux (Ubuntu) Assignee: Joseph Salisbury (jsalisbury) => (unassigned) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: Confirmed Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I did some experimentation, and I can reproduce @oystein-gisnas's result that nuking that file from `/lib/udev/rules.d/` and rebooting fixes the issue. However, removing that file and restarting udev does *not* seem to fix the issue. So I suspect the problem is not with udev, but rather with some kernel feature that udev is enabling or toggling as part of processing that file. Also, as an aside, editing that file isn't a good workaround, since upgrading udev may clobber it again. If you instead create an `/etc/udev/rules.d/40-vm-hotadd.rules` containing the edited version, that file will take precedence over the one in `/lib`, and also be robust against udev upgrades. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Confirmed Nelson Elhage's workaround: copy /lib/udev/rules.d/40-vm- hotadd.rules to /etc/udev/rules.d/ and comment out line 2 and reboot. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Some further debugging: On my t2.micro test case, the machine comes up with 9 memory devices, the last of which is offline at boot. Bringing that device online by hand (with the udev rules disabled) triggers the bug. # echo online > /sys/devices/system/memory/memory8/state So something about bringing that node online causes the problem. The udev bugfix causes udev to automatically bring it up, which is why that introduced the issue, but it doesn't seem like udev is really the root problem here. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
I've bisected systemd and found the commit that triggers the bug. http://bazaar.launchpad.net/~thopiekar/systemd/systemd-packaging-ubuntu- wily/revision/1706 seems to be the one. The commit is a bugfix that effectively enables hotadd for Xen. That can explain why more or less only Ubuntu 15.10 is affected, but the underlying mechanism is in the kernel. A working workaround on the latest 15.10 is to comment out line 2 (ATTR{[dmi/id]sys_vendor}=="Xen", GOTO="vm_hotadd_apply") in /lib/udev/rules.d/40-vm-hotadd.rules -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Could someone with good udev/kernel knowledge try to debug what's going wrong with this hotadd? I've tried to run systemd-udevd in debug mode, but it didn't show anything of interest. https://wiki.ubuntu.com/DebuggingUdev is outdated, and I don't know how to go on with debugging. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
A workaround that seems to work for me (and one that doesn't involve dropping caches periodically) is to downgrade udev to 219-7ubuntu6 (as suggested by Øystein above). On 15.10 wily, add vivid-updates to your apt sources to be downgrade to this version of udev. I noticed this bug is more likely to occur on servers with less memory / more memory pressure. I was only able to reproduce it on our t2.micro and t2.small instances on EC2, and using larger EC2 instances was our first workaround. (In fact, after downgrading udev I was also downgrade to smaller EC2 instances again.) This bug also seems more likely to occur on servers with faster disk I/O, given I was only able to reproduce it on instances with gp2 (SSD) storage and not on standard (magnetic) storage on EC2. Our use case is a bunch of EC2 instances all using the Ubuntu 15.10 wily AMI, and currently running the latest 4.2.0-23-generic kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
As an addendum, is there a quick-fix for this? Meaning: can I revert to kernel 3.xsomething easily in Ubuntu? Or are there all types of unwelcome side effects to dropping from new-hotness to old-coldness, kernel-wise? signed, pro linux sysadmin for many years, back in the 2000's, now only recently gotten back into it for only my home purposes, so I'm completely behind on how kernel stuff fits together with things. (where are my /dev/ devices, sob). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Marten Not a fix, but a workaround. (Well, of course every server is different - but works good for me for now!) Create in /etc/cron.hourly/ something like kswapd. chmod +x kswapd Then into it drop: echo 1 > /proc/sys/vm/drop_caches You'll get a lot of [649613.095072] kswapd (29670): drop_caches: 1 in your dmesg/syslog. But for me, this works great. Of course it depends on the bug not occurring within the first hour of the "fix" being applied (i.e. cron.hourly). And clearly this will vary from box to box. But on my AWS server (low traffic web-server with a few other odds and sods on it) this has effectively "solved" this issue enough, until we get a full fix. Sean -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1518457] Re: kswapd0 100% CPU usage
Egads, the thick plottens... turns out I have some type of error in my system. In particular: marten@MacMini-New:/usr/local/bin$ cat /proc/meminfo |grep MemTotal MemTotal: 859916 kB Yes you read that right, 860Mb. That's clearly incorrect. I'll go and see if I can put in something better. For this thread though this might be a good hint on how to debug things. Sean- To your point - Yeah I have a 'fixkswapd' script that says: marten@MacMini-New:/usr/local/bin$ cat fixkswapd schedtool -D -n 19 `pidof kswapd0` echo 1 > /proc/sys/vm/drop_caches Maybe my system has an *actual* memory problem rather than a bug. I'll report back if/when I fix this. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1518457 Title: kswapd0 100% CPU usage Status in linux package in Ubuntu: In Progress Bug description: As per bug 721896 and various others: I'm on an AWS t2.micro instance (Xeon E5-2670, 991MiB of memory). Occasionally (about once a day), kswapd0 falls into a busy loop and spins on 100% CPU usage indefinitely. This can be provoked by copying/writing large files (e.g. dding a 256MB file), but it happens occasionally otherwise. System memory usage (not including buffers/caches) currently sits at 36%, which is typical[1]. Initially I had no swap space configured; I've since tried enabling a 256MB swap file, but the problem continues to occur and no swap space is used. The system can be recovered with `echo 1 > /proc/sys/vm/drop_caches`. Happy to provide further information/take further debugging actions. [1] Full output from `free`: total used free sharedbuffers cached Mem: 1014936 483448 531488 28556 9756 112700 -/+ buffers/cache: 360992 653944 Swap: 262140 0 262140 ProblemType: Bug DistroRelease: Ubuntu 15.10 Package: linux-image-4.2.0-18-generic 4.2.0-18.22 ProcVersionSignature: Ubuntu 4.2.0-18.22-generic 4.2.3 Uname: Linux 4.2.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Nov 19 19:40 seq crw-rw 1 root audio 116, 33 Nov 19 19:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.19.1-0ubuntu5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Fri Nov 20 20:44:30 2015 Ec2AMI: ami-1c552a76 Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1d Ec2InstanceType: t2.micro Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99 MachineType: Xen HVM domU PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 xen ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic root=UUID=35bc01f4-4602-4823-976e-508edef899df ro console=tty1 console=ttyS0 net.ifnames=0 RelatedPackageVersions: linux-restricted-modules-4.2.0-18-generic N/A linux-backports-modules-4.2.0-18-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev' UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 05/06/2015 dmi.bios.vendor: Xen dmi.bios.version: 4.2.amazon dmi.chassis.type: 1 dmi.chassis.vendor: Xen dmi.modalias: dmi:bvnXen:bvr4.2.amazon:bd05/06/2015:svnXen:pnHVMdomU:pvr4.2.amazon:cvnXen:ct1:cvr: dmi.product.name: HVM domU dmi.product.version: 4.2.amazon dmi.sys.vendor: Xen To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518457/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp