[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
*** This bug is a duplicate of bug 1346917 *** https://bugs.launchpad.net/bugs/1346917 ** This bug has been marked a duplicate of bug 1346917 Using KSM on NUMA capable machines can cause KVM guest performance and stability issues -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Incomplete Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Jul 21 13:53 seq crw-rw 1 root audio 116, 33 Jul 21 13:53 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: Error: [Errno 2] No such file or directory DistroRelease: Ubuntu 14.04 HibernationDevice: RESUME=/dev/mapper/bootdisk-swap_1 IwConfig: Error: [Errno 2] No such file or directory MachineType: Supermicro H8DG6/H8DGi Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 VESA VGA ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-29-generic root=/dev/mapper/bootdisk-root ro quiet splash rootdelay=120 nomdmonddf nomdmonisw vt.handoff=7 ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2 RelatedPackageVersions: linux-restricted-modules-3.13.0-29-generic N/A linux-backports-modules-3.13.0-29-generic N/A linux-firmware 1.127.2 RfKill: Error: [Errno 2] No such file or directory Tags: trusty Uname: Linux 3.13.0-29-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True dmi.bios.date: 11/25/2013 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 3.5 dmi.board.asset.tag: To Be Filled By O.E.M. dmi.board.name: H8DG6/H8DGi dmi.board.vendor: Supermicro dmi.board.version: 1.0 dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 17 dmi.chassis.vendor: Supermicro dmi.chassis.version: 1234567890 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr3.5:bd11/25/2013:svnSupermicro:pnH8DG6/H8DGi:pvr1234567890:rvnSupermicro:rnH8DG6/H8DGi:rvr1.0:cvnSupermicro:ct17:cvr1234567890: dmi.product.name: H8DG6/H8DGi dmi.product.version: 1234567890 dmi.sys.vendor: Supermicro To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
Marking incomplete until requested testing is complete. ** Changed in: linux (Ubuntu) Importance: Undecided => High ** Changed in: linux (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Incomplete Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Jul 21 13:53 seq crw-rw 1 root audio 116, 33 Jul 21 13:53 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: Error: [Errno 2] No such file or directory DistroRelease: Ubuntu 14.04 HibernationDevice: RESUME=/dev/mapper/bootdisk-swap_1 IwConfig: Error: [Errno 2] No such file or directory MachineType: Supermicro H8DG6/H8DGi Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 VESA VGA ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-29-generic root=/dev/mapper/bootdisk-root ro quiet splash rootdelay=120 nomdmonddf nomdmonisw vt.handoff=7 ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2 RelatedPackageVersions: linux-restricted-modules-3.13.0-29-generic N/A linux-backports-modules-3.13.0-29-generic N/A linux-firmware 1.127.2 RfKill: Error: [Errno 2] No such file or directory Tags: trusty Uname: Linux 3.13.0-29-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True dmi.bios.date: 11/25/2013 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 3.5 dmi.board.asset.tag: To Be Filled By O.E.M. dmi.board.name: H8DG6/H8DGi dmi.board.vendor: Supermicro dmi.board.version: 1.0 dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 17 dmi.chassis.vendor: Supermicro dmi.chassis.version: 1234567890 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr3.5:bd11/25/2013:svnSupermicro:pnH8DG6/H8DGi:pvr1234567890:rvnSupermicro:rnH8DG6/H8DGi:rvr1.0:cvnSupermicro:ct17:cvr1234567890: dmi.product.name: H8DG6/H8DGi dmi.product.version: 1234567890 dmi.sys.vendor: Supermicro To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
I believe I've found the fix for this issue for 3.13. If you can, please test the kernel posted on comment #1 on this bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1346917 Make sure KSM is enabled; and any workarounds for this bug are disabled. If this fixes the issue for you, you are welcome to mark this bug as a duplicate of 1346917. Thanks! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Jul 21 13:53 seq crw-rw 1 root audio 116, 33 Jul 21 13:53 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: Error: [Errno 2] No such file or directory DistroRelease: Ubuntu 14.04 HibernationDevice: RESUME=/dev/mapper/bootdisk-swap_1 IwConfig: Error: [Errno 2] No such file or directory MachineType: Supermicro H8DG6/H8DGi Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 VESA VGA ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-29-generic root=/dev/mapper/bootdisk-root ro quiet splash rootdelay=120 nomdmonddf nomdmonisw vt.handoff=7 ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2 RelatedPackageVersions: linux-restricted-modules-3.13.0-29-generic N/A linux-backports-modules-3.13.0-29-generic N/A linux-firmware 1.127.2 RfKill: Error: [Errno 2] No such file or directory Tags: trusty Uname: Linux 3.13.0-29-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True dmi.bios.date: 11/25/2013 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 3.5 dmi.board.asset.tag: To Be Filled By O.E.M. dmi.board.name: H8DG6/H8DGi dmi.board.vendor: Supermicro dmi.board.version: 1.0 dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 17 dmi.chassis.vendor: Supermicro dmi.chassis.version: 1234567890 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr3.5:bd11/25/2013:svnSupermicro:pnH8DG6/H8DGi:pvr1234567890:rvnSupermicro:rnH8DG6/H8DGi:rvr1.0:cvnSupermicro:ct17:cvr1234567890: dmi.product.name: H8DG6/H8DGi dmi.product.version: 1234567890 dmi.sys.vendor: Supermicro To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
Rebooted into the old kernel and disabled KSM. Then ran apport-collect. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Jul 21 13:53 seq crw-rw 1 root audio 116, 33 Jul 21 13:53 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: Error: [Errno 2] No such file or directory DistroRelease: Ubuntu 14.04 HibernationDevice: RESUME=/dev/mapper/bootdisk-swap_1 IwConfig: Error: [Errno 2] No such file or directory MachineType: Supermicro H8DG6/H8DGi Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 VESA VGA ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-29-generic root=/dev/mapper/bootdisk-root ro quiet splash rootdelay=120 nomdmonddf nomdmonisw vt.handoff=7 ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2 RelatedPackageVersions: linux-restricted-modules-3.13.0-29-generic N/A linux-backports-modules-3.13.0-29-generic N/A linux-firmware 1.127.2 RfKill: Error: [Errno 2] No such file or directory Tags: trusty Uname: Linux 3.13.0-29-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True dmi.bios.date: 11/25/2013 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: 3.5 dmi.board.asset.tag: To Be Filled By O.E.M. dmi.board.name: H8DG6/H8DGi dmi.board.vendor: Supermicro dmi.board.version: 1.0 dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 17 dmi.chassis.vendor: Supermicro dmi.chassis.version: 1234567890 dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr3.5:bd11/25/2013:svnSupermicro:pnH8DG6/H8DGi:pvr1234567890:rvnSupermicro:rnH8DG6/H8DGi:rvr1.0:cvnSupermicro:ct17:cvr1234567890: dmi.product.name: H8DG6/H8DGi dmi.product.version: 1234567890 dmi.sys.vendor: Supermicro To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
apport information ** Description changed: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True + --- + AlsaDevices: + total 0 + crw-rw 1 root audio 116, 1 Jul 21 13:53 seq + crw-rw 1 root audio 116, 33 Jul 21 13:53 timer + AplayDevices: Error: [Errno 2] No such file or directory + ApportVersion: 2.14.1-0ubuntu3.2 + Architecture: amd64 + ArecordDevices: Error: [Errno 2] No such file or directory + AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: + CRDA: Error: [Errno 2] No such file or directory + DistroRelease: Ubuntu 14.04 + HibernationDevice: RESUME=/dev/mapper/bootdisk-swap_1 + IwConfig: Error: [Errno 2] No such file or directory + MachineType: Supermicro H8DG6/H8DGi + Package: linux (not installed) + PciMultimedia: + + ProcEnviron: + TERM=screen + PATH=(custom, no user) + XDG_RUNTIME_DIR= + LANG=en_US.UTF-8 + SHELL=/bin/bash + ProcFB: 0 VESA VGA + ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-29-generic root=/dev/mapper/bootdisk-root ro quiet splash rootdelay=120 nomdmonddf nomdmonisw vt.handoff=7 + ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2 + RelatedPackageVersions: + linux-restricted-modules-3.13.0-29-generic N/A + linux-backports-modules-3.13.0-29-generic N/A + linux-firmware 1.127.2 + RfKill: Error: [Errno 2] No such file or directory + Tags: trusty + Uname: Linux 3.13.0-29-generic x86_64 + UpgradeStatus: No upgrade log present (probably fresh install) + UserGroups: root + _MarkForUpload: True + dmi.bios.date: 11/25/2013 + dmi.bios.vendor: American Megatrends Inc. + dmi.bios.version: 3.5 + dmi.board.asset.tag: To Be Filled By O.E.M. + dmi.board.name: H8DG6/H8DGi + dmi.board.vendor: Supermicro + dmi.board.version: 1.0 + dmi.chassis.asset.tag: To Be Filled By O.E.M. + dmi.chassis.type: 17 + dmi.chassis.vendor: Supermicro + dmi.chassis.version: 1234567890 + dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr3.5:bd11/25/2013:svnSupermicro:pnH8DG6/H8DGi:pvr1234567890:rvnSupermicro:rnH8DG6/H8DGi:rvr1.0:cvnSupermicro:ct17:cvr1234567890: + dmi.product.name: H8DG6/H8DGi + dmi.product.version: 1234567890 + dmi.sys.vendor: Supermicro ** Attachment added: "BootDmesg.txt" https://bugs.launchpad.net/bugs/1344430/+attachment/4159604/+files/BootDmesg.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnvi
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
** Tags added: ksm-numa-guest-perf -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
One thing we are looking at is potential issues with KSM and KVM. Can you try disabling KSM and see if the results remain the same: - edit /etc/default/qemu: - change KSM_ENABLED to 0 - sudo service qemu restart I would recommend doing this when your VMs are shutdown; and ensuring nothing else in your system is using KSM. Thanks, -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
Some things (like the hardware of the host) we get through apport-collect. It would be good to get a better understanding of the guest usage. Like how many guests, how much memory they get, how many vcpus. virtio disks and network and how are the guests connected to the network (NAT bridge, transparent bridge, openvswitch?). Are vdisk using raw images or qcov2, image files or partitions or logical volumes. Maybe then we can also replicate the issue in a test environment. Up to now I am not sure I ever was able to see the same issue or not. Oh and was there a specific stat or combination of stats that hints this is happening. Laggy terminal is hard to test for in some scripted way. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
@william, Yes, please boot into the broken kernel and run apport-collect 1344430. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1344430] Re: 10-15% iowait on kvm instances on Ubuntu 14.04
I can boot into the old kernel and run apport-collect if you need that as well. ** Tags added: apport-collected ** Description changed: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux + --- + ApportVersion: 2.14.1-0ubuntu3.2 + Architecture: amd64 + DistroRelease: Ubuntu 14.04 + Package: linux (not installed) + ProcEnviron: + TERM=screen + PATH=(custom, no user) + XDG_RUNTIME_DIR= + LANG=en_US.UTF-8 + SHELL=/bin/bash + Tags: trusty + Uname: Linux 3.14.1-031401-generic x86_64 + UnreportableReason: The running kernel is not an Ubuntu kernel + UpgradeStatus: No upgrade log present (probably fresh install) + UserGroups: root + _MarkForUpload: True ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1344430 Title: 10-15% iowait on kvm instances on Ubuntu 14.04 Status in “linux” package in Ubuntu: Confirmed Bug description: After upgrading from Ubuntu 12.04 to Ubuntu 14.04 we began seeing 10-15% iowait on kvm guests. The guests had little to no load and yet the shell was laggy and commands took forever to respond (assuming you could even type them) # uname -a Linux kvm-andromeda 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:00:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # kvm --version QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.1), Copyright (c) 2003-2008 Fabrice Bellard I posted to Ubuntu Server and they suggested that I upgrade my kernel version. Relevant Mailing List discussion: https://lists.ubuntu.com/archives/ubuntu-server/2014-July/006925.html After installing a 3.14 kernel the performance returned to normal. kvm-andromeda:~# uname -a Linux kvm-andromeda 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux --- ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 DistroRelease: Ubuntu 14.04 Package: linux (not installed) ProcEnviron: TERM=screen PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=en_US.UTF-8 SHELL=/bin/bash Tags: trusty Uname: Linux 3.14.1-031401-generic x86_64 UnreportableReason: The running kernel is not an Ubuntu kernel UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: root _MarkForUpload: True To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1344430/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
