High CPU use on host by idle Windows 7 guest
Hi, I have gotten Windows 7 to install and boot on KVM now, but now I'm having another problem: The qemu process that the Windows guest is running in is constantly using about 27% of one cpu core when idle (more when not, obviously). This is a fresh install of Windows 7 Professional with SP1 done in the same qemu virtual machine that it is currently running in, with virtio-scsi for disk connectivity and virtio-net for network, using the signed guest drivers found here: http://www.linux-kvm.org/page/Windows7Install There is nothing else installed on this Windows installation. (I have seen the same problem with another win7 vm with sata and e1000, though.) After some googling, I found a hint to disable the hpet timer to reduce the problem. This didn't work for me, though, the idle cpu use rose to over 60% instead. Another post suggested disabling acpi and/or apic, but that rendered Windows unable to boot up. This is happening with qemu 1.7 and linux kernel 3.12.6 using a SandyBridge host cpu. Qemu has been started with libvirt, the commandline looks like this: LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin HOME=/ USER=root QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name win7_master2,process=qemu:win7_master2 -S -machine pc- i440fx-1.7,accel=kvm,usb=off -cpu SandyBridge,+pdpe1gb,+osxsave,+dca,+pcid, +pdcm,+xtpr,+tm2,+est,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi, +ds,+vme -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid a10e5a00-b1a4-e94d-c5a9-c3079834b400 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/win7_master2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/virtual_disks/win7_master2.img,if=none,id=drive-virtio- disk0,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio- disk0,bootindex=1 -drive file=/virtual_disks/virtio- win-0.1-74.iso,if=none,id=drive-ide0-0-0,readonly=on,format=raw,cache=none - device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=/virtual_disks/Win7STICK1_c.iso,if=none,id=drive- ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive- ide0-1-0,id=ide0-1-0 -netdev tap,fd=21,id=hostnet0 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:e1:31:22,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -device qxl- vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0- codec0,bus=sound0.0,cad=0 -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x7 Does anybody have an idea what's going wrong here? A linux guest (CentOS 6) on the same host will not use more than 1% cpu when idle. Regards, Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Internal error, emulation failure when trying to boot Win7 install
On Wednesday 08 January 2014 19:40:10 Marcelo Tosatti wrote: On Tue, Jan 07, 2014 at 07:48:41PM +0100, Guido Winkelmann wrote: Hi, When trying to boot a Windows 7 install from a local virtual disks, qemu stops with the messages: KVM internal error. Suberror: 1 emulation failure Can you please enable the following tracepoints via the # cd /sys/kernel/debug/tracing/ # echo kvm_emulate_insn kvm_exit set_event provided that debugfs is mounted at /sys/kernel/debug. Then execute the problematic guest, and make the content of /sys/kernel/debug/trace available somewhere? Unfortunately, today I found my workplace computer has been replaced with newer hardware, and now I cannot reproduce the problem anymore. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Internal error, emulation failure when trying to boot Win7 install
Hi, When trying to boot a Windows 7 install from a local virtual disks, qemu stops with the messages: KVM internal error. Suberror: 1 emulation failure I've started qemu with libvirt. This is the output from libvirt's logfile: 2014-01-07 18:22:10.988+: starting up LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin HOME=/ USER=root QEMU_AUDIO_DRV=spice /usr/bin/qemu-kvm -name win7_master,process=qemu:win7_master -S -machine pc- i440fx-1.7,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 0d640aa9-a88c-164b-398c-14186445d2a8 -no- user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/win7_master.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no- shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device ahci,id=ahci0,bus=pci.0,addr=0x7 -device virtio-serial-pci,id=virtio- serial0,bus=pci.0,addr=0x5 -drive file=/virtual_disks/systemrescuecd- x86-3.8.1.iso,if=none,id=drive-ide0-0-1,readonly=on,format=raw,cache=none - device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive file=/virtual_disks/win7_master.img,if=none,id=drive- sata0-0-0,format=qcow2,cache=none -device ide-hd,bus=ahci0.0,drive=drive- sata0-0-0,id=sata0-0-0,bootindex=1 -netdev tap,fd=22,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:e1:63:d3,bus=pci.0,addr=0x3 - chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 - chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio- serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 - device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable- ticketing,seamless-migration=on -k de -device VGA,id=video0,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0- codec0,bus=sound0.0,cad=0 -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x6 char device redirected to /dev/pts/3 (label charserial0) ((null):14946): SpiceWorker-Warning **: red_worker.c:11477:dev_destroy_primary_surface: double destroy of primary surface ((null):14946): SpiceWorker-Warning **: red_worker.c:9663:red_create_surface: condition `surface-context.canvas' reached main_channel_link: add main channel client main_channel_handle_parsed: net test: latency 0.137000 ms, bitrate 9799043062 bps (9345.095694 Mbps) red_dispatcher_set_cursor_peer: inputs_connect: inputs channel client create KVM internal error. Suberror: 1 emulation failure EAX=0200 EBX=aa55 ECX=0007 EDX=0080 ESI=7bd0 EDI=0800 EBP=07be ESP=7be0 EIP=0684 EFL=3202 [---] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES = 00809300 CS = 00809b00 SS = 00809300 DS = 00809300 FS = 00809300 GS = 00809300 LDT= 8200 TR = 8b00 GDT= 000f69e0 0037 IDT= 03ff CR0=0010 CR2= CR3= CR4= DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER= Code=7c 68 01 00 68 10 00 b4 42 8a 56 00 8b f4 cd 13 9f 83 c4 10 9e eb 14 b8 01 02 bb 00 7c 8a 56 00 8a 76 01 8a 4e 02 8a 6e 03 cd 13 66 61 73 1c fe 4e 11 qemu: terminating on signal 15 from pid 3795 2014-01-07 18:22:15.777+: shutting down This happened with qemu 1.5 and 1.7. The linux kernel in use is 3.10.25-gentoo The host CPU looks like this in lscpu: Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit Byte Order:Little Endian CPU(s):8 On-line CPU(s) list: 0-7 Thread(s) per core:2 Core(s) per socket:4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family:6 Model: 26 Stepping: 5 CPU MHz: 1600.000 BogoMIPS: 5345.33 Virtualization:VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 8192K NUMA node0 CPU(s): 0-7 The Windows install in question is an image that has been pulled from a working Windows 7 install (on a physical machine). Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Internal error, emulation failure when trying to boot Win7 install
Some more information: I have upgraded to Kernel 3.12.6, and the problem is still there. However, when starting the same machine manually with simpler command line like qemu-kvm -cpu Nehalem -machine accel=kvm -m 4G /virtual_disks/win7_master.img qemu will not crash, but Windows (in the guest machine) will still fail to boot up all the way. It bluescreens once, restarts, then manages to start some graphical computer repair wizard which will run for a long time and then display a message saying it could not repair the computer, offering to send the problem report to Microsoft... It didn't show an error message that would tell me what is actually wrong, though. Guido On Tuesday 07 January 2014 19:48:41 Guido Winkelmann wrote: Hi, When trying to boot a Windows 7 install from a local virtual disks, qemu stops with the messages: KVM internal error. Suberror: 1 emulation failure I've started qemu with libvirt. This is the output from libvirt's logfile: 2014-01-07 18:22:10.988+: starting up LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/b in:/usr/local/sbin HOME=/ USER=root QEMU_AUDIO_DRV=spice /usr/bin/qemu-kvm -name win7_master,process=qemu:win7_master -S -machine pc- i440fx-1.7,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 0d640aa9-a88c-164b-398c-14186445d2a8 -no- user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/win7_master.monitor,server,n owait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no- shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device ahci,id=ahci0,bus=pci.0,addr=0x7 -device virtio-serial-pci,id=virtio- serial0,bus=pci.0,addr=0x5 -drive file=/virtual_disks/systemrescuecd- x86-3.8.1.iso,if=none,id=drive-ide0-0-1,readonly=on,format=raw,cache=none - device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive file=/virtual_disks/win7_master.img,if=none,id=drive- sata0-0-0,format=qcow2,cache=none -device ide-hd,bus=ahci0.0,drive=drive- sata0-0-0,id=sata0-0-0,bootindex=1 -netdev tap,fd=22,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:e1:63:d3,bus=pci.0,addr=0x3 - chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 - chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio- serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 - device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable- ticketing,seamless-migration=on -k de -device VGA,id=video0,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0- codec0,bus=sound0.0,cad=0 -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x6 char device redirected to /dev/pts/3 (label charserial0) ((null):14946): SpiceWorker-Warning **: red_worker.c:11477:dev_destroy_primary_surface: double destroy of primary surface ((null):14946): SpiceWorker-Warning **: red_worker.c:9663:red_create_surface: condition `surface-context.canvas' reached main_channel_link: add main channel client main_channel_handle_parsed: net test: latency 0.137000 ms, bitrate 9799043062 bps (9345.095694 Mbps) red_dispatcher_set_cursor_peer: inputs_connect: inputs channel client create KVM internal error. Suberror: 1 emulation failure EAX=0200 EBX=aa55 ECX=0007 EDX=0080 ESI=7bd0 EDI=0800 EBP=07be ESP=7be0 EIP=0684 EFL=3202 [---] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES = 00809300 CS = 00809b00 SS = 00809300 DS = 00809300 FS = 00809300 GS = 00809300 LDT= 8200 TR = 8b00 GDT= 000f69e0 0037 IDT= 03ff CR0=0010 CR2= CR3= CR4= DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER= Code=7c 68 01 00 68 10 00 b4 42 8a 56 00 8b f4 cd 13 9f 83 c4 10 9e eb 14 b8 01 02 bb 00 7c 8a 56 00 8a 76 01 8a 4e 02 8a 6e 03 cd 13 66 61 73 1c fe 4e 11 qemu: terminating on signal 15 from pid 3795 2014-01-07 18:22:15.777+: shutting down This happened with qemu 1.5 and 1.7. The linux kernel in use is 3.10.25-gentoo The host CPU looks like this in lscpu: Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit Byte Order:Little Endian CPU(s):8 On-line CPU(s) list: 0-7 Thread(s) per core:2 Core(s) per socket:4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family:6 Model: 26 Stepping: 5 CPU MHz: 1600.000 BogoMIPS: 5345.33 Virtualization:VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 8192K NUMA node0 CPU(s): 0-7
Re: I/O errors in guest OS after repeated migration
Am Montag, 29. Oktober 2012, 12:29:01 schrieb Stefan Hajnoczi:
On Fri, Oct 19, 2012 at 2:55 PM, Guido Winkelmann
guido-k...@thisisnotatest.de wrote:
Am Donnerstag, 18. Oktober 2012, 18:05:39 schrieb Avi Kivity:
On 10/18/2012 05:50 PM, Guido Winkelmann wrote:
Am Mittwoch, 17. Oktober 2012, 13:25:45 schrieb Brian Jackson:
What about newer versions of qemu/kvm? But of course if those work,
your
next task is going to be git bisect it or file a bug with your distro
that
is using an ancient version of qemu/kvm.
I've just upgraded both hosts to qemu-kvm 1.2.0
(qemu-1.2.0-14.fc17.x86_64,
built from spec files under
http://pkgs.fedoraproject.org/cgit/qemu.git/).
The bug is still there.
If you let the guest go idle (no I/O), then migrate it, then restart the
I/O, do the errors show?
Just tested - yes, they do.
The -EIO error does not really reveal why there is a problem. You can
use SystemTap probes in QEMU to find out more about the nature of the
error.
# stap -e 'probe qemu.kvm.bdrv_*, qemu.kvm.virtio_blk_*,
qemu.kvm.paio_* { printf(%s(%s)\n, probefunc(), $$parms) }' -x
$PID_OF_QEMU
This does not work for me. When I try running this, I'm getting many pages of
errors like this:
==
# stap -e 'probe qemu.kvm.bdrv_*, qemu.kvm.virtio_blk_*, qemu.kvm.paio_* {
printf(%s(%s)\n, probefunc(), $$parms) }' -x 1623
parse error: expected statement
saw: keyword at /usr/share/systemtap/tapset/qemu-alpha.stp:1455:3
source: function = $arg3;
^
parse error: expected identifier
saw: operator '=' at /usr/share/systemtap/tapset/qemu-
alpha.stp:1455:12
source: function = $arg3;
^
2 parse errors.
==
Unfortunately, I don't know the first thing about systemtap, so I don't really
know what's happening here...
Guido
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: I/O errors in guest OS after repeated migration
Am Donnerstag, 18. Oktober 2012, 18:05:39 schrieb Avi Kivity: On 10/18/2012 05:50 PM, Guido Winkelmann wrote: Am Mittwoch, 17. Oktober 2012, 13:25:45 schrieb Brian Jackson: On Wednesday, October 17, 2012 10:45:14 AM Guido Winkelmann wrote: vda1, logical block 1858771 Oct 17 17:12:04 localhost kernel: [ 212.070600] Buffer I/O error on device vda1, logical block 1858772 Oct 17 17:12:04 localhost kernel: [ 212.070602] Buffer I/O error on device vda1, logical block 1858773 Oct 17 17:12:04 localhost kernel: [ 212.070605] Buffer I/O error on device vda1, logical block 1858774 Oct 17 17:12:04 localhost kernel: [ 212.070607] Buffer I/O error on device vda1, logical block 1858775 Oct 17 17:12:04 localhost kernel: [ 212.070610] Buffer I/O error on device vda1, logical block 1858776 Oct 17 17:12:04 localhost kernel: [ 212.070612] Buffer I/O error on device vda1, logical block 1858777 Oct 17 17:12:04 localhost kernel: [ 212.070615] Buffer I/O error on device vda1, logical block 1858778 Oct 17 17:12:04 localhost kernel: [ 212.070617] Buffer I/O error on device vda1, logical block 1858779 (I was writing a large file at the time, to make sure I actually catch I/O errors as they happen) What about newer versions of qemu/kvm? But of course if those work, your next task is going to be git bisect it or file a bug with your distro that is using an ancient version of qemu/kvm. I've just upgraded both hosts to qemu-kvm 1.2.0 (qemu-1.2.0-14.fc17.x86_64, built from spec files under http://pkgs.fedoraproject.org/cgit/qemu.git/). The bug is still there. If you let the guest go idle (no I/O), then migrate it, then restart the I/O, do the errors show? Just tested - yes, they do. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: I/O errors in guest OS after repeated migration
Am Mittwoch, 17. Oktober 2012, 13:25:45 schrieb Brian Jackson: On Wednesday, October 17, 2012 10:45:14 AM Guido Winkelmann wrote: vda1, logical block 1858771 Oct 17 17:12:04 localhost kernel: [ 212.070600] Buffer I/O error on device vda1, logical block 1858772 Oct 17 17:12:04 localhost kernel: [ 212.070602] Buffer I/O error on device vda1, logical block 1858773 Oct 17 17:12:04 localhost kernel: [ 212.070605] Buffer I/O error on device vda1, logical block 1858774 Oct 17 17:12:04 localhost kernel: [ 212.070607] Buffer I/O error on device vda1, logical block 1858775 Oct 17 17:12:04 localhost kernel: [ 212.070610] Buffer I/O error on device vda1, logical block 1858776 Oct 17 17:12:04 localhost kernel: [ 212.070612] Buffer I/O error on device vda1, logical block 1858777 Oct 17 17:12:04 localhost kernel: [ 212.070615] Buffer I/O error on device vda1, logical block 1858778 Oct 17 17:12:04 localhost kernel: [ 212.070617] Buffer I/O error on device vda1, logical block 1858779 (I was writing a large file at the time, to make sure I actually catch I/O errors as they happen) What about newer versions of qemu/kvm? But of course if those work, your next task is going to be git bisect it or file a bug with your distro that is using an ancient version of qemu/kvm. I've just upgraded both hosts to qemu-kvm 1.2.0 (qemu-1.2.0-14.fc17.x86_64, built from spec files under http://pkgs.fedoraproject.org/cgit/qemu.git/). The bug is still there. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: I/O errors in guest OS after repeated migration
Am Dienstag, 16. Oktober 2012, 12:44:27 schrieb Brian Jackson: On Tuesday, October 16, 2012 11:33:44 AM Guido Winkelmann wrote: [...] The commandline, as generated by libvirtd, looks like this: LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -S -M pc-0.15 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name migratetest2 -uuid ddbf11e9-387e-902b-4849-8c3067dc42a2 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/migratetest2.monitor,serv e r,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -no- shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/data/migratetest2_system,if=none,id=drive-virtio- disk0,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio- disk0,bootindex=1 -drive file=/data/migratetest2_data-1,if=none,id=drive- virtio-disk1,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 - netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=02:00:00:00:00:0c,bus=pci.0,addr=0x3 -vnc 127.0.0.1:2,password -k de -vga cirrus -incoming tcp:0.0.0.0:49153 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 I see qcow2 in there. Live migration of qcow2 was a new feature in 1.0. Have you tried other formats or different qemu/kvm versions? Are you sure about that? Because I'm fairly certain I have been using live migration since at least 0.14, if not 0.13, and I have always been using qcow2 as the image format for the disks... I can still try with other image formats, though. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: I/O errors in guest OS after repeated migration
Am Dienstag, 16. Oktober 2012, 12:44:27 schrieb Brian Jackson: On Tuesday, October 16, 2012 11:33:44 AM Guido Winkelmann wrote: The commandline, as generated by libvirtd, looks like this: LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -S -M pc-0.15 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name migratetest2 -uuid ddbf11e9-387e-902b-4849-8c3067dc42a2 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/migratetest2.monitor,serv e r,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -no- shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/data/migratetest2_system,if=none,id=drive-virtio- disk0,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio- disk0,bootindex=1 -drive file=/data/migratetest2_data-1,if=none,id=drive- virtio-disk1,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 - netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=02:00:00:00:00:0c,bus=pci.0,addr=0x3 -vnc 127.0.0.1:2,password -k de -vga cirrus -incoming tcp:0.0.0.0:49153 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 I see qcow2 in there. Live migration of qcow2 was a new feature in 1.0. Have you tried other formats or different qemu/kvm versions? I tried the same thing with a raw image file instead of qcow2, and the problem still happens. From the /var/log/messages of the guest: Oct 17 17:10:34 localhost sshd[2368]: nss_ldap: could not search LDAP server - Server is unavailable Oct 17 17:10:39 localhost kernel: [ 126.800075] eth0: no IPv6 routers present Oct 17 17:10:52 localhost kernel: [ 140.335783] Clocksource tsc unstable (delta = -70265501 ns) Oct 17 17:12:04 localhost /O error on device vda1, logical block 1858765 Oct 17 17:12:04 localhost kernel: [ 212.070584] Buffer I/O error on device vda1, logical block 1858766 Oct 17 17:12:04 localhost kernel: [ 212.070587] Buffer I/O error on device vda1, logical block 1858767 Oct 17 17:12:04 localhost kernel: [ 212.070589] Buffer I/O error on device vda1, logical block 1858768 Oct 17 17:12:04 localhost kernel: [ 212.070592] Buffer I/O error on device vda1, logical block 1858769 Oct 17 17:12:04 localhost kernel: [ 212.070595] Buffer I/O error on device vda1, logical block 1858770 Oct 17 17:12:04 localhost kernel: [ 212.070597] Buffer I/O error on device vda1, logical block 1858771 Oct 17 17:12:04 localhost kernel: [ 212.070600] Buffer I/O error on device vda1, logical block 1858772 Oct 17 17:12:04 localhost kernel: [ 212.070602] Buffer I/O error on device vda1, logical block 1858773 Oct 17 17:12:04 localhost kernel: [ 212.070605] Buffer I/O error on device vda1, logical block 1858774 Oct 17 17:12:04 localhost kernel: [ 212.070607] Buffer I/O error on device vda1, logical block 1858775 Oct 17 17:12:04 localhost kernel: [ 212.070610] Buffer I/O error on device vda1, logical block 1858776 Oct 17 17:12:04 localhost kernel: [ 212.070612] Buffer I/O error on device vda1, logical block 1858777 Oct 17 17:12:04 localhost kernel: [ 212.070615] Buffer I/O error on device vda1, logical block 1858778 Oct 17 17:12:04 localhost kernel: [ 212.070617] Buffer I/O error on device vda1, logical block 1858779 (I was writing a large file at the time, to make sure I actually catch I/O errors as they happen) Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I/O errors in guest OS after repeated migration
Hi, I'm experiencing I/O errors in a guest machine after migrating it from one host to another, and then back to the original host. After doing this, I find the following in the dmesg output of the guest machine: [ 345.390543] end_request: I/O error, dev vda, sector 273871 [ 345.391125] end_request: I/O error, dev vda, sector 273871 [ 345.391705] end_request: I/O error, dev vda, sector 273871 [ 345.394796] end_request: I/O error, dev vda, sector 1745983 [ 345.396005] end_request: I/O error, dev vda, sector 1745983 [ 346.083160] end_request: I/O error, dev vdb, sector 54528008 [ 346.083179] Buffer I/O error on device dm-0, logical block 6815745 [ 346.083181] lost page write due to I/O error on dm-0 [ 346.083193] end_request: I/O error, dev vdb, sector 54528264 [ 346.083195] Buffer I/O error on device dm-0, logical block 6815777 [ 346.083197] lost page write due to I/O error on dm-0 [ 346.083201] end_request: I/O error, dev vdb, sector 2056 [ 346.083204] Buffer I/O error on device dm-0, logical block 1 [ 346.083206] lost page write due to I/O error on dm-0 [ 346.083209] Buffer I/O error on device dm-0, logical block 2 [ 346.083211] lost page write due to I/O error on dm-0 [ 346.083215] end_request: I/O error, dev vdb, sector 10248 [ 346.083217] Buffer I/O error on device dm-0, logical block 1025 [ 346.083219] lost page write due to I/O error on dm-0 [ 346.091499] end_request: I/O error, dev vdb, sector 76240 [ 346.091506] Buffer I/O error on device dm-0, logical block 9274 [ 346.091508] lost page write due to I/O error on dm-0 [ 346.091572] JBD2: Detected IO errors while flushing file data on dm-0-8 [ 346.091915] end_request: I/O error, dev vdb, sector 38017360 [ 346.091956] Aborting journal on device dm-0-8. [ 346.092557] end_request: I/O error, dev vdb, sector 38012928 [ 346.092566] Buffer I/O error on device dm-0, logical block 4751360 [ 346.092569] lost page write due to I/O error on dm-0 [ 346.092624] JBD2: I/O error detected when updating journal superblock for dm-0-8. [ 346.100940] end_request: I/O error, dev vdb, sector 2048 [ 346.100948] Buffer I/O error on device dm-0, logical block 0 [ 346.100952] lost page write due to I/O error on dm-0 [ 346.101027] EXT4-fs error (device dm-0): ext4_journal_start_sb:327: Detected aborted journal [ 346.101038] EXT4-fs (dm-0): Remounting filesystem read-only [ 346.101051] EXT4-fs (dm-0): previous I/O error to superblock detected [ 346.101836] end_request: I/O error, dev vdb, sector 2048 [ 346.101845] Buffer I/O error on device dm-0, logical block 0 [ 346.101849] lost page write due to I/O error on dm-0 [ 373.006680] end_request: I/O error, dev vda, sector 624319 [ 373.007543] end_request: I/O error, dev vda, sector 624319 [ 373.008327] end_request: I/O error, dev vda, sector 624319 [ 374.886674] end_request: I/O error, dev vda, sector 624319 [ 374.887563] end_request: I/O error, dev vda, sector 624319 The hosts are both running Fedora 17 with qemu-kvm-1.0.1-1.fc17.x86_64. The guest machine has been started and migrated using libvirt (0.9.11). Kernel version is 3.5.6-1.fc17.x86_64 on the first host and 3.5.5-2.fc17.x86_64 on the second. The guest machine is on Kernel 3.3.8 and uses ext4 on its disks. The commandline, as generated by libvirtd, looks like this: LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -S -M pc-0.15 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name migratetest2 -uuid ddbf11e9-387e-902b-4849-8c3067dc42a2 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/migratetest2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -no- shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/data/migratetest2_system,if=none,id=drive-virtio- disk0,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio- disk0,bootindex=1 -drive file=/data/migratetest2_data-1,if=none,id=drive- virtio-disk1,format=qcow2,cache=none -device virtio-blk- pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 - netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=02:00:00:00:00:0c,bus=pci.0,addr=0x3 -vnc 127.0.0.1:2,password -k de -vga cirrus -incoming tcp:0.0.0.0:49153 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 The second host has an ext4 filesystem mounted under /data, which it exports using NFSv3 over TCP to the first host, which also mounts it under /data. So far, the problem seems reproducible: When I start another guest machine and do the same thing with it, the same problem happens. Can anybody help me with this problem? Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
Am Mittwoch, 11. April 2012, 20:37:56 schrieb Guido Winkelmann: Am Mittwoch, 11. April 2012, 17:25:12 schrieb Orit Wasserman: On 04/11/2012 04:43 PM, Guido Winkelmann wrote: Am Mittwoch, 11. April 2012, 16:29:55 schrieben Sie: I'm not sure if this is the problem but I noticed that the second layer and the third layer have the same memory size (8G), how about trying to reduce the memory for the third layer ? I tried reducing the third layer to 1G. That didn't change anything. There is a patch for fixing nVMX in 3.4. http://www.mail-archive.com/kvm@vger.kernel.org/msg68951.html you can try it . That worked, though the VM inside the VM is still very slow... Well, it appeared to work yesterday for a short time, but now a lot of userspace programs in the L2 guest will die with a segmentation fault after chrooting from the rescue CD to the installed OS... Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Nested virtualization on Intel does not work - second level freezes when third level is starting
Hi, Nested virtualization on Intel does not work for me with qemu-kvm. As soon as the third layer OS (second virtualised) is starting the Linux kernel, the entire second layer freezes up. The last thing I can see console of the third layer system before it freezes is Decompressing Linux... . (no done, though). When starting without nofb option, the kernel still manages to set the screen resolution before freezing. Grub/Syslinux still work, but are extremely slow. Both the first layer OS (i.e. the one running on bare metal) and the second layer OS are 64-bit-Fedora 16 with Kernel 3.3.1-3.fc16.x86_64. On both the first and second layer OS, the kvm_intel modules are loaded with nested=Y parameter. (I've also tried with nested=N in the second layer. Didn't change anything.) Qemu-kvm was originally the Fedora-shipped 0.14, but I have since upgraded to 1.0. (Using rpmbuild with the specfile and patches from http://pkgs.fedoraproject.org/gitweb/?p=qemu.git;a=blob;f=qemu.spec;hb=HEAD) The second layer machine has this CPU specification in libvirt on the first layer OS: cpu mode='custom' match='exact' model fallback='allow'Nehalem/model feature policy='require' name='vmx'/ /cpu which results in this qemu commandline (from libvirt's logs): LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu- kvm -S -M pc-0.15 -cpu kvm64,+lahf_lm,+popcnt,+sse4.2,+sse4.1,+ssse3,+vmx - enable-kvm -m 8192 -smp 8,sockets=8,cores=1,threads=1 -name vshost1 -uuid 192b8c4b-0ded-07aa-2545-d7fef4cd897f -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vshost1.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown - no-acpi -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/data/vshost1.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio- disk0,bootindex=1 -drive file=/data/Fedora-16-x86_64- netinst.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw - device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=21,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:84:7d:46,bus=pci.0,addr=0x3 -netdev tap,fd=23,id=hostnet1,vhost=on,vhostfd=24 -device virtio-net- pci,netdev=hostnet1,id=net1,mac=52:54:00:84:8d:46,bus=pci.0,addr=0x4 -vnc 127.0.0.1:0,password -k de -vga cirrus -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x6 I have also tried some other combinations for the cpu element, like changing the model to core2duo and/or including all the features reported by libvirt's capabalities command. The third level machine does not have a cpu element in libvirt, and its commandline looks like this: LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu- kvm -S -M pc-0.14 -enable-kvm -m 8192 -smp 4,sockets=4,cores=1,threads=1 -name gentoo -uuid 3cdcc902-4520-df25-92ac-31ca5c707a50 -nodefconfig -nodefaults - chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/gentoo.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-acpi -drive file=/data/gentoo.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 - drive file=/data/install-amd64- minimal-20120223.iso,if=none,media=cdrom,id=drive- ide0-1-0,readonly=on,format=raw -device ide- drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:84:6d:46,bus=pci.0,addr=0x3 -usb -vnc 127.0.0.1:0,password -k de -vga cirrus -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x5 The third layer OS is a recent Gentoo minimal install (amd64), but somehow I don't think that matters at this point... The metal is a Dell PowerEdge R710 server with two Xeon E5520 CPUs. I've tried updating the machine's BIOS and other firmware to the latest version. That took a lot of time and a lot of searching on Dell websites, but didn't change anything. Does anyone have any idea what might be going wrong here or how I could debug this further? Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
Am Mittwoch, 11. April 2012, 16:29:55 schrieben Sie: I'm not sure if this is the problem but I noticed that the second layer and the third layer have the same memory size (8G), how about trying to reduce the memory for the third layer ? I tried reducing the third layer to 1G. That didn't change anything. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
Am Mittwoch, 11. April 2012, 17:38:14 schrieb Nadav Har'El: On Wed, Apr 11, 2012, Guido Winkelmann wrote about Nested virtualization on Intel does not work - second level freezes when third level is starting: Nested virtualization on Intel does not work for me with qemu-kvm. As soon as the third layer OS (second virtualised) is starting the Linux kernel, the entire second layer freezes up. The last thing I can see console of the third Hi, From your description, I understand that ordinary (2-level) nested virtualization working for you (host, guest and 2nd-level guest), and it's the third nesting level (guest's guest's guest) which is broken? No, even 2-level nesting is broken. I can run Host-Guest, but not Host-Guest-2nd Level Guest. I haven't even tried with a third virtualized level. I suppose the misunderstanding happened because, in my original mail, I was counting the host as one level. This is the second report of this nature in a week (see the previous report in https://bugzilla.kernel.org/show_bug.cgi?id=43068 - the details there are different), so I guess I'll need to find the time to give this issue some attention. L3 did work for me when the nested VMX patches were included in KVM, so either something broke since, or (perhaps more likely) your slightly different setups have features that my setup didn't. But in any case, like I explain in the aforementioned URL, even if L3 would work, in the current implementation it would be extremenly slow - perhaps to the point of being unusable (I think you saw this with grub performance in L3). So I wonder if you'd really want to use it, even if it worked... Just curious, what were you thinking of doing with L3? I was trying to test network setups that involve migrating VMs between hosts a lot, and I was hoping to be able to use only one physical server for that. As I said, I really only need one level of nesting for that (i.e. two levels of virtualization, three levels of OSes when counting the host). Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
Am Mittwoch, 11. April 2012, 17:25:12 schrieben Sie: On 04/11/2012 04:43 PM, Guido Winkelmann wrote: Am Mittwoch, 11. April 2012, 16:29:55 schrieben Sie: I'm not sure if this is the problem but I noticed that the second layer and the third layer have the same memory size (8G), how about trying to reduce the memory for the third layer ? I tried reducing the third layer to 1G. That didn't change anything. There is a patch for fixing nVMX in 3.4. http://www.mail-archive.com/kvm@vger.kernel.org/msg68951.html you can try it . Should I use that patch on the host, or on the first virtualized layer, or on both? (Compiling 3.4-rc2 for both for now... ) Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
Am Mittwoch, 11. April 2012, 23:14:07 schrieb Kashyap Chamarthy: On Wed, Apr 11, 2012 at 6:14 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: [...] Here is my complete notes on nested virtualization w/ Intel -- http://kashyapc.wordpress.com/2012/01/14/nested-virtualization-with-kvm-inte l/ I had already found you blog post via Google :). In fact, I used some of the things you wrote for setting up my system, in the hopes that I would have more luck. BTW, the line for modprobe.d/dist.conf should read options kvm_intel nested=1 In your blog post, the options keyword is missing. My result: I was able to ssh into the nested guest (guest installed inside the regular guest), but, after a reboot, the nested-guest loses the IP rendering it inaccessible.(Info: the regular-guest has a bridged IP, and nested-guest has a NATed IP) Refer the comments in the above post for some more discussion. Though I haven't tried the suggestion of 'updating your system firmware and disabling VT for Direct I/O Access if you are able in the firmware' . And I wonder how does turning it off can alleviate the prob. Yeah, I've seen that comment, that was what prompted me to update the server's firmware. The Dell BIOS does not offer the option to disable VT for Direct I/O Access, though. And my AMD notes is here(which was completely successful) -- http://kashyapc.wordpress.com/2012/01/18/nested-virtualization-with-kvm-and- amd/ Unfortunately, AMD is not an option for me, at least not in this particular context. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
Am Mittwoch, 11. April 2012, 17:25:12 schrieb Orit Wasserman: On 04/11/2012 04:43 PM, Guido Winkelmann wrote: Am Mittwoch, 11. April 2012, 16:29:55 schrieben Sie: I'm not sure if this is the problem but I noticed that the second layer and the third layer have the same memory size (8G), how about trying to reduce the memory for the third layer ? I tried reducing the third layer to 1G. That didn't change anything. There is a patch for fixing nVMX in 3.4. http://www.mail-archive.com/kvm@vger.kernel.org/msg68951.html you can try it . That worked, though the VM inside the VM is still very slow... Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
SPEC-file for making RPMs (with rpmbuild)
Hi, Is there a spec-file somewhere for creating RPMs from the newest qemu-kvm release? Regards, Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM, iSCSI and High Availability
Am Monday 28 March 2011 schrieb David Martin: - Original Message - On 3/28/11 2:46 PM, Avi Kivity wrote: On 03/25/2011 10:26 PM, Marcin M. Jessa wrote: [...] One LUN per image allows you to implement failover, LVM doesn't (but cluster-LVM does). I recommend using one LUN per image; it's much simpler. Some people say Use one LUN, it's easier and use CLVM. Why is it easier to use CLVM and one LUN per virtual guest? I find it easier because i can do: lvcreate -n vm1 --size 40G iscsi_vg then virt-install or whatever If I were using 1 lun per vm then I would have to provision the lun, make ALL hosts aware of the lun, and finally screw with the multipath configs etc. Don't you have basically the same problem when using LVM in one LUN? You still have to make all the hosts aware of the new LV manually. I don't even know LVM even supports this, it wasn't exactly designed for a situation where multiple hosts might simultaneously read and write to a volume group, let alone create and destroy logical volumes while the VG is in use by any number of other hosts... Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM, iSCSI and High Availability
Am Thursday 31 March 2011 schrieben Sie: That's what CLVM is for, it propagates the volume changes to every member of the 'cluster'. Oh, right. I didn't know about clvm until now. It sounds very promising though, certainly better than working with the proprietary API of whoever your SAN-vendor is to create a new LUN for every VM. Also, the machine we have got here, a Dell PowerVault, appears to be limited to at most 255 LUNs. I don't if that's a limitation of iSCSI or just a problem of this particular array. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Virtual SCSI disks hangs on heavy IO
Am Wednesday 16 March 2011 schrieb Stefan Hajnoczi: On Tue, Mar 15, 2011 at 1:20 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: Am Tuesday 15 March 2011 schrieben Sie: On Mon, Mar 14, 2011 at 10:57 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: On Monday 14 March 2011 20:32:23 Stefan Hajnoczi wrote: On Mon, Mar 14, 2011 at 6:05 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: Does anybody have an idea what might cause this or what might be done about it? The lsi_scsi emulation code is incomplete. It does not handle some situations like the ORDERED commands or message 0x0c. There is a patch to address the message 0xc issue, it has not been applied to qemu.git or qemu-kvm.git yet: http://patchwork.ozlabs.org/patch/63926/ Basically there is no one actively maintaining or reviewing patches for the lsi53c895a SCSI controller. Does that mean that using the SCSI transport for virtual disks is officially unsupported or deprecated or that it should be? The LSI SCSI emulation in particular has not seen much attention. As for the wider SCSI emulation there has been work over the past few months so it's alive and being used. Well, I cannot find any other HBAs than LSI when I run qemu -device ? - or at least nothing I would recognize as a SCSI HBA. As far as I can see, that pretty much means I cannot use SCSI disks in KVM at all, unless I'm prepared to live with the problems described earlier... The LSI controller is the only available PCI SCSI HBA. Are you able to try the patch I linked? http://patchwork.ozlabs.org/patch/63926/ I haven't tried the patch yet. At work, it was decided that we are not going to use a manually patch version of qemu-kvm unless absolutely necessary, and at home, I'm unlikely to ever want to virtualize an OS without virtio-drivers. I can still try the patch on my home machine, if you want me to. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Virtual SCSI disks hangs on heavy IO
Am Tuesday 15 March 2011 schrieben Sie: On Mon, Mar 14, 2011 at 10:57 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: On Monday 14 March 2011 20:32:23 Stefan Hajnoczi wrote: On Mon, Mar 14, 2011 at 6:05 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: Does anybody have an idea what might cause this or what might be done about it? The lsi_scsi emulation code is incomplete. It does not handle some situations like the ORDERED commands or message 0x0c. There is a patch to address the message 0xc issue, it has not been applied to qemu.git or qemu-kvm.git yet: http://patchwork.ozlabs.org/patch/63926/ Basically there is no one actively maintaining or reviewing patches for the lsi53c895a SCSI controller. Does that mean that using the SCSI transport for virtual disks is officially unsupported or deprecated or that it should be? The LSI SCSI emulation in particular has not seen much attention. As for the wider SCSI emulation there has been work over the past few months so it's alive and being used. Well, I cannot find any other HBAs than LSI when I run qemu -device ? - or at least nothing I would recognize as a SCSI HBA. As far as I can see, that pretty much means I cannot use SCSI disks in KVM at all, unless I'm prepared to live with the problems described earlier... Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Virtual SCSI disks hangs on heavy IO
Hi, I am experiencing hangs in disk IO in systems hosted inside a virtual KVM machine. When the virtual system disk is SCSI and when I am doing a lot of I/O on it, I will eventually get error messages on the console and in dmesg like these: sd 0:0:0:0: [sda] ABORT operation started sd 0:0:0:0: ABORT operation failed. sd 0:0:0:0: [sda] DEVICE RESET operation started sd 0:0:0:0: DEVICE RESET operation complete. scsi target0:0:0: control msgout: c. scsi target0:0:0: has been reset sd 0:0:0:0: [sda] BUS RESET operation started sym0: SCSI BUS reset detected. sd 0:0:0:0: BUS RESET operation complete. sym0: SCSI BUS has been reset. sym0: unknown interrupt(s) ignored, ISTAT=0x1 DSTAT=0x80 SIST=0x0 sd 0:0:0:0: [sda] ABORT operation started sd 0:0:0:0: ABORT operation failed. sd 0:0:0:0: [sda] DEVICE RESET operation started sd 0:0:0:0: DEVICE RESET operation complete. scsi target0:0:0: control msgout: c. scsi target0:0:0: has been reset sd 0:0:0:0: [sda] BUS RESET operation started sd 0:0:0:0: BUS RESET operation complete. (this goes on for a while) During this time, all IO operations on this disk will hang intermittently for a fairly long time (minutes), but will complete eventually. After the heavy IO is over, things will go back to normal. Here's an excerpt of what I found in libvirt's logfile when the problem struck: lsi_scsi: error: ORDERED queue not implemented lsi_scsi: error: ORDERED queue not implemented lsi_scsi: error: ORDERED queue not implemented lsi_scsi: error: Unimplemented message 0x0c lsi_scsi: error: ORDERED queue not implemented lsi_scsi: error: ORDERED queue not implemented lsi_scsi: error: ORDERED queue not implemented (goes on for a while) I have witnessed this problem on two rather different host machines, one beefy server with two Intel Xeon processors and a hardware RAID running Fedora 14, and on my homeserver, an AMD machine with just a single HDD running Gentoo. On the first machine, this happened when I did cat /dev/zero /bigfile ; rm /bigfile (to fill unused parts of the filesystem with zeroes, so the HD image would compress better), on the second, it happened on unpacking the portage tree in a virtualized Gentoo guest. Qemu-kvm is 0.13.0 in both cases. The kernel version is 2.6.35.10-74.fc14.x86_64 for the Fedora machine and 2.6.36-gentoo-r5 for the Gentoo machine. In both cases, the guest was started with libvirt. The actual commandline on the Fedora machine was (from libvirt's logfile): LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 1024 -smp 4,sockets=4,cores=1,threads=1 -name basicgentooimage -uuid 348fd057-e318- da7b-9b3f-52d9ccf10b93 -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/basicgentooimage.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -no-acpi -boot d -device lsi,id=scsi0,bus=pci.0,addr=0x4 -drive file=/data/basicgentooimage.img,if=none,id=drive-scsi0-0-0,format=qcow2 -device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 -drive file=/data/install-amd64-minimal-20101021.iso,if=none,media=cdrom,id=drive- ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive- ide0-1-0,id=ide0-1-0 -device e1000,vlan=0,id=net0,mac=52:54:00:84:6d:49,bus=pci.0,addr=0x3 -net tap,fd=91,vlan=0,name=hostnet0 -usb -vnc 127.0.0.1:2 -k de -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 On the Gentoo machine it was: LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/sbin:/usr/local/bin\ :/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/i686-pc-linux- gnu/gcc-bin/4.4.5:/usr/x86_64-pc-linux-gnu/gcc-bin/4.4.5 HOME=/root USER=root QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 1024 -smp 2,sockets=2,cores=1,threads=1 -name testnet -uuid c92c7c02-4a8b-777a- ee3e-7d98b3108c02 -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/testnet.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=utc -boot d -device lsi,id=scsi0,bus=pci.0,addr=0x4 -drive file=/kvmdata/testnet- system,if=none,id=drive-scsi0-0-0,format=qcow2 -device scsi- disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 -drive file=/kvmdata/install-amd64-minimal-20110303.iso,if=none,media=cdrom,id=drive- ide0-0-0,readonly=on,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive- ide0-0-0,id=ide0-0-0 -netdev tap,fd=14,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=24:42:53:21:52:45,bus=pci.0,addr=0x3 -usb -vnc 127.0.0.1:0 -k de -vga cirrus -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x5 Does anybody have an idea what might cause this or what might be done about it? Regards, Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at
Re: Virtual SCSI disks hangs on heavy IO
On Monday 14 March 2011 20:32:23 Stefan Hajnoczi wrote: On Mon, Mar 14, 2011 at 6:05 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: Does anybody have an idea what might cause this or what might be done about it? The lsi_scsi emulation code is incomplete. It does not handle some situations like the ORDERED commands or message 0x0c. There is a patch to address the message 0xc issue, it has not been applied to qemu.git or qemu-kvm.git yet: http://patchwork.ozlabs.org/patch/63926/ Basically there is no one actively maintaining or reviewing patches for the lsi53c895a SCSI controller. Does that mean that using the SCSI transport for virtual disks is officially unsupported or deprecated or that it should be? Are things better with the IDE driver? virtio-blk works very will with Linux guests. Is there a reason you need to use SCSI emulation instead of virtio-blk? I can probably use virtio-blk most of the time, I was just hoping to be able to virtualize a wider array of operating systems, like the *BSDs, (Open)Solaris, Windows, or even just some linux distributions whose installers don't anticipate KVM and thus don't support virtio-anything. Regards, Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Cannot start new VMs when the host system is under load
Hi, I've got a problem where I cannot start any new VMs with KVM if the host machine is under high CPU load. The problem is not 100% reproducible (it works sometimes), but under load conditions, it happens most of the time - roughly 95%. I'm usually using libvirt to start and stop KVM VMs. When using virsh to start a new VM under those conditions, the output looks like this: virsh # start testserver-a error: Failed to start domain testserver-a error: monitor socket did not show up.: Connection refused (There is a very long wait after the command has been sent until the error message shows up.) This is (an example of) the command line that libvirtd uses to start up qemu: - snip - LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -S -M pc-0.12 -enable-kvm -m 256 -smp 1,sockets=1,cores=1,threads=1 -name testserver-a -uuid 7cbb3665-4d58-86b8- ce8f-20541995a99c -nodefaults -chardev socket,id=monitor,path=/usr/local/var/lib/libvirt/qemu/testserver- a.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -no- acpi -boot c -device lsi,id=scsi0,bus=pci.0,addr=0x7 -drive file=/data/testserver-a-system.img,if=none,id=drive-scsi0-0-1,boot=on -device scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 -drive file=/data/testserver-a-data1.img,if=none,id=drive-virtio-disk1 -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id=virtio-disk1 - drive file=/data/testserver-a-data2.img,if=none,id=drive-virtio-disk2 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk2,id=virtio-disk2 - drive file=/data/gentoo-install-amd64- minimal-20100408.iso,if=none,media=cdrom,id=drive-ide0-0-0,readonly=on -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=/data/testserver-a_configfloppy.img,if=none,id=drive-fdc0-0-0 -global isa-fdc.driveA=drive-fdc0-0-0 -device e1000,vlan=0,id=net0,mac=52:54:00:84:6d:69,bus=pci.0,addr=0x6 -net tap,fd=24,vlan=0,name=hostnet0 -usb -vnc 127.0.0.1:1,password -k de -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 - snip - Copy-pasting this to a commandline on the host to start qemu manually leads to a non-functional qemu process that just sits there with nothing happening. The monitor socket /usr/local/var/lib/libvirt/qemu/testserver-a.monitor will, indeed, not show up. I've tried starting qemu with the same commandline but without the parameters for redirecting the monitor to a socket, without the fd parameter for the network interface and without the vnc parameter. This resulted in a black window with the title QEMU (testserver-a) [Stopped]. I could not access the monitor console in graphical mode either. Some experimentation I've done suggests that this problem only happens if the high cpu load is caused by another KVM process, not if it is caused by something else running on the machine. The host machine I'm running this on has got 16 cores in total. It looks like it is sufficient for this bug to surface if at least one of these cores is brought to near 100% use by a KVM process. The version of qemu I'm using is qemu-kvm 0.12.4, built from source. Libvirt is version 0.8.1, built from source as well. The host OS is Fedora 12. The Kernel version is 2.6.32.12-115.fc12.x86_64. Does anybody have any idea what might be causing this and what might be done against it? Regards, Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot start new VMs when the host system is under load
I just noticed that if I leave off the -nodefaults from the qemu command line, the guest will reliably start up again. I don't know what reasons the libvirt developers had for putting it there, though... -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Virtual disks with SCSI or virtio hang after some time
Am Dienstag, 1. Juni 2010 schrieben Sie: On 06/01/2010 06:59 PM, Guido Winkelmann wrote: The host OS is Fedora Core 12, with qemu-kvm 0.11.0 Please try with at least qemu-kvm-0.11.1, preferably qemu-kvm-0.12.4. Also use Linux 2.6.32.latest in the guest. Okay, I've upgraded the userspace tools to qemu-kvm-0.12.4, and so far, the problem is not resurfacing. Before, it could usually be triggered by doing hdparm -t /dev/vda (or sda, respectively) once or twice. Unfortunately, now I cannot start new VMs with libvirt anymore, and I have no idea why... :( Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Virtual disks with SCSI or virtio hang after some time
Hi, When using KVM machines with virtual disks that are hooked up to the guest via either virtio or SCSI, the virtual disk will often hang completely after a short time of operation. When this happens, the network connectivity of the machine will usually go down, too. (I.e. it stops responding to pings.) When this happens when using SCSI, the following messages will appear in syslog (in the guest): sd 0:0:0:0: [sda] ABORT operation started sd 0:0:0:0: ABORT operation timed out sd 0:0:0:0: [sda] DEVICE RESET operation started When using virtio, no log messages appear at the time the hang occurs, but some time before, these messages appear: JBD: barrier-based sync failed on vda1-8 JBD: barrier-based sync failed on vda3-8 JBD: barrier-based sync failed on vda5-8 The hang seems to happen faster with virtio, usually within one minute after bootup, while with SCSI, it can take a few more minutes In the host, no interesting messages appear in syslog. The host OS is Fedora Core 12, with qemu-kvm 0.11.0, Kernel 2.6.32.11-99.fc12.x86_64 and libvirt 0.8.1. The guest kernel is 2.6.32-gentoo-r7. Other, non-linux, guest operating systems seem to have problems with the virtualised scsi disks, too. (I've tried OpenBSD.) Does anyone have an idea what is happening here and what can be done about it? Regards, Guido -- Too much multitasking isn't good for you. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
