invalid opcode: 0000 in guest after migration
Hi, on one of my clusters running qemu-kvm-1.6.1 on 3.10.22 x86_64 I noticed that after some time, migration of guest to other (same) node causes guest to go nuts... I get invalid opcode: [#1] PREEMPT SMP error, with long backtrace, ending with Fixing recursive fault but reboot is needed! since log is quite long, I haven't attached it but it's available here: http://nik.lbox.cz/download/kernel.log Is this some known issue? Any chance upgrading to 1.7.1 will fix it? The system is centos6 with latest updates, qemu and kernel packages are self compiled. CPU is Intel(R) Core(TM)2 CPU 6420 @ 2.13GHz If I should provide more debug info, please let me know. thanks a lot in advance! nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpxfeVjaHoDM.pgp Description: PGP signature
Re: windows 2008 falling asleep under KVM
Hi, just to let You know, with realtek instead of virtio-net, it seems to be reachable for almost three days.. I'd give it another day, then we can be pretty sure it's virtio related... BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpRgZflQSLxE.pgp Description: PGP signature
Re: windows 2008 falling asleep under KVM
Hello Gleb, please note, it's windows 2008, not windows8, not sure whether there's some significant difference, though.. Disable device powersave: - Control Panel - Network Internet - Network Connections - Right-click on desired interface, and select Properties - Click the Configure button on the interface properties - Under the Advanced tab, look for power-saving related options and set to Disabled - Under the Power Management tab, uncheck Allow computer to turn off this device to save power - Save Reboot I forgot to mention that, I'm using virtio network driver, and it doesn't seem to support any power managemt (which is not surprising I guess) In addition: - Control Panel - Hardware Sound - Power Options there's no sound card there.. - Select High performance done already. - By the selected power profile, select Change Plan Settings - In the Edit Plan Settings, select Change advanced power settings - See if there is something relevant to network connection there and if yes change it to not sleep everything was already disabled... - Save Reboot maybe I should try using realtek network device instead of virtio? BR nik -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpB6B0cnHdfP.pgp Description: PGP signature
Re: windows 2008 falling asleep under KVM
Hi, It is sound and _hardware_ not sure why MS separated sound from HW :) OK, I'll check it Try it and see if it changes anything. When you say it becomes unreachable over network what does it mean? Network share disappears or you could ping it before but cannot any more? OK, I'll report the results. unreachable means I'm not even able to ping it unless I open VNC connection which causes waking the machine.. we'll see whether changing network device changes something.. so far thanks for Your time! BR nik -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpMWPQDahgoJ.pgp Description: PGP signature
Re: windows 2008 falling asleep under KVM
This is cirrus, not qxl. This makes my S3 theory to be highly unlikely. OK, finally I got it hung again: virsh # qemu-monitor-command vmtop09 --hmp info status VM status: running nik -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpj5ib351V9X.pgp Description: PGP signature
windows 2008 falling asleep under KVM
Hello, I'm trying to solve strange issue with windows2008 VM under KVM. It seems to be somehow falling asleep after some time, becoming unreachable over network. However connecting to qemu VNC immediately makes windows reachable again... I wonder whether this is the same issue? http://lists.nongnu.org/archive/html/qemu-devel/2013-03/msg02278.html I've tried disabling HPET, same result. I'm using qemu-kvm 1.4.0, seabios 1.7.2.2, kernel 3.0.82 with kvm-kmod-3.8, x86_64 centos6. Any idea what should I check? Thanks a lot in advance with best regards nikola ciprich -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp2P5B0muzUk.pgp Description: PGP signature
Re: windows 2008 falling asleep under KVM
Hello Gleb, well, I'm not windows expert at all, but it shouldn't be. same instalation on different host works without problem... what could be worth noting is that CPU usage of qemu process drops to zero when this happens, otherwise windows guests eat quite a lot of CPU time even when idle. Maybe I should somehow check what the process is doing? (how?) BR nik Is Window configured to suspend itself after a timeout? -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpA6dhoA34Lt.pgp Description: PGP signature
Re: windows 2008 falling asleep under KVM
AFAIK this is default Windows2008 behaviour in case S3 is supported by the platform. But KVM supports S3 for Windows2008 guests only with QXL display, so this raises the question: what is qemu command line on the machine that it happens to and on the host where it does not happen. machine with the problem: /usr/bin/qemu-system-x86_64 -machine accel=kvm -name vmtop09 -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 4096 -smp 1,sockets=1,cores=1,threads=1 -uuid 1a858074-e30d-11e2-b1d7-002590968776 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vmtop09.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/dev/vgshared/vmtop09-1,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=18,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:18:09:00,bus=pci.0,addr=0x3 -device usb-tablet,id=input0 -vnc 0.0.0.0:24109 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 macine without the problem: (please note that there's currently newer qemu there, but it was OK even with 1.4.0) /usr/bin/qemu-system-x86_64 -machine accel=kvm -name vmwts02 -S -machine pc-i440fx-1.5,accel=kvm,usb=off -m 4096 -smp 1,sockets=1,cores=1,threads=1 -uuid 1e501300-dc48-11df-a690-00304834195b -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vmwts02.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/dev/vgshared/vmwts02-1,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=27,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:16:3e:61:01:00,bus=pci.0,addr=0x3 -device usb-tablet,id=input0 -vnc 0.0.0.0:30802 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 That does look like S3 suspend. Can you run info status command on qemu monitor when it happens? I'll check, however I don't have it in this state atm, so I'll report in few hours.. BR nik -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpQqFkaaLR01.pgp Description: PGP signature
Re: vmwrite error: reg 401e value a0332340 (err 12) (SOLVED)
Hi, I'll reply to myself - the kernel is 3.0.60, but kvm module was 3.6 needing 29282fde80d44e587f8c152b10049a56e61659f0. problem solved. nik On Fri, Apr 19, 2013 at 01:33:45PM +0200, Nikola Ciprich wrote: Hi, on one of my boxed I'm getting (maybe) KVM related crashes.. I noticed this in dmesg: [ 50.199372] vmwrite error: reg 401e value a0332340 (err 12) [ 50.199575] Pid: 5179, comm: qemu-system-x86 Not tainted 3.0.60lb6.01 #1 [ 50.199802] Call Trace: [ 50.199950] [a031925c] vmwrite_error+0x2c/0x30 [kvm_intel] [ 50.200182] [a0319280] vmcs_writel+0x20/0x30 [kvm_intel] [ 50.200407] [a031cb79] vmx_cpuid_update+0x79/0x160 [kvm_intel] [ 50.200656] [a02c9c9b] kvm_vcpu_ioctl_set_cpuid2+0xab/0xc0 [kvm] [ 50.200899] [a02a6b9a] kvm_arch_vcpu_ioctl+0x94a/0x1030 [kvm] [ 50.201135] [a031b97e] ? vmx_vcpu_load+0x3e/0x1c0 [kvm_intel] [ 50.201375] [a02a72d7] ? kvm_arch_vcpu_load+0x57/0x1b0 [kvm] [ 50.201606] [a0298d65] kvm_vcpu_ioctl+0x135/0x6b0 [kvm] [ 50.201824] [8102f410] ? do_page_fault+0x1d0/0x480 [ 50.202036] [81151cba] do_vfs_ioctl+0x9a/0x5e0 [ 50.202234] [811522a1] sys_ioctl+0xa1/0xb0 [ 50.202426] [813c0d82] system_call_fastpath+0x16/0x1b the host is x86_64 RHEL6 based, kernel 3.0.60, qemu-kvm-1.4.0 running only one guest (windows XP) any idea what could this be? thanks a lot in advance nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpY1l83xOY4w.pgp Description: PGP signature
vmwrite error: reg 401e value a0332340 (err 12)
Hi, on one of my boxed I'm getting (maybe) KVM related crashes.. I noticed this in dmesg: [ 50.199372] vmwrite error: reg 401e value a0332340 (err 12) [ 50.199575] Pid: 5179, comm: qemu-system-x86 Not tainted 3.0.60lb6.01 #1 [ 50.199802] Call Trace: [ 50.199950] [a031925c] vmwrite_error+0x2c/0x30 [kvm_intel] [ 50.200182] [a0319280] vmcs_writel+0x20/0x30 [kvm_intel] [ 50.200407] [a031cb79] vmx_cpuid_update+0x79/0x160 [kvm_intel] [ 50.200656] [a02c9c9b] kvm_vcpu_ioctl_set_cpuid2+0xab/0xc0 [kvm] [ 50.200899] [a02a6b9a] kvm_arch_vcpu_ioctl+0x94a/0x1030 [kvm] [ 50.201135] [a031b97e] ? vmx_vcpu_load+0x3e/0x1c0 [kvm_intel] [ 50.201375] [a02a72d7] ? kvm_arch_vcpu_load+0x57/0x1b0 [kvm] [ 50.201606] [a0298d65] kvm_vcpu_ioctl+0x135/0x6b0 [kvm] [ 50.201824] [8102f410] ? do_page_fault+0x1d0/0x480 [ 50.202036] [81151cba] do_vfs_ioctl+0x9a/0x5e0 [ 50.202234] [811522a1] sys_ioctl+0xa1/0xb0 [ 50.202426] [813c0d82] system_call_fastpath+0x16/0x1b the host is x86_64 RHEL6 based, kernel 3.0.60, qemu-kvm-1.4.0 running only one guest (windows XP) any idea what could this be? thanks a lot in advance nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpznrhrhQ3qd.pgp Description: PGP signature
continuous packet drops in KVM guests
Hello gentlemen, I'd like to ask somebody more experienced then me for help. I've got machine with 4 running KVM guests. The host is moderately loaded (load_avg ~4-5), same applies for the guests. On all of those guests, I see remarkable amount of dropped packets for no obvious reason. Packets are being dropped even during nonworking hours, when load drops to almost 0. Host is 8core box with 48GB RAM running x86_64 RHEL6 based, kernel 3.0.60, qemu-kvm-1.4.0. guests are SMP (2-4CPUS) RHEL6 based x86_64, kernel 3.0.51. I'm using bridge + virtio for guest networking. I tried to increase receive buffers etc, to no avail. Still few packets are dropped every second. Sometimes it's visible even on ssh session.. What could be the culprit of this? Didn't anyone have similar problem? thanks a lof for any hints.. BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpByOuhOC1vk.pgp Description: PGP signature
how to get discard working on KVM block devices?
Hi, I'm playing with dm-thin target, and I'd like to get discard working from my test KVM virtuals.. I've got thinprov pool, on top of which I've got test logical volume. On host, I can issue discard (using fstrim) on it, and thin pool usage lowers correctly, therefore I think it's working well... However, when I export such block device to KVM (using either virtio or scsi), discard seems not to be working - I can neither issue fstrim, nor mount ext4 with discard option (well, I can mount it, but it reports discard is not supported by underlying device, fstrim reports FITRIM ioctl failed: Operation not supported). I tried with kernel 3.8.2 on both host and guest, and qemu-1.4.0. Is there anything special needed to do to get discard working? thanks a lot in advance for any reply BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpEYKM02C5Ed.pgp Description: PGP signature
high host load from vmx_set_supported_cpuid call?
: kvm_mmio: mmio unsatisfied-read len 4 gpa 0xfed000f0 val 0x0 qemu-kvm-7767 [001] 235066.551684: kvm_userspace_exit: reason KVM_EXIT_MMIO (6) (I can store trace-cmd file somewhere if it can help). This host is x86_64 centos6, running kernel 3.0.53 with kvm-kmod-3.6 and qemu-kvm-1.2.0 If I could provide more information, I'll be happy to do so.. BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpJJdBoujqjn.pgp Description: PGP signature
Re: high host load from vmx_set_supported_cpuid call?
Hi Gleb, Something wrong with your symbols. This function cannot take that much. It is three and a half instruction long and should be called only once during vm startup. well, it didn't make any sense to me, glad I wasn't that wrong :) how could that be? I guess it could be perf/kernel mismatch right? I'll try to fix that and see if it helps.. thanks for Your time! n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpMhzIV2En3z.pgp Description: PGP signature
Re: high host load from vmx_set_supported_cpuid call?
More like loaded modules/installed modules mismatch. I see, the problem is, that I've got kvm-kmod compiled separately! thus kvm*.ko symboles don't match! I see that kvm-kmod build produces System.map file, I guess I need to merge it with kernel's System.map? -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpWo5au4KaYi.pgp Description: PGP signature
Re: high host load from vmx_set_supported_cpuid call?
I think you need to copy them over old modules in /lib/modules. hmm, that should be OK, new kvm*.ko modules are part of kernel rpm package, there's no old module there. (I checked by both inspecting kernel pkg and using modinfo)... Could it be something else? -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpnVxHTWTEmQ.pgp Description: PGP signature
Re: high host load from vmx_set_supported_cpuid call?
hmm, that should be OK, new kvm*.ko modules are part of kernel rpm package, there's no old module there. (I checked by both inspecting kernel pkg and using modinfo)... Could it be something else? hmm, I can reply to myself this time - perf seems to get the symbols using /proc/kallsyms and there apparently are symbols from inkernel KVM modules, not the externally built. I guess the problem with my kernel package is I build both kernel and external KVM modules and then replace kernel ones causing kallsyms mismatch.. I'll have to fix this first, reboot host and then see... -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpTi86wLYT8l.pgp Description: PGP signature
Re: qemu-kvm-1.2.0: double free or corruption in VNC code
Hello Stefan, thanks for Your time looking at it. Thanks, I looked at the backtrace in the source tree. Unfortunately the root cause is not obvious to me. I was looking for a double-free of the zrle buffers. If this bug repeatedly bites you, try a different VNC encoding as a workaround (not ZRLE). Well, when I reported the problem, it was first time it appeared, so I didn't consider it big deal, but yesterday we got it again, on different server (but the backtrace is completely the same) But it was the same person as before (and quite a new user to our KVM guests), so I suspect it can be something specific to his client or setup. I'll try to look at it deeper and let You know if I figure something out... cheers nik Perhaps someone more familiar with the VNC code will be able to see it. All the information you have provided is helpful. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpSgK59dAwch.pgp Description: PGP signature
Re: qemu-kvm-1.2.0: double free or corruption in VNC code
Please also post the exact package version you are using - the line numbers change between releases and depend on which patches have been applied to the source tree. The distro exact package version allows me to download the source tree that was used to build this binary and check the correct line numbers. Hello Stafan, it's based on fedora rawhide pkg 2:1.2.0-16 with few minor tweaks to compile on centos6. I've uploaded sources used for build here: http://nik.lbox.cz/download/qemu-kvm-1.2.0.tar.bz2 (after make clean) or http://nik.lbox.cz/download/qemu-1.2.0-lb6.01.src.rpm will this help? Stefan -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp7Roz0brVTP.pgp Description: PGP signature
Re: qemu-kvm-1.2.0: double free or corruption in VNC code
Hello Stefan, thanks! here it goes.. *** glibc detected *** /usr/bin/qemu-kvm: double free or corruption (!prev): 0x7fc634008cd0 *** === Backtrace: = /lib64/libc.so.6(+0x75916)[0x7fc9026f4916] /lib64/libc.so.6(+0x78443)[0x7fc9026f7443] /usr/bin/qemu-kvm(+0x1faeb1)[0x7fc907187eb1] /usr/bin/qemu-kvm(+0x1f0e1a)[0x7fc90717de1a] /usr/bin/qemu-kvm(+0x1fb681)[0x7fc907188681] /usr/bin/qemu-kvm(+0xed6a7)[0x7fc90707a6a7] /usr/bin/qemu-kvm(+0x195c31)[0x7fc907122c31] /usr/bin/qemu-kvm(main+0x106c)[0x7fc90711e5fc] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7fc90269dcdd] /usr/bin/qemu-kvm(+0x749f9)[0x7fc9070019f9] [...] [root@blg qemu-kvm-1.2.0]# addr2line -e /usr/lib/debug/usr/bin/qemu-kvm.debug 0x1faeb1 0x1f0e1a 0x1fb681 0xed6a7 0x195c31 0x106c /usr/src/debug/qemu-kvm-1.2.0/ui/vnc.c:499 /usr/src/debug/qemu-kvm-1.2.0/ui/vnc-enc-zrle.c:364 /usr/src/debug/qemu-kvm-1.2.0/ui/vnc.c:1037 /usr/src/debug/qemu-kvm-1.2.0/iohandler.c:159 /usr/src/debug/qemu-kvm-1.2.0/main-loop.c:499 ??:0 this makes some sense to me, since it crashed while there was VNC connection active.. It's important to fetch the debuginfo package for the exact same version of the qemu RPM you were running. sure, it's the same version. BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpCg25xs1tyP.pgp Description: PGP signature
qemu-kvm-1.2.0: double free or corruption
-7fc69800 ---p 00:00 0 7fc69800-7fc698021000 rw-p 00:00 0 . . . I guess this is not of much use, since I didn't have debuginfo package installed in time of crash. Is it possible to obtain more debuginfo after I installed it? Is there something else I should check to find where the problem could be? The system is quad core x86_64 with 32GB RAM, centos 6, running 3.0.51 kernel, qemu-kvm 1.2.0 I'd be very grateful if somebody could have a look at this. With best regards nikola ciprich -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28.rijna 168, 709 00 Ostrava tel.: +420 591 166 214 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpM1Ax3GZP1B.pgp Description: PGP signature
Re: windows PIO question
That's virtio, not IDE. well, this seems a bit strange to me, the only virtio device I can see is balloon dev, can this be it? how can I find out which operations are IDE PIO etc? sorry for asking silly questions, but I couldn't find this anywhere in the documentation... thanks in advance nik -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpHCarxQ6tkw.pgp Description: PGP signature
Re: windows PIO question
Not all PIO operations are DMA. Provide a trace and we can look. Hi, here's snippet: qemu-kvm-16441 [000] 12887154.698329: kvm_pio: pio_read at 0xc002 size 1 count 1 qemu-kvm-16441 [000] 12887154.698331: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-kvm-16441 [000] 12887154.698331: kvm_fpu: unload qemu-kvm-16441 [000] 12887154.698337: kvm_entry:vcpu 1 qemu-kvm-16441 [000] 12887154.698338: kvm_exit: [FAILED TO PARSE] exit_reason=30 guest_rip=0xf88000dabca3 isa=1 info1=3221356552 info2=0 qemu-kvm-16441 [000] 12887154.698339: kvm_emulate_insn: [FAILED TO PARSE] rip=18446735827386678435 csbase=0 len=1 insn=ECC3CC^OB7D1fEDC3 flags=9 failed=0 qemu-kvm-16441 [000] 12887154.698340: kvm_pio: pio_read at 0xc002 size 1 count 1 qemu-kvm-16441 [000] 12887154.698340: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-kvm-16441 [000] 12887154.698343: kvm_entry:vcpu 1 qemu-kvm-16441 [000] 12887154.698344: kvm_exit: [FAILED TO PARSE] exit_reason=30 guest_rip=0xf88000dabdbd isa=1 info1=3221225472 info2=0 qemu-kvm-16441 [000] 12887154.698345: kvm_pio: pio_write at 0xc000 size 1 count 1 qemu-kvm-16441 [000] 12887154.698346: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-kvm-16441 [000] 12887154.698348: kvm_entry:vcpu 1 qemu-kvm-16441 [000] 12887154.698349: kvm_exit: [FAILED TO PARSE] exit_reason=30 guest_rip=0xf88000dabdbd isa=1 info1=3221356544 info2=0 qemu-kvm-16441 [000] 12887154.698349: kvm_pio: pio_write at 0xc002 size 1 count 1 qemu-kvm-16441 [000] 12887154.698350: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-kvm-16441 [000] 12887154.698352: kvm_entry:vcpu 1 qemu-kvm-16441 [000] 12887154.698353: kvm_exit: [FAILED TO PARSE] exit_reason=30 guest_rip=0xf88000dabca3 isa=1 info1=32964616 info2=0 qemu-kvm-16441 [000] 12887154.698354: kvm_emulate_insn: [FAILED TO PARSE] rip=18446735827386678435 csbase=0 len=1 insn=ECC3CC^OB7D1fEDC3 flags=9 failed=0 qemu-kvm-16441 [000] 12887154.698354: kvm_pio: pio_read at 0x1f7 size 1 count 1 qemu-kvm-16441 [000] 12887154.698355: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-kvm-16441 [000] 12887154.698357: kvm_set_irq: gsi 14 level 0 source 0 qemu-kvm-16441 [000] 12887154.698358: kvm_pic_set_irq: chip 1 pin 6 (edge|masked) qemu-kvm-16441 [000] 12887154.698358: kvm_ioapic_set_irq: pin 14 dst 3 vec=97 (LowPrio|logical|edge) qemu-kvm-16441 [000] 12887154.698360: kvm_entry:vcpu 1 qemu-kvm-16441 [000] 12887154.698362: kvm_exit: [FAILED TO PARSE] exit_reason=30 guest_rip=0xf88000dabca3 isa=1 info1=32964616 info2=0 qemu-kvm-16441 [000] 12887154.698363: kvm_emulate_insn: [FAILED TO PARSE] rip=18446735827386678435 csbase=0 len=1 insn=ECC3CC^OB7D1fEDC3 flags=9 failed=0 qemu-kvm-16441 [000] 12887154.698363: kvm_pio: pio_read at 0x1f7 size 1 count 1 n. -- error compiling committee.c: too many arguments to function -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp9zA61R5gzG.pgp Description: PGP signature
windows PIO question
Hello everybody, I'm a bit confused regarding windows guests and PIO.. some time ago, I was trying to find out, why one of my windows guests was running terribly slow. After checking the trace-cmd output, Avi stated that windows were using PIO instead of DMA, after changing that, windows indeed got a lot faster. I didn't check traces after that, but I'm now testing host with two windows 2003 guests, and checking the trac-cmd output, I see a lot of kvm_pio calls. While windows don't seem to be running particularly slow, they're not especially fast either, and I'm wandering whether this could be related. But checking IDE bus in guests, they seem to be using DMA which confuses me... so my questions is, what are the kvm_pio operations I see in traces, and how can I tell what is OK and what is wrong, regarding disks DMA/PIO? If somebody could shed some light on this for me, I'd be very grateful... Thanks a lot in advance with best regards nikola ciprich -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpzZE8Xr5CfH.pgp Description: PGP signature
Re: [kvm] Re: [kvm] Re: [kvm] Re: [kvm] Re: [kvm] Re: [kvm] Re: tcpdump locks up kvm host for a while.
When I run tcpdump on a *guest*, the entire guest completely freezes up; no response even to hitting enter on the console. virsh list also locks up whenever it tries to print state about that VM (but the others work fine), as does any other operation that touches the state of that VM. The VM takes up 100% of CPU on one core while this is happening. Eventually it gets better. exactly the same I was describing here some time ago, with the difference I get a lot of ugly backtraces and sometimes the guest doesn't get to usable state at all (filesystems switches to read only due to errors or the whole guest locks up) n. pgpaWev9PeclA.pgp Description: PGP signature
Re: [kvm] Re: tcpdump locks up kvm host for a while.
well, give it a try ;) I still haven't fully resolved this, but I'm sure it has to do something with the timesource - with kvm-clock, it got much worse... n. [1.911056] Override clocksource tsc is not HRT compatible. Cannot switch while in HRT/NOHZ mode Will hpet do any better? -Robin -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpFV1OCGVgYN.pgp Description: PGP signature
Re: tcpdump locks up kvm host for a while.
Hello Robin, actually I'm stil hitting problems with tcpdump in KVM virtuals.. what is the content of /sys/devices/system/clocksource/clocksource0/current_clocksource? I guess it'll be kvm-clock, try using clocksource=tsc or hpet kernel parameter. does it help? n. -Robin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpHctssDgghH.pgp Description: PGP signature
KVM got slow after adding more physical memory to host
Hello, I'm trying to solve quite a weird problem on one of our customers' box. It's quad core Xeon X3430 @ 2.40GHz, running 64bit centos with 2.6.38 (I also tried upgrading to 2.6.39). After increasing physical memory from 16 to 20GB, all guest got incredibly slow, starting just one windows or linux guest causes host to be quite slow, and quest seems to be even slower. hardware virtualisation is on, KVM is used, no relevant update since the update. here's about 3 seconds of kvm_stat: kvm_exit659513 103922 kvm_entry 659505 103920 kvm_emulate_insn631118 99344 kvm_exit(IO_INSTRUCTION)625223 98398 kvm_userspace_exit 624979 98359 kvm_pio 624970 98357 but according to atop, there is almost no I/O activity from guests, and running top in them shows them to really be idle... but they eat like 60-70% of host CPU core time each.. Do You have any idea on what should I check? thanks a lot in advance nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp5f11Jp3yVZ.pgp Description: PGP signature
Re: KVM got slow after adding more physical memory to host
Hello Avi, thanks for quick reply! How many guests in there? two currently (one x86_64 centos, one winXP), with more, the box get even more unresponsive. Please post a trace as per http://www.linux-kvm.org/page/Tracing. here it goes: http://nelide.cz/nik/trace.dat.bz2 Please post /proc/mtrr and /proc/iomem. [root@virtualbox ~]# cat /proc/mtrr reg00: base=0x0 (0MB), size=16384MB, count=1: write-back reg01: base=0x4 (16384MB), size= 4096MB, count=1: write-back reg02: base=0x5 (20480MB), size= 1024MB, count=1: write-back reg03: base=0x0c000 ( 3072MB), size= 1024MB, count=1: uncachable [root@virtualbox ~]# cat /proc/iomem - : reserved 0001-0009d7ff : System RAM 0009d800-0009 : reserved 000a-000b : PCI Bus :00 000c-000c : pnp 00:0f 000d-000d : PCI Bus :00 000e-000f : reserved 0010-bf78 : System RAM 0100-013917ae : Kernel code 013917af-016c88ff : Kernel data 0176a000-0182 : Kernel bss bf79-bf79dfff : RAM buffer bf79e000-bf79 : reserved bf7a-bf7adfff : ACPI Tables bf7ae000-bf7c : ACPI Non-volatile Storage bf7d-bf7d : reserved bf7e-bf7ecfff : RAM buffer bf7ed000-bfff : reserved c000-dfff : PCI Bus :00 c000-c01f : PCI Bus :08 c020-c03f : PCI Bus :07 c040-c05f : PCI Bus :06 c060-c07f : PCI Bus :06 e000-efff : PCI MMCONFIG [bus 00-ff] e000-efff : reserved e000-efff : pnp 00:0e f000-fed8 : PCI Bus :00 fa00-faff : PCI Bus :09 fa00-faff : :09:03.0 fb4f6000-fb4f60ff : :00:1f.3 fb4f8000-fb4f87ff : :00:1f.2 fb4f8000-fb4f87ff : ahci fb4fa000-fb4fa3ff : :00:1d.0 fb4fa000-fb4fa3ff : ehci_hcd fb4fc000-fb4fc3ff : :00:1a.0 fb4fc000-fb4fc3ff : ehci_hcd fb50-fb5f : PCI Bus :07 fb5dc000-fb5d : :07:00.0 fb5dc000-fb5d : e1000e fb5e-fb5f : :07:00.0 fb5e-fb5f : e1000e fb60-fb6f : PCI Bus :08 fb6dc000-fb6d : :08:00.0 fb6dc000-fb6d : e1000e fb6e-fb6f : :08:00.0 fb6e-fb6f : e1000e fb70-fbff : PCI Bus :09 fb7fc000-fb7f : :09:03.0 fb80-fbff : :09:03.0 fc00-fcff : pnp 00:01 fd00-fdff : pnp 00:01 fe00-febf : pnp 00:01 fec0-fec003ff : IOAPIC 0 fed0-fed003ff : HPET 0 fed14000-fed19fff : pnp 00:01 fed1c000-fed1 : pnp 00:0b fed2-fed3 : reserved fed2-fed3 : pnp 00:0b fed4-fed8 : pnp 00:0b fee0-fee00fff : Local APIC fee0-fee00fff : reserved fee0-fee00fff : pnp 00:0d 1-53fff : System RAM n. -- error compiling committee.c: too many arguments to function -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpfKadJkI6ho.pgp Description: PGP signature
Re: KVM got slow after adding more physical memory to host
404 ouch, dumb me :-/ http://nelide.cz/downloads/nik/trace.dat.bz2 sorry n. Please post /proc/mtrr and /proc/iomem. [root@virtualbox ~]# cat /proc/mtrr reg00: base=0x0 (0MB), size=16384MB, count=1: write-back reg01: base=0x4 (16384MB), size= 4096MB, count=1: write-back reg02: base=0x5 (20480MB), size= 1024MB, count=1: write-back reg03: base=0x0c000 ( 3072MB), size= 1024MB, count=1: uncachable These seem fine - all your memory is write-back. -- error compiling committee.c: too many arguments to function -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp8uyErL3Qlu.pgp Description: PGP signature
Re: KVM got slow after adding more physical memory to host
OK, seems like Avi's right, at least the windows guest is not using DMA though the question is, why did it got disabled and how can I reenable it, I'm not experienced that much with XP :( It is rep/in-out. But why it became noticeable only after adding more physical memory to the host is a mystery. now that I'm thinking about it, we had similar problem in the past, the windows guest was unusably slow, and the whole host was slow as well... upgrading to 0.14.1 seemingly solved this issue, but maybe the DMA just got randomly enabled, and now after adding the memory (and restarting of course), it just got disabled again? but why could this be?? -- Gleb. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpTKZbqgtI02.pgp Description: PGP signature
Re: KVM got slow after adding more physical memory to host - SOLVED
Hello guys, thanks to both of You for Your replies. The problem is solved, exactly as Avi said, the DMA in windows got somehow disabled. So this certainly was not related to adding the memory... anyways, note for further generations: in windows XP, the DMA usage can be checked in Device Manager-IDE ATA/ATAPI Controllers - Properties-Advanced Settings Current Transfer Mode must be Multi-Word DMA2 or something similar, NOT PIO! The way I enabled this, was to uninstall both primary and secondary controller in device controller, then also uninstall Intel controller, and THEN rebooting (NOT sooner!) after reboot, controllers got detected and installed again, with DMA properly enabled. Note that when controller is in PIO mode, this is really a patience test, switching to DMA took me like half an hour to complete, so slow the system was :-/ So thanks a lot for Your help once more. have a nice day. nik On Tue, Sep 06, 2011 at 08:01:47PM +0300, Avi Kivity wrote: On 09/06/2011 07:30 PM, Nikola Ciprich wrote: OK, seems like Avi's right, at least the windows guest is not using DMA though the question is, why did it got disabled It can happen due to a timeout, perhaps a temporary load spike. and how can I reenable it, Google knows. I'm not experienced that much with XP :( It is rep/in-out. But why it became noticeable only after adding more physical memory to the host is a mystery. now that I'm thinking about it, we had similar problem in the past, the windows guest was unusably slow, and the whole host was slow as well... upgrading to 0.14.1 seemingly solved this issue, but maybe the DMA just got randomly enabled, and now after adding the memory (and restarting of course), it just got disabled again? but why could this be?? I don't think it's related to the memory. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpa4LQwmjTiz.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
So we've finally confirmed, that on cleanly installed guest, (or previously created image of w2k8) we can't reproduce the problem, only with this one particular image. I'd not worry about it anymore, I'll just have this one reinstalled. thanks for Your time and sorry for delays nik On Wed, Jun 15, 2011 at 07:26:43AM +0200, Nikola Ciprich wrote: Hello Avi, sorry, I've been off for vacation for few days now. I think even clean preinstalled guest worked properly, only the one particular production machine didn't. I'll confirm during this week. n. Does installing the same guest software into a new image work? -- error compiling committee.c: too many arguments to function -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpoc5AtDrctP.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
Hello Avi, sorry, I've been off for vacation for few days now. I think even clean preinstalled guest worked properly, only the one particular production machine didn't. I'll confirm during this week. n. Does installing the same guest software into a new image work? -- error compiling committee.c: too many arguments to function -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp25uS86ycSW.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
Hello Avi, Try appending ,cache=none to the -drive parameter? nope, unfortunately same result :( n. Maybe we have a regression with writethrough block devices (a bad idea anyway). -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpUDApdjTIY2.pgp Description: PGP signature
qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
Hello, after I tried updating our production host to 0.14.0 or 0.14.1, our windows terminal server stopped booting. Here's BSOD screen: http://nik.lbox.cz/public/wincrash.png reverting to 0.13.5 fixes the problem. I can't reproduce this on testing hardware though :( exact guest version: Microsoft Windows Server 2008 R2 Enterprise 6.1.7600 build 7600 x86_64, 4GB RAM, 1CPU host is currently running 2.6.38.7 (but I had the problem also with 2.6.37) it's 8core intel E5310 with 16GB RAM since the hosts on which I'm experiencing this problem are production machines, my possibilities of testing are a bit limited (bisecting this will be quite problematic), but I'll do what I can to help fix this... Could somebody please have a look on it? thanks a lot in advance! n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpR7DT5kOvBM.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
Hello Gleb! What is you command line? currently it's: /usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -name vmwts02 -uuid 1e501300-dc48-11df-a690-00304834195b -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vmwts02.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime -boot c -drive file=/dev/vgshared/vmwts02-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=15,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:16:3e:61:01:00,bus=pci.0,addr=0x3 -usb -vnc 0.0.0.0:30802 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 but this is with 0.13.5 (so it's running well). it's started using libvirt, I guess with 0.14.0 the command line should be very similar. (I can provide it later if needed). You can make a copy from your production image, install 0.14 version in different place from 0.13 and experiment. yup, I think I've tried it also with exact copy and wasn't able to reproduce it, but I'll try it again and report soon. n. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpg199JWTMf1.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
OK, I tried copying the whole image to my test box running 0.14.0 again and it crashes with exactly the same BSOD! So now I have the test environment to play with... What should I check/try now? n. PS: I'm not sure what You mean by permissions, but I'm using LVM partitions and qemu-kvm is started by root user anyways.. On Thu, May 26, 2011 at 01:22:37PM +0300, Gleb Natapov wrote: On Thu, May 26, 2011 at 12:20:56PM +0200, Nikola Ciprich wrote: You can make a copy from your production image, install 0.14 version in different place from 0.13 and experiment. yup, I think I've tried it also with exact copy and wasn't able to reproduce it, but I'll try it again and report soon. Then check image file permission please. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgphmXcFxNkYX.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
This BSOD usually indicates that Windows can't write to the boot disk. This is usually happens if qemu has no permission to write to the image file. But if you are starting qemu as a root this is probably is not the OK, I see. case. So what is your 0.14 command line? here it goes: /usr/bin/qemu-kvm -S -M pc-0.14 -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -name vmtst04 -uuid 1f8328b8-8849-11e0-91e9-00259009d78c -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vmtst04.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime -boot c -drive file=/dev/vgshared/vmtst04-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=14,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:16:3e:18:04:00,bus=pci.0,addr=0x3 -usb -vnc 0.0.0.0:24104 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 On Thu, May 26, 2011 at 01:22:37PM +0300, Gleb Natapov wrote: On Thu, May 26, 2011 at 12:20:56PM +0200, Nikola Ciprich wrote: You can make a copy from your production image, install 0.14 version in different place from 0.13 and experiment. yup, I think I've tried it also with exact copy and wasn't able to reproduce it, but I'll try it again and report soon. Then check image file permission please. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpVWXFpwtBQ6.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
May be libvirt does something funny with selinux. it shouldn't, I don't have selinux enabled in host kernel at all.. Try to run with -M pc-0.13. tried now, same result... -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpFCnydWiE6q.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
Hmm. And what if you start qemu directly (without using libvirt) with 0.13 command line? /usr/bin/qemu-kvm -M pc-0.13 -enable-kvm -m 4096 -boot c -drive file=/dev/vgshared/vmtst04-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -vnc 0.0.0.0:24104 same result... -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpkaghaz1g42.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
Should be more like that one with correct image path: huh, now I got a bit lost :) I tried running both: /usr/bin/qemu-kvm -M pc-0.13 -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -name vmtst04 -uuid 1f8328b8-8849-11e0-91e9-00259009d78c -nodefconfig -nodefaults -chardev socket,id=char monitor,path=/var/lib/libvirt/qemu/vmtst04.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime -boot c -drive file=/dev/vgshared/vmtst04-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=14,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:16:3e:18:04:00,bus=pci.0,addr=0x3 -usb -vnc 0.0.0.0:24104 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 and simplified: /usr/bin/qemu-kvm -M pc-0.13 -enable-kvm -m 4096 -rtc base=localtime -boot c -drive file=/dev/vgshared/vmtst04-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,driv e=drive-ide0-0-0,id=ide0-0-0 -vnc 0.0.0.0:24104 /usr/bin/qemu-kvm -M pc-0.13 -enable-kvm -m 4096 -boot c -drive file=/dev/vgshared/vmtst04-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -vnc 0.0.0.0:24104 where /dev/vgshared/vmtst04-1 is the copy of windows volume from production server. so is this ok? n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpe5c27D1b1Q.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
No, try to run the one I gave you. Just replace the image file path. I do no have much hope it will work though, but worth trying. OK, I tried, I just had to remove the monitor device. Still, it fails the same way... n. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpnpjUPCJSjF.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
According to this: http://social.msdn.microsoft.com/Forums/en-US/embeddedwindowscomponents/thread/09aae527-ff6d-4003-9e59-962d73d409ed such bsod happens because Windows can't access boot device. Your boot device is IDE. Nothing changed in this area from 13 to 14. Are you sure your image was copied correctly and is not corrupted? definitly. I just tried downgrading qemu-kvm to 0.13.0 and it works again... n. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgppxrkeAkXEy.pgp Description: PGP signature
Re: qemu-kvm-0.14.x regression - windows 2K8 R2 stopped booting
In this case I am very puzzled :) well, so am I :) but at least, I now know there seems to be problem with disk access. I'll ask some of our windows guys to try some failsave mode or something and find out what is going on... I'll report when I know something. thanks for Your help! n. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgp9091LC9pNf.pgp Description: PGP signature
Re: 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)
The guest, because latest kernels do not suffer this problem, so I'd like to find fix so it can be pushed to -stable (we're using 2.6.32.x) host is currently 2.6.37 (and i'm currently testing 2.6.38 as well) n. On Mon, May 09, 2011 at 10:32:26AM -0700, Zachary Amsden wrote: On 05/08/2011 12:06 PM, Nikola Ciprich wrote: OK, I see.. the problem is, that I'm trying to hunt down bug causing hangs when 2.6.32 guests try to run tcpdump - this seems to be reproducible even on latest 2.6.32.x, and seems like it depends on kvm-clock.. So I was thinking about bisecting between 2.6.32 and latest git which doesn't seem to suffer this problem but hitting another (different) problem in 2.6.32 complicates thinks a bit :( If somebody would have some hint on how to proceed, I'd be more then grateful.. cheers n. What are you bisecting, the host kernel or the guest kernel, and what version is the host kernel? -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpbiOBiVY8Rf.pgp Description: PGP signature
Re: 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)
That's a pretty wide range to be bisecting, and I think we know for a fact there were some kvmclock related bugs in that range. thats true, I might try to pick those that seem related and see if it helpts.. If you are looking for something causing problems with tcpdump, I'd suggest getting rid of kvmclock in your testing and using TSC instead; that's the problem, I can't reproduce the problems without kvm-clock enabled, so it must be related to it somehow.. if you're looking to verify that kvmclock related changed have been backported to -stable, rather than bisect and run into bugs, it would probably be faster to check the commit logs for arch/x86/kvm/x86.c and make sure you're not missing anything from me or Glauber that has been applied to the most recent branch. yup, I'll try and report... thanks for the hints! n. Zach -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpYcvv8ncavM.pgp Description: PGP signature
2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)
Hello everyboy, while installing new virt machine today, I noticed that 2.6.32 x86_64 SMP guests are hanging if they have paravirt-clock enabled... Either they don't finish booting at all, or boot but hang soon after.. Such a hanged guest fully loads all host cpus.. The host is 6core x86_64 runnig 2.6.37.6 with 24GB RAM. kvm_stat: kvm_exit(EXTERNAL_INTERRUPT) 156821643 kvm_exit 155141643 kvm_entry154161643 kvm_set_irq 1 0 kvm_msi_set_irq 1 0 kvm_apic_accept_irq 1 0 kvm_exit(VMCLEAR)6 0 kvm_exit(VMON) 6 0 kvm_exit(PAUSE_INSTRUCTION) 5 0 kvm_exit(MCE_DURING_VMENTRY) 5 0 kvm_exit(MWAIT_INSTRUCTION) 5 0 kvm_exit(DR_ACCESS) 5 0 kvm_exit(EPT_VIOLATION) 5 0 kvm_exit(NMI_WINDOW) 5 0 kvm_exit(VMPTRLD)5 0 kvm_exit(TASK_SWITCH)5 0 kvm_exit(VMREAD) 5 0 kvm_exit(VMLAUNCH) 5 0 kvm_exit(RDPMC) 5 0 perf top: 16.00 10.9% add_preempt_count[kernel.kallsyms] 16.00 10.9% do_raw_spin_lock [kernel.kallsyms] 15.00 10.2% sub_preempt_count[kernel.kallsyms] 8.00 5.4% irq_exit [kernel.kallsyms] 7.00 4.8% vmx_vcpu_run /lib/modules/2.6.37lb.09/kernel/arch/x86/kvm/kvm-intel.ko 7.00 4.8% page_fault [kernel.kallsyms] 5.00 3.4% mempool_free [kernel.kallsyms] info registers: RAX=00f42400 RBX=81533f00 RCX=0016 RDX=00077358f500 RSI=1dcd6500 RDI=0001 RBP=880009a03ee8 RSP=880009a03ee8 R8 =0016 R9 =000a R10= R11= R12=2a4d17d38f3303c1 R13=815fd000 R14=81592140 R15=00093510 RIP=810767cb RFL=0006 [-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0018 00c09300 DPL=0 DS [-WA] CS =0010 00a09b00 DPL=0 CS64 [-RA] SS =0018 00c09300 DPL=0 DS [-WA] DS =0018 00c09300 DPL=0 DS [-WA] FS = 000f GS = 880009a0 000f LDT= 000f TR =0040 880009a11880 2087 8b00 DPL=0 TSS64-busy GDT= 880009a04000 007f IDT= 815fd000 0fff CR0=8005003b CR2=7f424e540700 CR3=00021690c000 CR4=06f0 DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER=0d01 FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80 FPR0= FPR1= FPR2= FPR3= FPR4= FPR5= FPR6= FPR7= XMM00= XMM01=3c23d70a XMM02= XMM03= XMM04= XMM05= XMM06= XMM07= XMM08= XMM09= XMM10= XMM11= XMM12= XMM13= XMM14= XMM15= info cpus: * CPU #0: pc=0x8105d4a0 thread_id=19639 CPU #1: pc=0x81013140 thread_id=19640 CPU #2: pc=0x8102a1b6 (halted) thread_id=19641 CPU #3: pc=0x81341521 thread_id=19642 CPU #4: pc=0x810415d8 thread_id=19643 CPU #5: pc=0x811ca521 thread_id=19644 CPU #6: pc=0x81013140 thread_id=19646 CPU #7: pc=0x8102a1b6 (halted) thread_id=19647 and here are trace-cmds for all cpus: http://nik.lbox.cz/public/trace-cmd.tar.bz2 Could somebody please have a look at this? I also tried 2.6.38.5, but the result is the same... cheers nik -- - Ing. Nikola CIPRICH
Re: 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)
OK, I see.. the problem is, that I'm trying to hunt down bug causing hangs when 2.6.32 guests try to run tcpdump - this seems to be reproducible even on latest 2.6.32.x, and seems like it depends on kvm-clock.. So I was thinking about bisecting between 2.6.32 and latest git which doesn't seem to suffer this problem but hitting another (different) problem in 2.6.32 complicates thinks a bit :( If somebody would have some hint on how to proceed, I'd be more then grateful.. cheers n. On Sun, May 08, 2011 at 10:53:56PM +0400, Michael Tokarev wrote: 08.05.2011 22:33, Nikola Ciprich wrote: Hello everyboy, while installing new virt machine today, I noticed that 2.6.32 x86_64 SMP guests are hanging if they have paravirt-clock enabled... There were about 10 bugfixes pushed to 2.6.32.y stable series, some of them were for kvm-clock, and some were for problems which manifested itself like you described. You may actually take a look which guests you're booting. FWIW. /mjt -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpza8Cr1Af9M.pgp Description: PGP signature
Re: 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)
(CC Zachary) well, I should also note that while testing 2.6.37 host, I had Zach's patch fixing guest clock regression applied... n. On Sun, May 08, 2011 at 08:33:04PM +0200, Nikola Ciprich wrote: Hello everyboy, while installing new virt machine today, I noticed that 2.6.32 x86_64 SMP guests are hanging if they have paravirt-clock enabled... Either they don't finish booting at all, or boot but hang soon after.. Such a hanged guest fully loads all host cpus.. The host is 6core x86_64 runnig 2.6.37.6 with 24GB RAM. kvm_stat: kvm_exit(EXTERNAL_INTERRUPT) 156821643 kvm_exit 155141643 kvm_entry154161643 kvm_set_irq 1 0 kvm_msi_set_irq 1 0 kvm_apic_accept_irq 1 0 kvm_exit(VMCLEAR)6 0 kvm_exit(VMON) 6 0 kvm_exit(PAUSE_INSTRUCTION) 5 0 kvm_exit(MCE_DURING_VMENTRY) 5 0 kvm_exit(MWAIT_INSTRUCTION) 5 0 kvm_exit(DR_ACCESS) 5 0 kvm_exit(EPT_VIOLATION) 5 0 kvm_exit(NMI_WINDOW) 5 0 kvm_exit(VMPTRLD)5 0 kvm_exit(TASK_SWITCH)5 0 kvm_exit(VMREAD) 5 0 kvm_exit(VMLAUNCH) 5 0 kvm_exit(RDPMC) 5 0 perf top: 16.00 10.9% add_preempt_count[kernel.kallsyms] 16.00 10.9% do_raw_spin_lock [kernel.kallsyms] 15.00 10.2% sub_preempt_count[kernel.kallsyms] 8.00 5.4% irq_exit [kernel.kallsyms] 7.00 4.8% vmx_vcpu_run /lib/modules/2.6.37lb.09/kernel/arch/x86/kvm/kvm-intel.ko 7.00 4.8% page_fault [kernel.kallsyms] 5.00 3.4% mempool_free [kernel.kallsyms] info registers: RAX=00f42400 RBX=81533f00 RCX=0016 RDX=00077358f500 RSI=1dcd6500 RDI=0001 RBP=880009a03ee8 RSP=880009a03ee8 R8 =0016 R9 =000a R10= R11= R12=2a4d17d38f3303c1 R13=815fd000 R14=81592140 R15=00093510 RIP=810767cb RFL=0006 [-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0018 00c09300 DPL=0 DS [-WA] CS =0010 00a09b00 DPL=0 CS64 [-RA] SS =0018 00c09300 DPL=0 DS [-WA] DS =0018 00c09300 DPL=0 DS [-WA] FS = 000f GS = 880009a0 000f LDT= 000f TR =0040 880009a11880 2087 8b00 DPL=0 TSS64-busy GDT= 880009a04000 007f IDT= 815fd000 0fff CR0=8005003b CR2=7f424e540700 CR3=00021690c000 CR4=06f0 DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER=0d01 FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80 FPR0= FPR1= FPR2= FPR3= FPR4= FPR5= FPR6= FPR7= XMM00= XMM01=3c23d70a XMM02= XMM03= XMM04= XMM05= XMM06= XMM07= XMM08= XMM09= XMM10= XMM11= XMM12= XMM13= XMM14= XMM15= info cpus: * CPU #0: pc=0x8105d4a0 thread_id=19639 CPU #1: pc=0x81013140 thread_id=19640 CPU #2: pc=0x8102a1b6 (halted) thread_id=19641 CPU #3: pc=0x81341521 thread_id=19642 CPU #4: pc=0x810415d8 thread_id=19643 CPU #5: pc=0x811ca521 thread_id=19644 CPU #6: pc=0x81013140 thread_id=19646 CPU #7: pc=0x8102a1b6 (halted
Re: 2.6.32.x guest dies when trying to run tcpdump
OK, just wanted to let You know I were testing it quite a lot, and I'm not able to reproduce this with 2.6.38.3-rc1. So the bug must have been fixed. I'll bisect it to find proper fix so it can be posted to stable... n. On Sat, Apr 02, 2011 at 09:42:26PM +0200, Nikola Ciprich wrote: Hello Stefan! It looks like your guests are SMP. How many vcpus are you running? How many physical cpus does /proc/cpuinfo list on the host? one of guests is SMP (8cpus), one is UP, host has 2x4 cores. Is the host overloaded when this occurs? nope Are there any clues in host dmesg? nothing :( I guess I shall try 2.6.38 or maybe latest git to check if the problem is still present... Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCHv2] fix regression caused by e48672fa25e879f7ae21785c7efd187738139593
Hello Zachary, what is the current status, are You going to post this patch to Avi? I'd like to see one (or both) in stable eventually, I think it's good candidate.. BR nik - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2.6.32.x guest dies when trying to run tcpdump
] [81060c54] ? run_timer_softirq+0x214/0x320 [162335.696127] [8101d410] ? mcheck_timer+0x0/0x120 [162335.696127] [81059cdc] ? __do_softirq+0xcc/0x230 [162335.696127] [8100c59c] ? call_softirq+0x1c/0x30 [162335.696127] EOI [8100e6a5] ? do_softirq+0x65/0xa0 [162335.696127] [8105a819] ? ksoftirqd+0x79/0x1a0 [162335.696127] [8105a7a0] ? ksoftirqd+0x0/0x1a0 [162335.696127] [81070f0e] ? kthread+0x8e/0xa0 [162335.696793] [8158a140] ? early_idt_handler+0x0/0x71 [162335.696793] [8100c49a] ? child_rip+0xa/0x20 [162335.696793] [8158a140] ? early_idt_handler+0x0/0x71 [162335.696793] [81070e80] ? kthread+0x0/0xa0 [162335.696793] [8100c490] ? child_rip+0x0/0x20 [162335.697039] Aborting journal on device sda-8. [162335.697082] EXT4-fs error (device sda): ext4_journal_start_sb: [162335.697146] EXT4-fs error (device sda) in ext4_reserve_inode_write: Journal has aborted [162335.697226] EXT4-fs error (device sda) in ext4_reserve_inode_write: Journal has aborted [162335.702749] Detected aborted journal [162335.702752] EXT4-fs (sda): Remounting filesystem read-only [162335.705754] EXT4-fs error (device sda) in ext4_dirty_inode: Journal has aborted [162335.707725] EXT4-fs (sda): ext4_da_writepages: jbd2_start: 1024 pages, ino 32392; err -30 I'm using virtio network channel, and on one of the guests (the one with aborted ext4) I use it also for one of virtual disks. One more interesting thing, I can't reproduce this immediately after guest boot, but for example second day after boot, I can reproduce this. perhaps this can suggest something? Could somebody please help me to find and possibly fix this bug? If needed, I can provide further debugging information, bisect etc... Thanks a lot in advance! nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 2.6.32.x guest dies when trying to run tcpdump
Hello Stefan! It looks like your guests are SMP. How many vcpus are you running? How many physical cpus does /proc/cpuinfo list on the host? one of guests is SMP (8cpus), one is UP, host has 2x4 cores. Is the host overloaded when this occurs? nope Are there any clues in host dmesg? nothing :( I guess I shall try 2.6.38 or maybe latest git to check if the problem is still present... Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCHv2] fix regression caused by e48672fa25e879f7ae21785c7efd187738139593
So something bothers me still about this bug. What you did correctly
restores the old behavior - but it shouldn't be fixing a bug.
The only reason you need to schedule an update for the KVM clock area is
if a new VCPU has been created, you have an unstable TSC.. or something
changes the VM's kvmclock offset.
So this change could in fact be hiding an underlying bug - either an
unstable TSC is not being properly reported, the KVM clock offset is
being changed, we are missing a KVM clock update for secondary VCPUs -
or something else we don't yet understand is going on.
Nikola, can you try the patch below, which reverts your change and
attempts to fix other possible sources of the problem, and see if it
still reproduces?
with Your patch, 32b SMP guests boot fine as well, without it, they
don't, so if You consider this better fix, I can acknowledge it as
working :)
n.
Thanks,
Zach
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 58f517b..42618fb 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2127,8 +2127,10 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
if (check_tsc_unstable()) {
kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
vcpu-arch.tsc_catchup = 1;
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
}
- kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+ if (vcpu-cpu == -1)
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
if (vcpu-cpu != cpu)
kvm_migrate_timers(vcpu);
vcpu-cpu = cpu;
@@ -3534,6 +3536,8 @@ long kvm_arch_vm_ioctl(struct file *filp,
struct kvm_clock_data user_ns;
u64 now_ns;
s64 delta;
+ struct kvm_vcpu *vcpu;
+ int i;
r = -EFAULT;
if (copy_from_user(user_ns, argp, sizeof(user_ns)))
@@ -3549,6 +3553,8 @@ long kvm_arch_vm_ioctl(struct file *filp,
delta = user_ns.clock - now_ns;
local_irq_enable();
kvm-arch.kvmclock_offset = delta;
+ kvm_for_each_vcpu(i, vcpu, kvm)
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
break;
}
case KVM_GET_CLOCK: {
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
msix_unset_mask_notifier: Assertion `dev-msix_mask_notifier' failed.
Hello, I wanted to give vhost_net a try, but I can't make it work, when I try to start qemu-kvm with vhost interface, it crashers with msix_unset_mask_notifier: Assertion `dev-msix_mask_notifier' failed. according to some threads I've digged up, it is related to missing eventfd support (I'm using centos 5 for host). host kernel is 2.6.37, arch x86_64, I'm using qemu-kvm-0.14.0 is there some workaround for this? thanks in advance nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.14.0 msix_mask_notifier failed.
Hello Dietmar, did You somehow fixed this problem? BR nik On Tue, Mar 01, 2011 at 11:43:39AM +, Dietmar Maurer wrote: Seems the bug is related to EVENTFD support. int event_notifier_init(EventNotifier *e, int active) That simply return -ENOSYS on older glibc ( 2.8) I can define the syscall manually, but I wonder if there is a better way? - Dietmar I get this error when I run new kvm 0.14.0 on kernel 2.6.35 (vhost=on): # kvm -netdev type=tap,id=n0,vhost=on -device virtio-net-pci,netdev=n0 -cdrom ubuntu-10.10-desktop-amd64.iso kvm: /home/code/qemu-kvm/hw/msix.c:639: msix_unset_mask_notifier: Assertion `dev-msix_mask_notifier' failed. Any ideas? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] add missing guest clock update removed by e48672fa25e879f7ae21785c7efd187738139593
Can you try moving the kvm_make_request() inside the if conditional and
see if it that also fixes it?
yes, changing to:
if (unlikely(vcpu-cpu != cpu) || check_tsc_unstable()) {
kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
/* Make sure TSC doesn't go backwards */
s64 tsc_delta = !vcpu-arch.last_host_tsc ? 0 :
is also OK.
what about changing:
if (check_tsc_unstable()) {
kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
vcpu-arch.tsc_catchup = 1;
kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
}
to:
if (check_tsc_unstable()) {
kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
vcpu-arch.tsc_catchup = 1;
}
kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
this fixes thinks for me as well..
n.
?
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv2] fix regression caused by e48672fa25e879f7ae21785c7efd187738139593
commit 387b9f97750444728962b236987fbe8ee8cc4f8c moved
kvm_request_guest_time_update(vcpu),
breaking 32bit SMP guests using kvm-clock. Fix this by moving (new) clock
update function
to proper place.
Signed-off-by: Nikola Ciprich nikola.cipr...@linuxbox.cz
---
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 4c27144..ba3f76f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2101,8 +2101,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
if (check_tsc_unstable()) {
kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
vcpu-arch.tsc_catchup = 1;
- kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
}
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
if (vcpu-cpu != cpu)
kvm_migrate_timers(vcpu);
vcpu-cpu = cpu;
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] add missing guest clock update removed by e48672fa25e879f7ae21785c7efd187738139593
e48672fa25e879f7ae21785c7efd187738139593 removed
kvm_request_guest_time_update(vcpu);
this breaks 32bit SMP guests using virtio-clock.
thus add unconditional call to kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) to
fix the
problem.
Signed-off-by: Nikola Ciprich nikola.cipr...@linuxbox.cz
---
arch/x86/kvm/x86.c |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bcc0efc..4c27144 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2091,6 +2091,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
}
kvm_x86_ops-vcpu_load(vcpu, cpu);
+ kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
if (unlikely(vcpu-cpu != cpu) || check_tsc_unstable()) {
/* Make sure TSC doesn't go backwards */
s64 tsc_delta = !vcpu-arch.last_host_tsc ? 0 :
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
I did read your mail, but I was working on an old tree... because of that transformation, this fix will unfortunately have to be back and forward ported by hand. OK, sorry, I didn't mean to be adverse... Did you try just that change right applied on top of the patch (e48672fa25e879f7ae21785c7efd187738139593) implicated by bisect? yes, with host running e48672fa25e879f7ae21785c7efd187738139593, 32bit SMP guest doesn't boot, when I add kvm_request_guest_time_update(vcpu), it helps. It will be great to know if that change alone fixes the problem, if so, the fix you propose is probably the right one for upstream. ok, so shell I submit patch adding kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu)? this fixes things for me for 2.6.37. Thanks, Zach -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Hello Zachary, You don't see any messages about TSC being unstable or switching clocksource after loading the KVM module? And you are not suspending the host or anything? no messages, no suspending, nothing. Can you try using processor.max_cstate=1 on the host as a kernel parameter and see if it makes a difference? I tried it, no change.. n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Zach, I don't understand 100 % the logic behind all your tsc changes. But kvm-clock-wise, most of the problems we had in the past were related to the difference in resolution between the tsc and the host clocksource (hpet, acpi_pm, etc), which in his case, it is a non-issue. It does seem to me like some compensation logic kicked in, dismantling an otherwise good tsc. He does have nonstop_tsc, which means it can't get any better. One thing I noticed when reading the culprit patch in bisect, is that in vcpu_load(), there were previously a call to kvm_request_guest_time_update(vcpu) that was removed without a counterpart addition. Any idea about why it was done? Nikola, does adding that line back alleviate the problem for you ? Hello Glauber, kvm_request_guest_time_update seems to have been renamed and then removed since then, but I've added kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu); instead and now the guest boots! So maybe missing clock update is really the culprit here? What do You guys think? n. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
I think although the long term plan is to just do this update once in
your case (stable tsc), this update is needed.
Why don't you send a patch to re-include it ?
Yes, I'll gladly submit patch, one question, is this OK
to just add calling kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) before
the conditional (as I did in my test), or should it go somewhere to else {..}
section? it's called inside the conditional again, which will cause it
to be called twice in some cases, is it OK?
n.
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Can you try this patch to see if it fixes the problem?
You haven't read my replies, did you? ;-)
kvm_request_guest_time_update seems to have been
removed, and kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu)
seems to be used instead, adding it fixes the problem.
That's what I was going to use in the patch... :)
Thanks,
Zach
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 468fafa..ba05303 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1866,6 +1866,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
}
kvm_x86_ops-vcpu_load(vcpu, cpu);
+ kvm_request_guest_time_update(vcpu);
if (unlikely(vcpu-cpu != cpu)) {
/* Make sure TSC doesn't go backwards */
s64 tsc_delta = !vcpu-arch.last_host_tsc ? 0 :
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
What is the exact kernel version you are using in the guest. It's latest centos (2.6.18-194.32.1.el5), so I guess there are a lot of fixes, but it's possible the kvm-clock is broken in it. I can't influence what kernel is used there (at least not on customer's guests), but I guess asking for adding clocksource kernel parameter is not problem. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
That sounds like a kernel which will be vulnerable to broken KVM clock on 32-bit. There's a kernel side fix that is needed, but why the server side change triggers the problem needs more investigation. OK, it's important for me that I can fix this by kernel parameter, but if I can help somehow with debugging, please let me know. thanks for Your time! nik Zach -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
No worries. What mess? twice sending the same mail, nevermind :) I have two things you can try: first is running a single VCPU guest, if you have not done so already. yup, UP guest is fine, just SMP doesn't work. Second is adding the bootparameter clocksource=acpi_pm to your guest kernel. yes, this makes SMP work too! I just realized when You were asking about current clocksource, I told You only host source, not the guest. So I checked now, and (at least for UP, I guess for SMP it's the same), the clocksource is kvm-clock! So seems like it got broken with the TSC changes? If either of those fixes the problem, it very well have to do with this change and not that you may be missing later dependent patches. This change should be nearly a 1-1 transformation, and if it is not, something is wrong. What branch are you bisecting on, the kvm branch or the kernel tree itself? It would be helpful to see the exact code in case any surrouding logic changed. I was bisecting linus' linux-2.6.git main branch, between 2.6.36..2.6.37 Thanks, Zach -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
recommended timer frequency for host and guest
Hello, I'd like to ask, are there any recommendation about kernel timer frequency for guest and for the host? I guess that on host, the higher frequency can be better, but for guest, 100HZ should be better because it causes lower overhead for host, right? Thanks a lot for reply in advance cheers nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Does the bug you are hitting manifest on both Intel and AMD platforms? I don't have any AMD box here, I'll try this out at my home box. Further, do the systems you are hitting this on have stable or unstable TSCs? how do I find this out? I don't see any warning about TSC in guest, but I've just started it.. n. Thanks, Zach -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
On Mon, Feb 28, 2011 at 10:17:24AM -0500, Zachary Amsden wrote: On 02/28/2011 09:32 AM, Nikola Ciprich wrote: Does the bug you are hitting manifest on both Intel and AMD platforms? I don't have any AMD box here, I'll try this out at my home box. Further, do the systems you are hitting this on have stable or unstable TSCs? how do I find this out? I don't see any warning about TSC in guest, but I've just started it.. n. Before worrying about the guest, is the host TSC stable? What is the host clocksource? not sure, I'm not setting anything specifically, is this snippet of dmesg relevant: [1.148829] HPET: 8 timers in total, 5 timers will be used for per-cpu timer [1.148934] hpet0: at MMIO 0xfed0, IRQs 2, 8, 40, 41, 42, 43, 44, 0 [1.149331] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [1.151831] hpet: hpet2 irq 40 for MSI [1.151962] hpet: hpet3 irq 41 for MSI [1.155930] hpet: hpet4 irq 42 for MSI [1.159937] hpet: hpet5 irq 43 for MSI [1.163943] hpet: hpet6 irq 44 for MSI [1.175955] Switching to clocksource tsc so I guess I'm using hpet? n. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
(resend, sorry for the mess) cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 30 model name: Intel(R) Xeon(R) CPU X3440 @ 2.53GHz stepping : 5 cpu MHz : 2533.185 cache size : 8192 KB physical id : 0 siblings : 8 core id: 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu: yes fpu_exception : yes cpuid level: 11 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca c= mov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rd= tscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_= tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pd= cm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vp= id bogomips : 5066.37 clflush size : 64 cache_alignment : 64 address sizes: 36 bits physical, 48 bits virtual power management: . . . . processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 30 model name: Intel(R) Xeon(R) CPU X3440 @ 2.53GHz stepping : 5 cpu MHz : 2533.185 cache size : 8192 KB physical id : 0 siblings : 8 core id: 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu: yes fpu_exception : yes cpuid level: 11 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca c= mov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rd= tscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_= tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pd= cm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vp= id bogomips : 5066.35 clflush size : 64 cache_alignment : 64 address sizes: 36 bits physical, 48 bits virtual power management: cat /sys/devices/system/clocksource/clocksource0/current_clocksource [root@vbox5 ~]# cat /sys/devices/system/clocksource/clocksource0/current_cl= ocksource tsc and grep -i dmesg for these keywords: TSC, clock, hpet, stable, khz, kvm [root@vbox5 ~]# dmesg | grep -i tsc\|clock\|hpet\|stable\|stable\|khz\|kvm [0.00] ACPI: HPET bf7aa5f0 00038 (v01 052710 OEMHPET 20100= 527 MSFT 0097) [0.00] ACPI: HPET id: 0x8086a701 base: 0xfed0 [0.00] hpet clockevent registered [0.00] Fast TSC calibration using PIT [1.148829] HPET: 8 timers in total, 5 timers will be used for per-cpu t= imer [1.148934] hpet0: at MMIO 0xfed0, IRQs 2, 8, 40, 41, 42, 43, 44, 0 [1.149331] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [1.151831] hpet: hpet2 irq 40 for MSI [1.151962] hpet: hpet3 irq 41 for MSI [1.155930] hpet: hpet4 irq 42 for MSI [1.159937] hpet: hpet5 irq 43 for MSI [1.163943] hpet: hpet6 irq 44 for MSI [1.175955] Switching to clocksource tsc [1.260015] CE: hpet3 increased min_delta_ns to 7500 nsec [1.260117] CE: hpet3 increased min_delta_ns to 11250 nsec [1.294150] Real Time Clock Driver v1.12b [7.564355] CE: hpet4 increased min_delta_ns to 7500 nsec [7.564367] CE: hpet4 increased min_delta_ns to 11250 nsec [ 299.307242] CE: hpet2 increased min_delta_ns to 7500 nsec [ 299.307251] CE: hpet2 increased min_delta_ns to 11250 nsec [ 1414.616685] CE: hpet5 increased min_delta_ns to 7500 nsec [ 1414.616694] CE: hpet5 increased min_delta_ns to 11250 nsec [ 5241.474310] CE: hpet6 increased min_delta_ns to 7500 nsec [ 5241.474321] CE: hpet6 increased min_delta_ns to 11250 nsec -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 30 model name: Intel(R) Xeon(R) CPU X3440 @ 2.53GHz stepping : 5 cpu MHz : 2533.185 cache size : 8192 KB physical id : 0 siblings : 8 core id: 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu: yes fpu_exception : yes cpuid level: 11 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid bogomips : 5066.37 clflush size : 64 cache_alignment : 64 address sizes: 36 bits physical, 48 bits virtual power management: . . . . . processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 30 model name: Intel(R) Xeon(R) CPU X3440 @ 2.53GHz stepping : 5 cpu MHz : 2533.185 cache size : 8192 KB physical id : 0 siblings : 8 core id: 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu: yes fpu_exception : yes cpuid level: 11 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid bogomips : 5066.35 clflush size : 64 cache_alignment : 64 address sizes: 36 bits physical, 48 bits virtual power management: cat /sys/devices/system/clocksource/clocksource0/current_clocksource [root@vbox5 ~]# cat /sys/devices/system/clocksource/clocksource0/current_clocksource tsc and grep -i dmesg for these keywords: TSC, clock, hpet, stable, khz, kvm [root@vbox5 ~]# dmesg | grep -i tsc\|clock\|hpet\|stable\|stable\|khz\|kvm [0.00] ACPI: HPET bf7aa5f0 00038 (v01 052710 OEMHPET 20100527 MSFT 0097) [0.00] ACPI: HPET id: 0x8086a701 base: 0xfed0 [0.00] hpet clockevent registered [0.00] Fast TSC calibration using PIT [1.148829] HPET: 8 timers in total, 5 timers will be used for per-cpu timer [1.148934] hpet0: at MMIO 0xfed0, IRQs 2, 8, 40, 41, 42, 43, 44, 0 [1.149331] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [1.151831] hpet: hpet2 irq 40 for MSI [1.151962] hpet: hpet3 irq 41 for MSI [1.155930] hpet: hpet4 irq 42 for MSI [1.159937] hpet: hpet5 irq 43 for MSI [1.163943] hpet: hpet6 irq 44 for MSI [1.175955] Switching to clocksource tsc [1.260015] CE: hpet3 increased min_delta_ns to 7500 nsec [1.260117] CE: hpet3 increased min_delta_ns to 11250 nsec [1.294150] Real Time Clock Driver v1.12b [7.564355] CE: hpet4 increased min_delta_ns to 7500 nsec [7.564367] CE: hpet4 increased min_delta_ns to 11250 nsec [ 299.307242] CE: hpet2 increased min_delta_ns to 7500 nsec [ 299.307251] CE: hpet2 increased min_delta_ns to 11250 nsec [ 1414.616685] CE: hpet5 increased min_delta_ns to 7500 nsec [ 1414.616694] CE: hpet5 increased min_delta_ns to 11250 nsec [ 5241.474310] CE: hpet6 increased min_delta_ns to 7500 nsec [ 5241.474321] CE: hpet6 increased min_delta_ns to 11250 nsec -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpa7UOdTfwcn.pgp Description: PGP signature
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
I was not aware of the thread. Please cc me directly, or add a keyword
I track - timekeeping, TSC..
Hello Zachary, thanks for Your time looking at this!
That change alone may not bisect well; without further fixes on top of
it, you may end up with a hang or stall, which is likely to manifest in
a vendor-specific way.
I'm not sure I really understand You here, but this change is exactly to
what I got while bisecting. With later revisions, including this one,
32bit SMP guests don't boot, before it, they do..
Basically there were a few differences in the platform code about how
TSC was dealt with on systems which did not have stable clocks, this
brought the logic into one location, but there was a slight change to
the logic here.
Note very carefully, the logic on SVM is gated by a condition before
this change:
if (unlikely(cpu != vcpu-cpu)) {
- u64 delta;
-
- if (check_tsc_unstable()) {
- /*
-* Make sure that the guest sees a monotonically
-* increasing TSC.
-*/
- delta = vcpu-arch.host_tsc - native_read_tsc();
- svm-vmcb-control.tsc_offset += delta;
- if (is_nested(svm))
- svm-nested.hsave-control.tsc_offset +=
delta;
- }
- vcpu-cpu = cpu;
- kvm_migrate_timers(vcpu);
So this only happens with a system which reports TSC as unstable. After
the change, KVM itself may report the TSC as unstable:
+ if (unlikely(vcpu-cpu != cpu)) {
+ /* Make sure TSC doesn't go backwards */
+ s64 tsc_delta = !vcpu-arch.last_host_tsc ? 0 :
+ native_read_tsc() -
vcpu-arch.last_host_tsc;
+ if (tsc_delta 0)
+ mark_tsc_unstable(KVM discovered backwards TSC);
+ if (check_tsc_unstable())
+ kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
+ kvm_migrate_timers(vcpu);
+ vcpu-cpu = cpu;
+ }
If the platform has very small TSC deltas across CPUs, but indicates the
TSC is stable, this could result in KVM marking the TSC unstable. If
that is the case, this compensation logic will kick in to avoid
backwards TSCs.
Note however, that the logic is not perfect; time which passes while not
running on any CPU will be erased, as the delta compensation removes not
just backwards, but any elapsed time from the TSC. In extreme cases,
this could result in time appearing to stand still with guests
failing to boot.
This was addressed with a later change, which catches up the missing time:
commit c285545f813d7b0ce989fd34e42ad1fe785dc65d
yes, but this change is already included in 2.6.37, so maybe some other fix is
needed?
if You have some idea what could be changed, I'll gladly test whatever You
recommend,
but I'm afraid that's all I can do, since this is a bit of a rocket science for
me, sorry :(
nik
Author: Zachary Amsden zams...@redhat.com
Date: Sat Sep 18 14:38:15 2010 -1000
KVM: x86: TSC catchup mode
Negate the effects of AN TYM spell while kvm thread is preempted by
tracking
conversion factor to the highest TSC rate and catching the TSC up
when it has
fallen behind the kernel view of time. Note that once triggered, we
don't
turn off catchup mode.
A slightly more clever version of this is possible, which only does
catchup
when TSC rate drops, and which specifically targets only CPUs with
broken
TSC, but since these all are considered unstable_tsc(), this patch
covers
all necessary cases.
Signed-off-by: Zachary Amsden zams...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
-
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28. rijna 168, 709 01 Ostrava
tel.: +420 596 603 142
fax:+420 596 621 273
mobil: +420 777 093 799
www.linuxbox.cz
mobil servis: +420 737 238 656
email servis: ser...@linuxbox.cz
-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
(CC: Zachary) Hello, Zachary, in case You haven't noticed the thread, we're trying to find out the reason why 32bit SMP guests stopped working in 2.6.37. bisect shows this as the culprit: e48672fa25e879f7ae21785c7efd187738139593 is first bad commit commit e48672fa25e879f7ae21785c7efd187738139593 Author: Zachary Amsden zams...@redhat.com Date: Thu Aug 19 22:07:23 2010 -1000 KVM: x86: Unify TSC logic Move the TSC control logic from the vendor backends into x86.c by adding adjust_tsc_offset to x86 ops. Now all TSC decisions can be done in one place. Signed-off-by: Zachary Amsden zams...@redhat.com Signed-off-by: Marcelo Tosatti mtosa...@redhat.com Unfortunately I couldn't try 2.6.37 with just this one reverted, certainly other patches rely on it, but hopefully I've not screwed something while bisecting... so what now? n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
On Thu, Feb 24, 2011 at 12:17:40PM +0200, Avi Kivity wrote: On 02/24/2011 01:42 AM, Nikola Ciprich wrote: Hello Avi et al, seems like I've hit regression in 2.6.37: 32bit SMP centos guest stopped booting, they just hang during initrd phase. (haven't tried different distros) UP guest are OK. when I (forcibly) compiled kvm-kmod-2.6.36.2 and used it in 2.6.37, even the SMP guests boot fine. does somebody have a tip on where the problem could be, or should I bisect this? I tried on 2 different machines, host is x86_64, qemu-kvm 0.13.0, 0.14.0. If I shall provide more information (or bisect), please let me know. Bisect is of course great, if laborious. Meanwhile can you post 'info registers' for all cpus? Is the guest consuming cpu? kvm_stat output? yes, it's eating 100% of one CPU core. kvm_stat for few seconds (hunged guest is the only one running on the host): kvm_entry293279091 kvm_exit 293579090 kvm_inj_virq 245887609 kvm_apic_accept_irq 171465310 kvm_emulate_insn 126823931 kvm_apic 125303879 kvm_mmio 125253879 kvm_exit(APIC_ACCESS)125253879 kvm_exit(HLT)112623466 kvm_ioapic_set_irq65322024 kvm_set_irq 65382024 kvm_pic_set_irq 65362024 kvm_exit(EXTERNAL_INTERRUPT) 42551300 kvm_ack_irq 2442 756 kvm_exit(PENDING_INTERRUPT) 1030 335 kvm_exit(IO_INSTRUCTION) 313 104 kvm_pio312 104 kvm_age_page18 6 kvm_exit(EPT_VIOLATION) 14 4 kvm_page_fault 12 4 kvm_exit(INVALID_STATE) 4 0 kvm_exit(VMLAUNCH) 3 0 kvm_exit(CPUID) 3 0 kvm_exit(DR_ACCESS) 2 0 kvm_exit(MSR_READ) 2 0 kvm_exit(PAUSE_INSTRUCTION) 1 0 info registers: EAX= EBX=6a00 ECX=000a EDX=000f41a8 ESI=000f41a8 EDI= EBP=c0690320 ESP=c0769f58 EIP=c042d137 EFL=0002 [---] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =007b 00c0f300 DPL=3 DS [-WA] CS =0060 00c09b00 DPL=0 CS32 [-RA] SS =0068 00c09300 DPL=0 DS [-WA] DS =007b 00c0f300 DPL=3 DS [-WA] FS = GS = LDT=0088 c0747020 0027 8200 DPL=0 LDT TR =0080 c300f380 2073 8b00 DPL=0 TSS32-busy GDT= c302b000 00ff IDT= c06f7000 07ff CR0=8005003b CR2=ffc46000 CR3=00743000 CR4=06d0 DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER= FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80 FPR0= FPR1= FPR2= FPR3= FPR4= FPR5= FPR6=800bf600 4015 FPR7= XMM00= XMM01= XMM02= XMM03= XMM04= XMM05= XMM06= XMM07= I'll wait a bit with bisect whether You'll spot some obvious bug or not ;) thanks for Your time! PS: I still owe You the kvm_stat comparison about this slow windows chkdsk problem, I'm aware of it, I just had to postpone this due to more urgent matters :( but I'll get back to it sooner or later.. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Not very useful when the guest is making progress, I'm afraid. can perf report help here? Can you try a little trace-cmd -e kvm -b 2? ugh, I'm afraid I'll have some dumb questions here :-[ You mean this: git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git ? and then re-execute qemu-kvm using it? or I'm totally wrong? Sure. Something similar that came up - sometimes Windows IDE drivers fall back to PIO mode. Are you using IDE? If so, please check whether it's using DMA or PIO. I'll check, but this problem occurs only during fsck phase, when to guest boots, then it runs pretty fast.. so maybe during boot it might fall back to PIO, but from guest, I guess I won't have a chance to find out.. can I somehow check it from host? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
The only activity I can see is the timer interrupt, so I'm afraid a bisect is needed. OK, nevermind, it's easy to reproduce, so I'll just bisect it and report. n. If you let git bisect just kvm, it'll be a bit faster: $ git bisect $BAD $GOOD virt/kvm arch/x86/kvm -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Yes. If you have udis86 and udis86-devel installed when building it, it's even better. yes, now I remember! I've done some tracing for You already.. You don't have to execute qemu-kvm under it, if you have a running instance you can run trace-cmd in parallel and it will record whatever's happening. I've uploaded the report for You here: nelide.cz/downloads/nik/report.txt.xz The trace-cmd output will show. Please run trace-cmd report afterwards and post the results somewhere. OK, I'll prepare some new windows testing machine, try and report.. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - pgpVQJypywZSY.pgp Description: PGP signature
Re: [ANNOUNCE] qemu-kvm-0.14.0
On Wed, Feb 23, 2011 at 09:05:37AM +0100, klondike wrote: I hit this problem too and after a long night I tracked it back to some odd merge, this patch solved the issue for me. 2010-11-10Francisco Blas Izquierdo Riera klond...@xiscosoft.es Somehow pci.o was added so it was needed on all targets breaking NO_PCI ones. This patch intends to fix that * Makefile.objs:Add pci.o back into hw-obj-$(CONFIG_PCI) * Makefile.target: Remove pci.o diff -ru /tmp/aaa/qemu-kvm-0.14.0.old/Makefile.objs /tmp/aaa/qemu-kvm-0.14.0/Makefile.objs --- /tmp/aaa/qemu-kvm-0.14.0.old/Makefile.objs2011-02-23 06:02:16.818182597 +0100 +++ /tmp/aaa/qemu-kvm-0.14.0/Makefile.objs2011-02-23 06:00:35.439182597 +0100 @@ -170,7 +170,7 @@ hw-obj-y += loader.o hw-obj-$(CONFIG_VIRTIO) += virtio.o virtio-console.o hw-obj-y += fw_cfg.o -hw-obj-$(CONFIG_PCI) += pci_bridge.o +hw-obj-$(CONFIG_PCI) += pci.o pci_bridge.o hw-obj-$(CONFIG_PCI) += msix.o msi.o hw-obj-$(CONFIG_PCI) += pci_host.o pcie_host.o hw-obj-$(CONFIG_PCI) += ioh3420.o xio3130_upstream.o xio3130_downstream.o diff -ru /tmp/aaa/qemu-kvm-0.14.0.old/Makefile.target /tmp/aaa/qemu-kvm-0.14.0/Makefile.target --- /tmp/aaa/qemu-kvm-0.14.0.old/Makefile.target 2011-02-23 06:02:16.835182597 +0100 +++ /tmp/aaa/qemu-kvm-0.14.0/Makefile.target 2011-02-23 06:00:35.454182597 +0100 @@ -195,7 +195,7 @@ # System emulator target ifdef CONFIG_SOFTMMU -obj-y = arch_init.o cpus.o monitor.o pci.o machine.o gdbstub.o vl.o balloon.o +obj-y = arch_init.o cpus.o monitor.o machine.o gdbstub.o vl.o balloon.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly obj-$(CONFIG_NO_PCI) += pci-stub.o Hello, yup, I've noticed Your patch just after sending my report.. thansk a lot, I'll try it right away! have a nice day nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] qemu-kvm-0.14.0
] Error 2 what could this be? -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot
Hello Avi et al, seems like I've hit regression in 2.6.37: 32bit SMP centos guest stopped booting, they just hang during initrd phase. (haven't tried different distros) UP guest are OK. when I (forcibly) compiled kvm-kmod-2.6.36.2 and used it in 2.6.37, even the SMP guests boot fine. does somebody have a tip on where the problem could be, or should I bisect this? I tried on 2 different machines, host is x86_64, qemu-kvm 0.13.0, 0.14.0. If I shall provide more information (or bisect), please let me know. cheers! nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] qemu-kvm-0.14.0
Hello Avi, when trying to compile(link) various targets, I get: LINK sparc-softmmu/qemu-system-sparc pci-stub.o: In function `do_pci_info_print': /usr/src/redhat/BUILD/qemu-kvm-0.14.0/hw/pci-stub.c:36: multiple definition of `do_pci_info_print' pci.o:/usr/src/redhat/BUILD/qemu-kvm-0.14.0/hw/pci.c:1398: first defined here /usr/bin/ld: Warning: size of symbol `do_pci_info_print' changed from 147 in pci.o to 50 in pci-stub.o pci-stub.o: In function `do_pci_info': /usr/src/redhat/BUILD/qemu-kvm-0.14.0/hw/pci-stub.c:31: multiple definition of `do_pci_info' pci.o:/usr/src/redhat/BUILD/qemu-kvm-0.14.0/hw/pci.c:1560: first defined here /usr/bin/ld: Warning: size of symbol `do_pci_info' changed from 150 in pci.o to 50 in pci-stub.o pci.o: In function `pci_update_mappings': pci.c:(.text+0xa91): undefined reference to `pci_bridge_get_base' pci.c:(.text+0xaa5): undefined reference to `pci_bridge_get_limit' pci.c:(.text+0xb7b): undefined reference to `pci_bridge_get_base' pci.c:(.text+0xb8e): undefined reference to `pci_bridge_get_limit' pci.o: In function `pci_get_devices_list': pci.c:(.text+0x2024): undefined reference to `pci_bridge_get_limit' pci.c:(.text+0x2036): undefined reference to `pci_bridge_get_base' pci.c:(.text+0x2043): undefined reference to `pci_bridge_get_limit' pci.c:(.text+0x2050): undefined reference to `pci_bridge_get_base' pci.c:(.text+0x2060): undefined reference to `pci_bridge_get_limit' pci.c:(.text+0x2070): undefined reference to `pci_bridge_get_base' collect2: ld returned 1 exit status make[1]: *** [qemu-system-sparc] Error 1 make: *** [subdir-sparc-softmmu] Error 2 I'm compiling on x86_64 centos5. Any idea on what I might be doing wrong or how can it be fixed? thanks a lot in advance! BR nik On Tue, Feb 22, 2011 at 07:14:41PM +0200, Avi Kivity wrote: qemu-kvm-0.14.0 is now available. This release is based on the upstream qemu 0.14.0, plus kvm-specific enhancements. Please see the original qemu 0.14.0 release announcement for details. This release can be used with the kvm kernel modules provided by your distribution kernel, or by the modules in the kvm-kmod package, such as kvm-kmod-2.6.37. http://www.linux-kvm.org -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
qemu-kvm-0.13.0, 2.6.37.1 - after migration, some of guests got stuck
Hello, I've just migrated a bunch of guests from one node to another due to upgrading host kernel from 2.6.37 to 2.6.37.1. After that, some of guests started consuming 100% of CPU time and their os seems to be stuck. Looks like all the stuck guest were running 2.6.32 with kvm paravirt enabled. others running same as well as different kernels seem to be ok, including windows guests. I guess I can keep stuck guest in this state for few hours, so if somebody would be willing to have a look at this, I'll be happy to try to help. running kvm_stat for few seconds reports: kvm_exit237284 19119 kvm_entry 237183 19119 kvm_emulate_insn123548 10088 kvm_mmio 929477622 kvm_exit(IO_INSTRUCTION) 907447395 kvm_pio 907267395 kvm_exit(EXCEPTION_NMI) 699935416 kvm_page_fault 692545347 kvm_apic 515284157 host is x86_64, guests were x86_64 too.. BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: who cames from xen?
Hi, I switched from XEN to KVM long time ago, and haven't felt sorry since then... Are You interestid in something in particular? n. On Thu, Feb 10, 2011 at 03:28:10PM +, Mauro wrote: I'm using xen for years with no problems in my production environments. Now I want to try kvm. Any experiences here from xen to kvm? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: who cames from xen?
Well, mine are pretty much the same as of those who already replied. to emphasize the most important for me: - xen developers didn't seem that much interested to push everything into mainline, in general kvm developmen process seem much open to me.., - it was problematic for me to use some of new features we needed for such old kernels XEN's been based on - after the xen has been bought by citrix, future course was unclear - redhat which we've based our distro upon switched to KVM as well (and bought qumranet) - since KVM runs VMs as normal processes, there are better possibilities to use various types of shaping using cgroups etc. - KVM seems to be simpler to debug to me and community is pretty friendly here well, thats enough I guess :) all I have to say is that I too am pretty gratefull to KVM and also QEMU developers. thanks guys! nik On Thu, Feb 10, 2011 at 09:20:17PM +, Mauro wrote: On 10 February 2011 19:30, Nikola Ciprich extmaill...@linuxbox.cz wrote: Hi, I switched from XEN to KVM long time ago, and haven't felt sorry since then... Are You interestid in something in particular? Then.I'm interested on your motivations to switch from xen to kvm. If it's important I use debian squeeze. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Avi, I was forced to interrupt checkdisk on this machine, I tried it again and it's reproducible (even on another (testing) W2K8 guests). I transfered the image to testing machine where chkdsk runs OK and finished it there. So in case You'd like to continue debugging, I'm at Your disposal, we can create more testing guests and play with them on host where I can reproduce. So far thanks a lot for all the time You've spent with it. BR nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
What are the differences between the host the completes chkdsk and the host that fails? Hardware-wise and software-wise. Software-wise, I'm not aware of any, both nodes use: - kernel 2.6.36.2 - qemu-kvm-0.13.0 - seabios-0.6.0 hardware-wise, problematic machine is older 8core, 16GB RAM, cpuinfo snippet: processor : 7 vendor_id : GenuineIntel cpu family : 6 model: 15 model name : Intel(R) Xeon(R) CPU E5310 @ 1.60GHz stepping : 7 cpu MHz: 1599.873 cache size : 4096 KB physical id: 1 siblings : 4 core id: 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu: yes fpu_exception : yes cpuid level: 10 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts tpr_shadow bogomips : 3200.27 clflush size : 64 cache_alignment : 64 address sizes: 36 bits physical, 48 bits virtual nonproblematic machine is 8core 4GB, cpuinfo snippet: processor : 7 vendor_id : GenuineIntel cpu family : 6 model: 30 model name : Intel(R) Xeon(R) CPU X3440 @ 2.53GHz stepping : 5 cpu MHz: 2533.260 cache size : 8192 KB physical id: 0 siblings : 8 core id: 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu: yes fpu_exception : yes cpuid level: 11 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid bogomips : 5066.35 clflush size : 64 cache_alignment : 64 address sizes: 36 bits physical, 48 bits virtual important to note is, that botch machines are failover clusters, I can reproduce even on second node of problematic cluster, although CPU is a bit different, snippet follows: processor: 7 vendor_id: GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5420 @ 2.50GHz stepping : 6 cpu MHz : 2500.038 cache size : 6144 KB physical id : 1 siblings : 4 core id: 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu: yes fpu_exception : yes cpuid level: 10 wp : yes flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dts tpr_shadow vnmi flexpriority bogomips : 5000.40 clflush size : 64 cache_alignment : 64 address sizes: 38 bits physical, 48 bits virtual What else could I check? n. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
On Sun, Jan 09, 2011 at 02:36:55PM +0100, Nikola Ciprich wrote: What are the differences between the host the completes chkdsk and the host that fails? Hardware-wise and software-wise. Software-wise, I'm not aware of any, both nodes use: - kernel 2.6.36.2 - qemu-kvm-0.13.0 - seabios-0.6.0 huh, I noticed now, it's not clean 0.13.0, but 0.13.0 + patches up to 420fe74769cc67baec6f3d962dc054e2972ca3ae. but it's the same on both machines, problematic and nonproblematic. hope it's not problem, sorry for the mess :( n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Please try the newer machine, after rmmoding kvm-intel, and reloading it with the module parameter ept=0. seems to be OK even with parameter.. n. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Ok. Is chkdsk outside the preboot environment working okay on the bad host? I'll check, give me some time, I'll have to add another large volume which I can check while the system is up.. in the meantime, on hosts, I noticed following messages: [322172.007569] kvm: 17743: cpu1 unhandled wrmsr: 0x198 data 0 it's on all (problematic and nonproblematic machines). dunno if this could be somehow related.. n. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Hi Avi, sorry it took so long. running chkdsk from windows is OK, runs pretty fast. so there must be something different in this preboot environment. n. On Sun, Jan 09, 2011 at 04:15:19PM +0100, Nikola Ciprich wrote: Ok. Is chkdsk outside the preboot environment working okay on the bad host? I'll check, give me some time, I'll have to add another large volume which I can check while the system is up.. in the meantime, on hosts, I noticed following messages: [322172.007569] kvm: 17743: cpu1 unhandled wrmsr: 0x198 data 0 it's on all (problematic and nonproblematic machines). dunno if this could be somehow related.. n. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Hi, I'd like to ask for advice with following problem. I have windows 2008 terminal server guest running on 2.6.36 x86_64 host (kvm 0.13.0). guest has 4GB of RAM, 40GB storage on top of LVM volume and two cores. So far everything was running fine, but during periodic maintenance I wanted to force chkdisk after reboot. So windows started checking disk integrity, but the problem is, that it's waaay too slow - after ~12 hours, it's still running and seeems like it'll take ages to finish. Both CPU cores seem to be fully loaded. Is there some way I could check why it's taking so long, and fix it eventually? can I use kvm_trace to achieve this task? how? I'll be very gratefull for any help... with best regards nik -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Hello Stefan! Please post your KVM command-line. /usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -name vmwts02 -uuid 1e501300-dc48-11df-a690-00304834195b -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/vmwts02.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=localtime -boot c -drive file=/dev/vgshared/vmwts02-1,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=22,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=00:16:3e:61:01:00,bus=pci.0,addr=0x3 -usb -vnc 0.0.0.0:30801 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 I see I could disable cache for storage, but I don't want to kill fsck now just to test if it helps (and I guess it shouldn't make such a difference). Have you run storage benchmarks on the host to check what sort of maximum I/O performance you can expect? Do you have a RAID setup underneath LVM? Not for windows, but in general it is running quite fast, only the chkdsk seems to be bad. In other VMs (linux), I'm achieving write speeds 40MB/s. Storage configuration is a bit comples, it's DRBD replicated storage, on top of it sits clustered LVM and KVMS use logical volumes on top of it. but as I said, overall performance is OK. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Hello Avi! On Thu, Jan 06, 2011 at 11:08:32AM +0200, Avi Kivity wrote: Let's start with a few 'kvm_stat -1' snapshots while this is going on. here it is, but note that there are few more machines running on this host. but they're almost idle in contrast to this windows one, so I hope it's not problem. kvm_ack_irq 299 290 kvm_age_page 7169 kvm_apic1126 1087 kvm_apic_accept_irq 453 438 kvm_apic_ipi 236 228 kvm_cpuid 0 0 kvm_cr 780 772 kvm_emulate_insn 46181 44982 kvm_entry 55012 53867 kvm_exit 55182 53884 kvm_exit(APIC_ACCESS) 3 0 kvm_exit(CPUID)3 0 kvm_exit(CR_ACCESS) 784 772 kvm_exit(DR_ACCESS)3 0 kvm_exit(EPT_MISCONFIG) 4 0 kvm_exit(EPT_VIOLATION) 4 0 kvm_exit(EXCEPTION_NMI) 50194 49039 kvm_exit(EXTERNAL_INTERRUPT) 2314 2213 kvm_exit(HLT)260 248 kvm_exit(INVALID_STATE) 3 0 kvm_exit(INVLPG) 10398 kvm_exit(IO_INSTRUCTION) 1538 1534 kvm_exit(MCE_DURING_VMENTRY) 5 0 kvm_exit(MONITOR_INSTRUCTION) 3 0 kvm_exit(MSR_READ) 5 0 kvm_exit(MSR_WRITE)4 0 kvm_exit(MWAIT_INSTRUCTION) 3 0 kvm_exit(NMI_WINDOW) 4 0 kvm_exit(PAUSE_INSTRUCTION) 4 0 kvm_exit(PENDING_INTERRUPT)12 7 kvm_exit(RDPMC)4 0 kvm_exit(RDTSC)4 0 kvm_exit(TASK_SWITCH) 3 0 kvm_exit(TPR_BELOW_THRESHOLD) 6 2 kvm_exit(TRIPLE_FAULT) 5 0 kvm_exit(VMCALL) 4 0 kvm_exit(VMCLEAR) 3 0 kvm_exit(VMLAUNCH) 3 0 kvm_exit(VMOFF)4 0 kvm_exit(VMON) 3 0 kvm_exit(VMPTRLD) 4 0 kvm_exit(VMPTRST) 4 0 kvm_exit(VMREAD) 5 0 kvm_exit(VMRESUME) 3 0 kvm_exit(VMWRITE) 4 0 kvm_exit(WBINVD) 3 0 kvm_exit(XSETBV) 4 0 kvm_fpu 247 243 kvm_hv_hypercall 0 0 kvm_hypercall 0 0 kvm_inj_exception 0 0 kvm_inj_virq 514 500 kvm_invlpga0 0 kvm_ioapic_set_irq 363 353 kvm_mmio 67725 65974 kvm_msi_set_irq0 0 kvm_msr0 0 kvm_nested_intercepts 0 0 kvm_nested_intr_vmexit 0 0 kvm_nested_vmexit 0 0 kvm_nested_vmexit_inject 0 0 kvm_nested_vmrun 0 0 kvm_page_fault 50200 48785 kvm_pic_set_irq 363 353 kvm_pio 1541 1541 kvm_set_irq 363 353 kvm_skinit 0 0 http://git.kernel.org/?p=virt/kvm/qemu-kvm.git;a=blob_plain;f=kvm/kvm_stat;hb=HEAD -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
- run trace-cmd record -e kvm -b 10 -P pid1 -P pid2, ctrl-C after a seems like it's not possible to specify multiple pids, so I've run 4 commands in parallel. Also I can't get monitor information since vm is started using libvirt, so I've just used all machine's qemu-kvm pids.. hope it's OK here's the trace: http://nelide.cz/downloads/nik/trace.tar.bz2 n. -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
Did you get 'overrun: something' reports from trace-cmd, where something != 0? nope, all entries were 0. Dan, is there a way to hijack the monitor so we can run some commands on it? Things like 'info registers' and disassembly. AFAIK that's intentionally not possible :( pity.. Looks like vcpu 1 is spinning; perhaps that's normal. If you get hold of the monitor, please disassemble around 0xf80001575d59. vcpu 0 is busy writing to vga (can you confirm)? looks like bank switching is hitting synchronize_srcu_expedited(), which is known slow. Unfortunately that only gets better in 2.6.38. I guess it might help if I would just risk killing the machine and run it without libvirt so we can debug better right? or even better, I'll try to reproduce on another 2K8 windows on testing machine and then we can play more with it. gimme a few minutes please.. You can try applying http://git.kernel.org/?p=linux/kernel/git/sfr/linux-next.git;a=commit;h=46fdb0937f26124700fc9fc80da4776330cc00d3 and see if it helps. I'll also prepare testing kernel including this patch.. BTW Is it just me, or is git.kernel.org pretty slow today? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
OK, got test environment running, but it seems to be running much faster there :( but as dan suggested, I can type monitor commands using virsh, so I can (carefully:)) continue debugging on this production machine.. here's info registers: RAX=0007 RBX=00ac RCX=f880009d1015 RDX=03ce RSI=018a RDI=f8000163f737 RBP=0007 RSP=f88002588b08 R8 =000f R9 =00ac R10=7b20 R11=0008 R12=00a8 R13= R14=0012c690 R15=001d52d0 RIP=f8000156ae48 RFL=0202 [---] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =002b 00c0f300 DPL=3 DS [-WA] CS =0010 00209b00 DPL=0 CS64 [-RA] SS =0018 00c09300 DPL=0 DS [-WA] DS =002b 00c0f300 DPL=3 DS [-WA] FS =0053 fffe 3c00 0040f300 DPL=3 DS [-WA] GS =002b f80001644d00 00c0f300 DPL=3 DS [-WA] LDT= TR =0040 f80002767080 0067 8b00 DPL=0 TSS64-busy GDT= f80002766000 007f IDT= f80002766080 0fff CR0=80050031 CR2=0047029a CR3=94925000 CR4=06f8 DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER=0d01 FCW=027f FSW= [ST=0] FTW=00 MXCSR=1f80 FPR0= FPR1= FPR2= FPR3= FPR4= FPR5= FPR6= FPR7= XMM00=00288c3000a000a0 XMM01= XMM02= XMM03= XMM04= XMM05= XMM06= XMM07= XMM08= XMM09= XMM10= XMM11= XMM12= XMM13= XMM14= XMM15= Looks like vcpu 1 is spinning; perhaps that's normal. If you get hold of the monitor, please disassemble around 0xf80001575d59. ouch, can You advice me on how do I do it? :-[ vcpu 0 is busy writing to vga (can you confirm)? looks like bank yes, seems like screen refreshing is quite slow, certainly in this rescue mode or what it is, it's not using any acceleration... switching is hitting synchronize_srcu_expedited(), which is known slow. Unfortunately that only gets better in 2.6.38. You can try applying http://git.kernel.org/?p=linux/kernel/git/sfr/linux-next.git;a=commit;h=46fdb0937f26124700fc9fc80da4776330cc00d3 I'll be able to test this only on testing machine, or on this production maybe overnight.. I'll prepare the kernel anyways.. and see if it helps. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
If you have a new enough libvirt, then you can also send commands directly using 'virsh qemu-monitor-command' (checking whether you need JSON or HMP syntax first - in this case you can see it needs HMP). Thanks Dan! didn't know this is possible, works pretty well! n. Regards, Daniel -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm-0.13.0 - winsows 2008 - chkdisk too slow
] 33.00 0.7% gen_rot_rm_T1/usr/bin/qemu-kvm 31.00 0.7% do_raw_spin_unlock [kernel.kallsyms] 30.00 0.6% __enqueue_rt_entity [kernel.kallsyms] 30.00 0.6% gen_shift_rm_T1 /usr/bin/qemu-kvm 29.00 0.6% load_balance [kernel.kallsyms] 29.00 0.6% sched_clock_cpu [kernel.kallsyms] 29.00 0.6% _raw_spin_unlock_irqrestore [kernel.kallsyms] 28.00 0.6% __might_sleep[kernel.kallsyms] 28.00 0.6% finish_task_switch [kernel.kallsyms] 28.00 0.6% pick_next_task_rt[kernel.kallsyms] 28.00 0.6% tick_nohz_restart_sched_tick [kernel.kallsyms] . . . I'm no longer sure this is the problem. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
