kvm_queue_exception - what is going wrong here?
Hi *, we recently have encountered kvm_queue_exception events in dmesg. What is causing these exceptions? It appears that the guest is at least suffering performance loss. Is the qemu-system-x86 hang event linked to the exception? Is there anything we can do to prevent these exceptions to happen? Best regards, Sebastian - [1811816.496620] set_cr3: #GP, reserved bits [1811816.496624] [ cut here ] [1811816.496640] WARNING: at /usr/src/linux-2.6.31/arch/x86/kvm/x86.c:202 kvm_queue_exception_e+0x61/0x70 [kvm ]() [1811816.496643] Hardware name: S5520HC [1811816.496644] Modules linked in: drbd tun ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv 4 xt_state nf_conntrack ipt_REJECT xt_tcpudp ppdev iptable_filter ip_tables lp parport x_tables kvm_intel kvm bridge stp bonding e100 via_rhine 3c59x 8139too mii shpchp iTCO_wdt iTCO_vendor_support pcspkr joydev ses encl osure usbhid bnx2 aacraid e1000e igb dca fbcon tileblit font bitblit softcursor [1811816.496669] Pid: 26170, comm: qemu-system-x86 Not tainted 2.6.31-14-server #48 [1811816.496672] Call Trace: [1811816.496682] [a0140651] ? kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496689] [8105f458] warn_slowpath_common+0x78/0xd0 [1811816.496692] [8105f4bf] warn_slowpath_null+0xf/0x20 [1811816.496701] [a0140651] kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496711] [a01406ff] load_guest_segment_descriptor+0x9f/0xb0 [kvm] [1811816.496721] [a0140753] kvm_load_segment_descriptor+0x43/0x110 [kvm] [1811816.496730] [a0140a20] ? kvm_inject_gp+0x10/0x20 [kvm] [1811816.496740] [a0140bed] ? kvm_set_cr3+0x15d/0x170 [kvm] [1811816.496749] [a0140d67] load_state_from_tss32+0x167/0x240 [kvm] [1811816.496758] [a0142a69] kvm_task_switch_32+0x119/0x130 [kvm] [1811816.496768] [a0142f93] kvm_task_switch+0x153/0x300 [kvm] [1811816.496777] [a0137df1] ? gfn_to_hva+0x11/0x90 [kvm] [1811816.496786] [a0138122] ? kvm_read_guest_page+0x62/0x70 [kvm] [1811816.496792] [a016cbab] handle_task_switch+0x6b/0x160 [kvm_intel] [1811816.496796] [a0169b45] vmx_handle_exit+0xf5/0x280 [kvm_intel] [1811816.496801] [a016d822] ? vmx_vcpu_run+0x292/0x856 [kvm_intel] [1811816.496811] [a013ddbf] vcpu_enter_guest+0x2cf/0x600 [kvm] [1811816.496814] [8107a12e] ? finish_wait+0x5e/0x80 [1811816.496819] [81541710] ? _spin_lock_irq+0x10/0x20 [1811816.496822] [81541603] ? __down_read+0xc3/0xce [1811816.496825] [81079fa0] ? autoremove_wake_function+0x0/0x40 [1811816.496834] [a013e153] __vcpu_run+0x63/0x330 [kvm] [1811816.496844] [a01440ab] kvm_arch_vcpu_ioctl_run+0x8b/0x1f0 [kvm] [1811816.496853] [a01367f2] kvm_vcpu_ioctl+0x2e2/0x630 [kvm] [1811816.496856] [8106e99f] ? dequeue_signal+0x9f/0x180 [1811816.496859] [81131d41] vfs_ioctl+0x31/0xa0 [1811816.496862] [811321b3] do_vfs_ioctl+0x373/0x400 [1811816.496864] [811322d9] sys_ioctl+0x99/0xa0 [1811816.496862] [811321b3] do_vfs_ioctl+0x373/0x400 [1811816.496864] [811322d9] sys_ioctl+0x99/0xa0 [1811816.496867] [8106d3e9] ? do_sigpending+0xa9/0xc0 [1811816.496870] [81012082] system_call_fastpath+0x16/0x1b [1811816.496872] ---[ end trace c6ea0da2c3c9d0e4 ]--- [1811816.496887] set_cr3: #GP, pdptrs reserved bits [1811816.496889] [ cut here ] [1811816.496898] WARNING: at /usr/src/linux-2.6.31/arch/x86/kvm/x86.c:202 kvm_queue_exception_e+0x61/0x70 [kvm]() [1811816.496900] Hardware name: S5520HC [1811816.496901] Modules linked in: drbd tun ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp ppdev iptable_filter ip_tables lp parport x_tables kvm_intel kvm bridge stp bonding e100 via_rhine 3c59x 8139too mii shpchp iTCO_wdt iTCO_vendor_support pcspkr joydev ses enclosure usbhid bnx2 aacraid e1000e igb dca fbcon tileblit font bitblit softcursor [1811816.496923] Pid: 26170, comm: qemu-system-x86 Tainted: GW 2.6.31-14-server #48 [1811816.496925] Call Trace: [1811816.496934] [a0140651] ? kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496937] [8105f458] warn_slowpath_common+0x78/0xd0 [1811816.496940] [8105f4bf] warn_slowpath_null+0xf/0x20 [1811816.496950] [a0140651] kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496959] [a01406ff] load_guest_segment_descriptor+0x9f/0xb0 [kvm] [1811816.496969] [a0140753] kvm_load_segment_descriptor+0x43/0x110 [kvm] [1811816.496978] [a0140b52] ? kvm_set_cr3+0xc2/0x170 [kvm] [1811816.496988] [a0140d9c] load_state_from_tss32+0x19c/0x240 [kvm] [1811816.496997] [a0142a69] kvm_task_switch_32+0x119/0x130 [kvm] [1811816.497006] [a0142f93] kvm_task_switch+0x153/0x300 [kvm] [1811816.497015] [a0137df1] ?
Re: kvm_queue_exception - what is going wrong here?
On Wed, Jun 16, 2010 at 12:05:10PM +0200, Sebastian Hetze wrote: Hi *, we recently have encountered kvm_queue_exception events in dmesg. What is causing these exceptions? It appears that the guest is at least suffering performance loss. Is the qemu-system-x86 hang event linked to the exception? Is there anything we can do to prevent these exceptions to happen? What is your guest? It look like it tries to switch task and tss it tries to switch to is broken, so task switch emulation code encounters multiple exception and it complains about it. Theoretically we should abort task switch after fist exception condition (set_cr3: #GP, reserved bits), but error handling in task switch emulation code is far from perfect especially in such old kernel like 2.6.31. Best regards, Sebastian - [1811816.496620] set_cr3: #GP, reserved bits [1811816.496624] [ cut here ] [1811816.496640] WARNING: at /usr/src/linux-2.6.31/arch/x86/kvm/x86.c:202 kvm_queue_exception_e+0x61/0x70 [kvm ]() [1811816.496643] Hardware name: S5520HC [1811816.496644] Modules linked in: drbd tun ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv 4 xt_state nf_conntrack ipt_REJECT xt_tcpudp ppdev iptable_filter ip_tables lp parport x_tables kvm_intel kvm bridge stp bonding e100 via_rhine 3c59x 8139too mii shpchp iTCO_wdt iTCO_vendor_support pcspkr joydev ses encl osure usbhid bnx2 aacraid e1000e igb dca fbcon tileblit font bitblit softcursor [1811816.496669] Pid: 26170, comm: qemu-system-x86 Not tainted 2.6.31-14-server #48 [1811816.496672] Call Trace: [1811816.496682] [a0140651] ? kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496689] [8105f458] warn_slowpath_common+0x78/0xd0 [1811816.496692] [8105f4bf] warn_slowpath_null+0xf/0x20 [1811816.496701] [a0140651] kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496711] [a01406ff] load_guest_segment_descriptor+0x9f/0xb0 [kvm] [1811816.496721] [a0140753] kvm_load_segment_descriptor+0x43/0x110 [kvm] [1811816.496730] [a0140a20] ? kvm_inject_gp+0x10/0x20 [kvm] [1811816.496740] [a0140bed] ? kvm_set_cr3+0x15d/0x170 [kvm] [1811816.496749] [a0140d67] load_state_from_tss32+0x167/0x240 [kvm] [1811816.496758] [a0142a69] kvm_task_switch_32+0x119/0x130 [kvm] [1811816.496768] [a0142f93] kvm_task_switch+0x153/0x300 [kvm] [1811816.496777] [a0137df1] ? gfn_to_hva+0x11/0x90 [kvm] [1811816.496786] [a0138122] ? kvm_read_guest_page+0x62/0x70 [kvm] [1811816.496792] [a016cbab] handle_task_switch+0x6b/0x160 [kvm_intel] [1811816.496796] [a0169b45] vmx_handle_exit+0xf5/0x280 [kvm_intel] [1811816.496801] [a016d822] ? vmx_vcpu_run+0x292/0x856 [kvm_intel] [1811816.496811] [a013ddbf] vcpu_enter_guest+0x2cf/0x600 [kvm] [1811816.496814] [8107a12e] ? finish_wait+0x5e/0x80 [1811816.496819] [81541710] ? _spin_lock_irq+0x10/0x20 [1811816.496822] [81541603] ? __down_read+0xc3/0xce [1811816.496825] [81079fa0] ? autoremove_wake_function+0x0/0x40 [1811816.496834] [a013e153] __vcpu_run+0x63/0x330 [kvm] [1811816.496844] [a01440ab] kvm_arch_vcpu_ioctl_run+0x8b/0x1f0 [kvm] [1811816.496853] [a01367f2] kvm_vcpu_ioctl+0x2e2/0x630 [kvm] [1811816.496856] [8106e99f] ? dequeue_signal+0x9f/0x180 [1811816.496859] [81131d41] vfs_ioctl+0x31/0xa0 [1811816.496862] [811321b3] do_vfs_ioctl+0x373/0x400 [1811816.496864] [811322d9] sys_ioctl+0x99/0xa0 [1811816.496862] [811321b3] do_vfs_ioctl+0x373/0x400 [1811816.496864] [811322d9] sys_ioctl+0x99/0xa0 [1811816.496867] [8106d3e9] ? do_sigpending+0xa9/0xc0 [1811816.496870] [81012082] system_call_fastpath+0x16/0x1b [1811816.496872] ---[ end trace c6ea0da2c3c9d0e4 ]--- [1811816.496887] set_cr3: #GP, pdptrs reserved bits [1811816.496889] [ cut here ] [1811816.496898] WARNING: at /usr/src/linux-2.6.31/arch/x86/kvm/x86.c:202 kvm_queue_exception_e+0x61/0x70 [kvm]() [1811816.496900] Hardware name: S5520HC [1811816.496901] Modules linked in: drbd tun ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp ppdev iptable_filter ip_tables lp parport x_tables kvm_intel kvm bridge stp bonding e100 via_rhine 3c59x 8139too mii shpchp iTCO_wdt iTCO_vendor_support pcspkr joydev ses enclosure usbhid bnx2 aacraid e1000e igb dca fbcon tileblit font bitblit softcursor [1811816.496923] Pid: 26170, comm: qemu-system-x86 Tainted: GW 2.6.31-14-server #48 [1811816.496925] Call Trace: [1811816.496934] [a0140651] ? kvm_queue_exception_e+0x61/0x70 [kvm] [1811816.496937] [8105f458] warn_slowpath_common+0x78/0xd0 [1811816.496940] [8105f4bf] warn_slowpath_null+0xf/0x20
