Re: [PATCH RESEND] KVM: nVMX: mask unrestricted_guest if disabled on L0
On Tue, Mar 17, 2015 at 02:02:32PM +0100, Radim Krčmář wrote:
If EPT was enabled, unrestricted_guest was allowed in L1 regardless of
L0. L1 triple faulted when running L2 guest that required emulation.
Another side effect was 'WARN_ON_ONCE(vmx-nested.nested_run_pending)'
in L0's dmesg:
WARNING: CPU: 0 PID: 0 at arch/x86/kvm/vmx.c:9190
nested_vmx_vmexit+0x96e/0xb00 [kvm_intel] ()
Prevent this scenario by masking SECONDARY_EXEC_UNRESTRICTED_GUEST when
the host doesn't have it enabled.
Fixes: 78051e3b7e35 (KVM: nVMX: Disable unrestricted mode if ept=0)
Cc: sta...@vger.kernel.org
Tested-By: Kashyap Chamarthy kcham...@redhat.com
Thanks Radim, for resending it. I was trying to find it in one of the
recent pull requests. :-)
Just for forerence, test confirmation:
http://article.gmane.org/gmane.comp.emulators.kvm.devel/133075
Complete thread related to this topic:
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/132863
--
/kashyap
Signed-off-by: Radim Krčmář rkrc...@redhat.com
---
The resend Cc'd stable, as recommended by Bandan, and added Tested-by.
arch/x86/kvm/vmx.c | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 10a481b7674d..ae4f6d35d19c 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2479,8 +2479,7 @@ static void nested_vmx_setup_ctls_msrs(struct vcpu_vmx
*vmx)
if (enable_ept) {
/* nested EPT: emulate EPT also to L1 */
vmx-nested.nested_vmx_secondary_ctls_high |=
- SECONDARY_EXEC_ENABLE_EPT |
- SECONDARY_EXEC_UNRESTRICTED_GUEST;
+ SECONDARY_EXEC_ENABLE_EPT;
vmx-nested.nested_vmx_ept_caps = VMX_EPT_PAGE_WALK_4_BIT |
VMX_EPTP_WB_BIT | VMX_EPT_2MB_PAGE_BIT |
VMX_EPT_INVEPT_BIT;
@@ -2494,6 +2493,10 @@ static void nested_vmx_setup_ctls_msrs(struct vcpu_vmx
*vmx)
} else
vmx-nested.nested_vmx_ept_caps = 0;
+ if (enable_unrestricted_guest)
+ vmx-nested.nested_vmx_secondary_ctls_high |=
+ SECONDARY_EXEC_UNRESTRICTED_GUEST;
+
/* miscellaneous data */
rdmsr(MSR_IA32_VMX_MISC,
vmx-nested.nested_vmx_misc_low,
--
2.3.2
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: nVMX: mask unrestricted_guest if disabled on L0
On Tue, Feb 24, 2015 at 05:30:06PM +0100, Radim Krčmář wrote:
2015-02-23 19:05+0100, Kashyap Chamarthy:
Tested with the _correct_ Kernel[1] (that has Radim's patch) now --
applied it on both L0 and L1.
Result: Same as before -- Booting L2 causes L1 to reboot. However, the
stack trace from `dmesg` on L0 is took slightly different path than
before -- it's using MSR handling:
Thanks, the problem was deeper ... L1 enabled unrestricted mode while L0
had it disabled. L1 could then vmrun a L2 state that L0 would have to
emulate, but that doesn't work. There are at least these solutions:
1) don't expose unrestricted_guest when L0 doesn't have it
2) fix unrestricted mode emulation code
3) handle the failure a without killing L1
I'd do just (1) -- emulating unrestricted mode is a loss.
I have done initial testing and at least qemu-sanity-check works now:
---8---
If EPT was enabled, unrestricted_guest was allowed in L1 regardless of
L0. L1 triple faulted when running L2 guest that required emulation.
Another side effect was 'WARN_ON_ONCE(vmx-nested.nested_run_pending)'
in L0's dmesg:
WARNING: CPU: 0 PID: 0 at arch/x86/kvm/vmx.c:9190
nested_vmx_vmexit+0x96e/0xb00 [kvm_intel] ()
Prevent this scenario by masking SECONDARY_EXEC_UNRESTRICTED_GUEST when
the host doesn't have it enabled.
Fixes: 78051e3b7e35 (KVM: nVMX: Disable unrestricted mode if ept=0)
Signed-off-by: Radim Krčmář rkrc...@redhat.com
I just built[1] a Kernel with this patch and tested it on L0 and L1 and
can confirm, the patch fixes the issue -- Booting L2 does not cause L1
to reboot.
So:
Tested-By: Kashyap Chamarthy kcham...@redhat.com
Thanks for investigating, Radim!
[1]
https://kashyapc.fedorapeople.org/kernel-4.0.0-0.rc1.git1.1.kashyap1.fc23-with-nvmx-fix2-radim/
---
arch/x86/kvm/vmx.c | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f7b20b417a3a..dbabea21357b 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2476,8 +2476,7 @@ static void nested_vmx_setup_ctls_msrs(struct vcpu_vmx
*vmx)
if (enable_ept) {
/* nested EPT: emulate EPT also to L1 */
vmx-nested.nested_vmx_secondary_ctls_high |=
- SECONDARY_EXEC_ENABLE_EPT |
- SECONDARY_EXEC_UNRESTRICTED_GUEST;
+ SECONDARY_EXEC_ENABLE_EPT;
vmx-nested.nested_vmx_ept_caps = VMX_EPT_PAGE_WALK_4_BIT |
VMX_EPTP_WB_BIT | VMX_EPT_2MB_PAGE_BIT |
VMX_EPT_INVEPT_BIT;
@@ -2491,6 +2490,10 @@ static void nested_vmx_setup_ctls_msrs(struct vcpu_vmx
*vmx)
} else
vmx-nested.nested_vmx_ept_caps = 0;
+ if (enable_unrestricted_guest)
+ vmx-nested.nested_vmx_secondary_ctls_high |=
+ SECONDARY_EXEC_UNRESTRICTED_GUEST;
+
/* miscellaneous data */
rdmsr(MSR_IA32_VMX_MISC,
vmx-nested.nested_vmx_misc_low,
--
/kashyap
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Mon, Feb 23, 2015 at 02:56:11PM +0100, Radim Krčmář wrote: 2015-02-22 16:46+0100, Kashyap Chamarthy: Radim, I just tested with your patch[1] in this thread. I built a Fedora Kernel[2] with it, and installed (and booted into) it on both L0 and L1. Result: I don't have good news, I'm afraid: L1 *still* reboots when an L2 guest is booted. And, L0 throws the stack trace that was previously noted on this thread: Thanks, I'm puzzled though ... isn't it possible that a wrong kernel sneaked into grub? Hmm, unlikely - I just double-confirmed that I'm running the same patched Kernel (3.20.0-0.rc0.git9.1.fc23.x86_64) on both L0 and L1. . . . [ 57.747345] [ cut here ] [0.004638] WARNING: CPU: 5 PID: 50206 at arch/x86/kvm/vmx.c:8962 nested_vmx_vmexit+0x7ee/0x880 [kvm_intel]() [0.060404] CPU: 5 PID: 50206 Comm: qemu-system-x86 Not tainted 3.18.7-200.fc21.x86_64 #1 This looks like a new backtrace, but the kernel is not [2]. Err, looks like I pasted the wrong one, but here it is again. I just tested with the patched Kernel (that I linked below) on both L0 and L1, the same behavior (L1 reboot on L2 boot) manifests: . . . [0.058440] CPU: 8 PID: 1828 Comm: qemu-system-x86 Not tainted 3.20.0-0.rc0.git9.1.fc23.x86_64 #1 [0.008856] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2 10/25/2012 [0.007475] 97b7f39b 883f5acc3bf8 818773cd [0.007477] 883f5acc3c38 810ab3ba [0.007495] 883f5acc3c68 887f62678000 [0.007489] Call Trace: [0.002455] [818773cd] dump_stack+0x4c/0x65 [0.005139] [810ab3ba] warn_slowpath_common+0x8a/0xc0 [0.006001] [810ab4ea] warn_slowpath_null+0x1a/0x20 [0.005831] [a220cf8e] nested_vmx_vmexit+0xbde/0xd30 [kvm_intel] [0.006957] [a220fda3] ? vmx_handle_exit+0x213/0xd80 [kvm_intel] [0.006956] [a220d3fa] vmx_queue_exception+0x10a/0x150 [kvm_intel] [0.007160] [a03c8cdb] kvm_arch_vcpu_ioctl_run+0x107b/0x1b60 [kvm] [0.007138] [a03c833a] ? kvm_arch_vcpu_ioctl_run+0x6da/0x1b60 [kvm] [0.007219] [8110725d] ? trace_hardirqs_on+0xd/0x10 [0.005837] [a03b0666] ? vcpu_load+0x26/0x70 [kvm] [0.005745] [8110385f] ? lock_release_holdtime.part.29+0xf/0x200 [0.006966] [a03c3a68] ? kvm_arch_vcpu_load+0x58/0x210 [kvm] [0.006618] [a03b0a73] kvm_vcpu_ioctl+0x383/0x7e0 [kvm] [0.006175] [81027b9d] ? native_sched_clock+0x2d/0xa0 [0.006000] [810d5c56] ? creds_are_invalid.part.1+0x16/0x50 [0.006518] [810d5cb1] ? creds_are_invalid+0x21/0x30 [0.005918] [813a77fa] ? inode_has_perm.isra.48+0x2a/0xa0 [0.006350] [8128c9a8] do_vfs_ioctl+0x2e8/0x530 [0.005514] [8128cc71] SyS_ioctl+0x81/0xa0 [0.005051] [81880969] system_call_fastpath+0x12/0x17 [0.005999] ---[ end trace 3e4dca7180cdddab ]--- [5.529564] kvm [1766]: vcpu0 unhandled rdmsr: 0x1c9 [0.005026] kvm [1766]: vcpu0 unhandled rdmsr: 0x1a6 [0.004998] kvm [1766]: vcpu0 unhandled rdmsr: 0x3f6 . . . [ +0.006055] [810992ea] warn_slowpath_null+0x1a/0x20 [ +0.005889] [a02f00ee] nested_vmx_vmexit+0x7ee/0x880 [kvm_intel] [ +0.007014] [a02f05af] ? vmx_handle_exit+0x1bf/0xaa0 [kvm_intel] [ +0.007015] [a02f039c] vmx_queue_exception+0xfc/0x150 [kvm_intel] [ +0.007130] [a028cdfd] kvm_arch_vcpu_ioctl_run+0xd9d/0x1290 [kvm] (There is only one execution path and unless there is a race, it would be prevented by [1].) [ +0.007111] [a0288528] ? kvm_arch_vcpu_load+0x58/0x220 [kvm] [ +0.006670] [a0274cbc] kvm_vcpu_ioctl+0x32c/0x5c0 [kvm] [...] [1] http://article.gmane.org/gmane.comp.emulators.kvm.devel/132937 [2] http://koji.fedoraproject.org/koji/taskinfo?taskID=9004708 -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Mon, Feb 23, 2015 at 05:14:37PM +0100, Kashyap Chamarthy wrote:
On Mon, Feb 23, 2015 at 02:56:11PM +0100, Radim Krčmář wrote:
2015-02-22 16:46+0100, Kashyap Chamarthy:
Radim,
I just tested with your patch[1] in this thread. I built a Fedora
Kernel[2] with it, and installed (and booted into) it on both L0 and L1.
Result: I don't have good news, I'm afraid: L1 *still* reboots when an
L2 guest is booted. And, L0 throws the stack trace that was
previously noted on this thread:
Thanks, I'm puzzled though ... isn't it possible that a wrong kernel
sneaked into grub?
Hmm, unlikely - I just double-confirmed that I'm running the same
patched Kernel (3.20.0-0.rc0.git9.1.fc23.x86_64) on both L0 and L1.
[Correcting myself here.]
Unfortunately, I was double-wrong and your guess is right -- I seemed to
have made _two_ Kernel builds (one doesn't contain your patch, and the
other) and now not sure _which_ one I used as I didn't add a custom tag.
To confuse more, I pointed the URL to wrong build (without your fix)
previously in this thread - so likely I must have used that in my last
test.
The correct build is here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=9006612
And, the build log does confirm the 'nvmx-fix.patch' that was applied
https://kojipkgs.fedoraproject.org//work/tasks/6612/9006612/build.log
The contents of the patch, I just generated a patch with `diff -u orig
new nvmx-fix.patch` forgetting that the Fedora Kernel handles git
formatted patches just fine.
$ cat nvmx-fix.patch
--- vmx.c.orig 2015-02-20 19:09:49.850841320 +0100
+++ vmx.c 2015-02-20 19:11:12.153491715 +0100
@@ -2038,6 +2038,9 @@
{
struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
+if (to_vmx(vcpu)-nested.nested_run_pending)
+return 0;
+
if (!(vmcs12-exception_bitmap (1u nr)))
return 0;
So, my conclusion was wrong and need to report back with the _proper_
Kernel build.
--
/kashyap
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
Tested with the _correct_ Kernel[1] (that has Radim's patch) now -- applied it on both L0 and L1. Result: Same as before -- Booting L2 causes L1 to reboot. However, the stack trace from `dmesg` on L0 is took slightly different path than before -- it's using MSR handling: . . . [Feb23 12:14] [ cut here ] [ +0.004658] WARNING: CPU: 5 PID: 1785 at arch/x86/kvm/vmx.c:9973 nested_vmx_vmexit+0xbde/0xd30 [kvm_intel]() [ +0.009897] Modules linked in: vhost_net vhost macvtap macvlan xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_i pv4 xt_conntrack nf_conntrack tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iTCO_wdt ipmi_devintf gpio_ich iTCO_vendor_support coretemp kvm_intel dcdbas kvm crc32c_in tel joydev ipmi_ssif serio_raw ipmi_si tpm_tis i7core_edac lpc_ich ipmi_msghandler edac_core tpm mfd_core shpchp wmi acpi_power_meter acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc mgag200 i2c_algo_bit drm_kms_helper ttm ata_generic drm pata_acpi megaraid_sas bnx2 [ +0.060790] CPU: 5 PID: 1785 Comm: qemu-system-x86 Not tainted 3.20.0-0.rc0.git9.1.fc23.x86_64 #1 [ +0.008938] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2 10/25/2012 [ +0.007476] 8ba15f99 88ff5d627b38 818773cd [ +0.007727] 88ff5d627b78 810ab3ba [ +0.007660] 88ff5d627b68 883f5fd2 [ +0.007729] Call Trace: [ +0.002543] [818773cd] dump_stack+0x4c/0x65 [ +0.005205] [810ab3ba] warn_slowpath_common+0x8a/0xc0 [ +0.006085] [810ab4ea] warn_slowpath_null+0x1a/0x20 [ +0.005915] [a0244f8e] nested_vmx_vmexit+0xbde/0xd30 [kvm_intel] [ +0.007061] [a0245976] vmx_set_msr+0x416/0x420 [kvm_intel] [ +0.006549] [a029f0c0] ? kvm_set_msr+0x70/0x70 [kvm] [ +0.006018] [a029f091] kvm_set_msr+0x41/0x70 [kvm] [ +0.005840] [a029f0f3] do_set_msr+0x33/0x50 [kvm] [ +0.005692] [a02a3a80] msr_io+0x100/0x1c0 [kvm] [ +0.005567] [a02a3a10] ? msr_io+0x90/0x1c0 [kvm] [ +0.005657] [a023de70] ? handle_task_switch+0x1f0/0x1f0 [kvm_intel] [ +0.007321] [a02ac799] kvm_arch_vcpu_ioctl+0xb79/0x11a0 [kvm] [ +0.006788] [a023f7fe] ? vmx_vcpu_load+0x15e/0x1e0 [kvm_intel] [ +0.006878] [a0298666] ? vcpu_load+0x26/0x70 [kvm] [ +0.005825] [a02abac3] ? kvm_arch_vcpu_load+0xb3/0x210 [kvm] [ +0.006712] [a02987da] kvm_vcpu_ioctl+0xea/0x7e0 [kvm] [ +0.006140] [81027b9d] ? native_sched_clock+0x2d/0xa0 [ +0.006063] [810d5c56] ? creds_are_invalid.part.1+0x16/0x50 [ +0.006583] [810d5cb1] ? creds_are_invalid+0x21/0x30 [ +0.005984] [813a77fa] ? inode_has_perm.isra.48+0x2a/0xa0 [ +0.006436] [8128c9a8] do_vfs_ioctl+0x2e8/0x530 [ +0.005559] [8128cc71] SyS_ioctl+0x81/0xa0 [ +0.005135] [81880969] system_call_fastpath+0x12/0x17 [ +0.006065] ---[ end trace a7f3bc31fb0ddbff ]--- . . . [1] https://kashyapc.fedorapeople.org/kernel-3.20.0-0.rc0.git9.1.fc23.rpms-with-nvmx-test-fix-from-radim/ - I uploaded the Fedora Koji scratch build for this Kernel to a more permanant location, as these type of builds will be removed automatically after 3 weeks -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
Radim, I just tested with your patch[1] in this thread. I built a Fedora Kernel[2] with it, and installed (and booted into) it on both L0 and L1. Result: I don't have good news, I'm afraid: L1 *still* reboots when an L2 guest is booted. And, L0 throws the stack trace that was previously noted on this thread: . . . [ 57.747345] [ cut here ] [0.004638] WARNING: CPU: 5 PID: 50206 at arch/x86/kvm/vmx.c:8962 nested_vmx_vmexit+0x7ee/0x880 [kvm_intel]() [0.009903] Modules linked in: vhost_net vhost macvtap macvlan xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defra g_ipv4 xt_conntrack nf_conntrack tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables kvm_intel coretemp iTCO_wdt kvm ipmi_devintf iTCO_vendor_support i7core_edac gpio_ich c rc32c_intel serio_raw edac_core ipmi_si dcdbas shpchp tpm_tis lpc_ich mfd_core tpm ipmi_msghandler wmi acpi_power_meter acpi_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc mgag200 i2c_algo_bit drm_kms_helper ttm drm ata_generic megaraid_sas bnx2 pata_acpi [last unloaded: kvm_intel] [0.060404] CPU: 5 PID: 50206 Comm: qemu-system-x86 Not tainted 3.18.7-200.fc21.x86_64 #1 [ +0.008220] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2 10/25/2012 [ +0.007526] a30d0ba3 883f2489fc48 8175e686 [ +0.007688] 883f2489fc88 810991d1 [ +0.007613] 883f2489fc98 88bece1ba000 0014 [ +0.007611] Call Trace: [ +0.002518] [8175e686] dump_stack+0x46/0x58 [ +0.005202] [810991d1] warn_slowpath_common+0x81/0xa0 [ +0.006055] [810992ea] warn_slowpath_null+0x1a/0x20 [ +0.005889] [a02f00ee] nested_vmx_vmexit+0x7ee/0x880 [kvm_intel] [ +0.007014] [a02f05af] ? vmx_handle_exit+0x1bf/0xaa0 [kvm_intel] [ +0.007015] [a02f039c] vmx_queue_exception+0xfc/0x150 [kvm_intel] [ +0.007130] [a028cdfd] kvm_arch_vcpu_ioctl_run+0xd9d/0x1290 [kvm] [ +0.007111] [a0288528] ? kvm_arch_vcpu_load+0x58/0x220 [kvm] [ +0.006670] [a0274cbc] kvm_vcpu_ioctl+0x32c/0x5c0 [kvm] [ +0.006236] [810d0f7b] ? put_prev_entity+0x5b/0x400 [ +0.005887] [810cbb37] ? set_next_entity+0x67/0x80 [ +0.005802] [810d4549] ? pick_next_task_fair+0x6c9/0x8c0 [ +0.006324] [810126d6] ? __switch_to+0x1d6/0x5f0 [ +0.005626] [8122a1c0] do_vfs_ioctl+0x2d0/0x4b0 [ +0.005543] [81760764] ? __schedule+0x2f4/0x8a0 [ +0.005537] [8122a421] SyS_ioctl+0x81/0xa0 [ +0.005106] [81765429] system_call_fastpath+0x12/0x17 [ +0.006056] ---[ end trace 646ed2360b84865c ]--- [ +7.000298] kvm [50179]: vcpu0 unhandled rdmsr: 0x1c9 [ +0.005061] kvm [50179]: vcpu0 unhandled rdmsr: 0x1a6 [ +0.005053] kvm [50179]: vcpu0 unhandled rdmsr: 0x3f6 . . . [1] http://article.gmane.org/gmane.comp.emulators.kvm.devel/132937 [2] http://koji.fedoraproject.org/koji/taskinfo?taskID=9004708 -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Fri, Feb 20, 2015 at 05:14:15PM +0100, Radim Krčmář wrote: 2015-02-19 23:28+0100, Kashyap Chamarthy: On Thu, Feb 19, 2015 at 10:10:11PM +0100, Kashyap Chamarthy wrote: On Thu, Feb 19, 2015 at 05:02:22PM +0100, Radim Krčmář wrote: [. . .] Then, I did another test: - Rebooted into Kernel 3.20.0-0.rc0.git5.1.fc23.x86_64 on physical host (L0). - In L1, checked out the KVM tree, applied your patch and built Kernel[*] from the current KVM tree and booted into the newly built one, here too, I'm thrown into a dracut shell Weird, but considering that boot fails on L0 as well, I think it that basing off a different commit could help ... What I missed to do was to build initramfs: $ cd /boot $ dracut initramfs-3.19.0+.img 3.19.0+ --force Then I can boot. However, networking was hosed due to this bug[1] in `dhclient` (Andrea Arcangeli said it's fixed for him in newest Kernels, but unfortunately it's still not fixed for me as I noted in the bug. Anyway, for the nVMX bug in question, I actually built a Fedora scratch Kernel build[2], with your fix, which was successful[3]. I will test with it once I get the networking fixed on the physical machine, hopefully, early next week. [*] Exactly, I built it this way: # Clone the tree $ git://git.kernel.org/pub/scm/virt/kvm/kvm.git # Make a new branch: $ git checkout -b nvmx_test $ git describe warning: tag 'for-linus' is really 'kvm-3.19-1' here for-linus-14459-g49776d5 Hm, it should say v3.19 -- does it stay the same if you do `git fetch git checkout origin/master`? If it still does, please try to apply it on top of `git checkout v3.18`. (The one that one failed too.) # Make a config file $ make defconfig It would be safer to copy the fedora config (from /boot) to .config and do `make olddefconfig`. That's actually what I did on my later compiles. For now, as noted above, will test with the Fedora Kernel scratch build I made. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1194809 -- `dhclient` crashes on boot [2] http://koji.fedoraproject.org/koji/taskinfo?taskID=9004708 [3] https://kojipkgs.fedoraproject.org//work/tasks/4708/9004708/build.log -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Thu, Feb 19, 2015 at 05:02:22PM +0100, Radim Krčmář wrote:
2015-02-19 16:01+0100, Radim Krčmář:
2015-02-19 13:07+0100, Kashyap Chamarthy:
5f3d5799974b8 KVM: nVMX: Rework event injection and recovery:
This concept is based on the rule that a pending vmlaunch/vmresume is
not canceled. Otherwise, we would risk to lose injected events or leak
them into the wrong queues. Encode this rule via a WARN_ON_ONCE at the
entry of nested_vmx_vmexit.
I wonder if we have broken the invariant since 3.9 ...
e011c663b9c786d115c0f45e5b0bfae0c39428d4
KVM: nVMX: Check all exceptions for intercept during delivery to L2
All exceptions should be checked for intercept during delivery to L2,
but we check only #PF currently. Drop nested_run_pending while we are
at it since exception cannot be injected during vmentry anyway.
The last sentence is not true.
Can you try if the following patch works?
Sure, will test a Kernel built with the below patch and report back.
Thanks for taking a look.
--
/kashyap
(I know little about nested, so it might be introducing another bug.)
Thanks.
---8---
KVM: nVMX: fix L2 to L1 interrupt leak
When vmx-nested.nested_run_pending is set, we aren't expected to exit
to L1, but nested_vmx_check_exception() could, since e011c663b9c7.
Prevent that.
Fixes: e011c663b9c7 (Check all exceptions for intercept during delivery to
L2)
Signed-off-by: Radim Krčmář rkrc...@redhat.com
---
arch/x86/kvm/vmx.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 3f73bfad0349..389166a1b79a 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2098,6 +2098,9 @@ static int nested_vmx_check_exception(struct kvm_vcpu
*vcpu, unsigned nr)
{
struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
+ if (to_vmx(vcpu)-nested.nested_run_pending)
+ return 0;
+
if (!(vmcs12-exception_bitmap (1u nr)))
return 0;
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Wed, Feb 18, 2015 at 05:42:37PM +0100, Paolo Bonzini wrote: On 17/02/2015 12:24, Kashyap Chamarthy wrote: Afraid, I didn't bisect it, but I just wanted to note that the above specific WARN was introduced in the above commit. I'm sure this Kernel (on L0) does not exhibit the problem: kernel-3.17.4-301.fc21.x86_64. But, if I had either of these two Kernels on the physical host, then the said problem manifests (L1 reboots): 3.19.0-1.fc22 or kernel-3.20.0-0.rc0.git5.1.fc23 Nested APICv is not part of 3.19, so it cannot be the culprit. Can you try 3.18? Just did two tests with 3.18: (1) Kernel 3.18 on L0 and 3.20 on L1 Result: Booting L2 guest causes L1 to reboot, and the same[*] stack trace on L0 (mentioned on this thread previously). But, annoyingly enough, when I did test (2) below, and then switched back to test (1), I don't notice the said stack trace in L0's `dmesg` however many times I boot an L2 guest. (2) Kernel 3.18 on both L0 and L1 Result: Booting L2 guest causes L1 to reboot, but *no* stack trace on L0 [*] Stack trace from test (1) . . . [ 4120.296552] [ cut here ] [ 4120.301190] WARNING: CPU: 6 PID: 1841 at arch/x86/kvm/vmx.c:8962 nested_vmx_vmexit+0x7ee/0x880 [kvm_intel]() [ 4120.311048] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack tun bridge stp llc ip6table_filter ip6_tables cfg80211 rfkill coretemp kvm_intel kvm iTCO_wdt gpio_ich iTCO_vendor_support joydev crc32c_intel lpc_ich ipmi_devintf ipmi_si tpm_tis shpchp i7core_edac dcdbas mfd_core tpm ipmi_msghandler serio_raw edac_core acpi_power_meter wmi acpi_cpufreq mgag200 i2c_algo_bit drm_kms_helper ttm drm megaraid_sas ata_generic bnx2 pata_acpi [ 4120.361643] CPU: 6 PID: 1841 Comm: qemu-system-x86 Not tainted 3.18.7-200.fc21.x86_64 #1 [ 4120.369757] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2 10/25/2012 [ 4120.377269] e947d406 88bf21f27c48 8175e686 [ 4120.384866] 88bf21f27c88 810991d1 [ 4120.392469] 88bf21f27c98 887f1f73e000 0014 [ 4120.400033] Call Trace: [ 4120.402533] [8175e686] dump_stack+0x46/0x58 [ 4120.407714] [810991d1] warn_slowpath_common+0x81/0xa0 [ 4120.413740] [810992ea] warn_slowpath_null+0x1a/0x20 [ 4120.419611] [a1cee0ee] nested_vmx_vmexit+0x7ee/0x880 [kvm_intel] [ 4120.426609] [a1cee5af] ? vmx_handle_exit+0x1bf/0xaa0 [kvm_intel] [ 4120.433585] [a1cee39c] vmx_queue_exception+0xfc/0x150 [kvm_intel] [ 4120.440697] [a0192dfd] kvm_arch_vcpu_ioctl_run+0xd9d/0x1290 [kvm] [ 4120.447783] [a018e528] ? kvm_arch_vcpu_load+0x58/0x220 [kvm] [ 4120.454436] [a017acbc] kvm_vcpu_ioctl+0x32c/0x5c0 [kvm] [ 4120.460650] [817634cd] ? down_read+0x1d/0x30 [ 4120.465915] [8122a1c0] do_vfs_ioctl+0x2d0/0x4b0 [ 4120.471431] [8122a421] SyS_ioctl+0x81/0xa0 [ 4120.476477] [81765429] system_call_fastpath+0x12/0x17 [ 4120.482533] ---[ end trace 5410644656637166 ]--- [ 4128.015867] kvm [1768]: vcpu0 unhandled rdmsr: 0x1c9 [ 4128.020849] kvm [1768]: vcpu0 unhandled rdmsr: 0x1a6 [ 4128.025848] kvm [1768]: vcpu0 unhandled rdmsr: 0x3f6 . . . -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Thu, Feb 19, 2015 at 10:10:11PM +0100, Kashyap Chamarthy wrote: On Thu, Feb 19, 2015 at 05:02:22PM +0100, Radim Krčmář wrote: [. . .] Can you try if the following patch works? Sure, will test a Kernel built with the below patch and report back. Hmm, I'm stuck with a meta issue. I checked out the KVM tree[1] on L0, applied your patch and built[*] the Kernel, and booted into it. Boot fails and drops into a dracut shell on because: . . . dracut-initqueue[3045]: Warning: Cancelling resume operation. Device not found. [ TIME ] Timed out waiting for device dev-ma...per910\x2d\x2d02\x2droot.device. [DEPEND] Dependency failed for /sysroot. [DEPEND] Dependency failed for Initrd Root File SyWarning: /dev/disk/by-uuid/4ccddb2d-4d63-4fce-b4d4-9b2f119a30cc does not exist . . . I saved the report from /run/initramfs/rdsosreport.txt here[2]. Then, I did another test: - Rebooted into Kernel 3.20.0-0.rc0.git5.1.fc23.x86_64 on physical host (L0). - In L1, checked out the KVM tree, applied your patch and built Kernel[*] from the current KVM tree and booted into the newly built one, here too, I'm thrown into a dracut shell [1] git://git.kernel.org/pub/scm/virt/kvm/kvm.git [2] https://kashyapc.fedorapeople.org/temp/kernel-boot-failure.txt [*] Exactly, I built it this way: # Clone the tree $ git://git.kernel.org/pub/scm/virt/kvm/kvm.git # Make a new branch: $ git checkout -b nvmx_test $ git describe warning: tag 'for-linus' is really 'kvm-3.19-1' here for-linus-14459-g49776d5 # Make a config file $ make defconfig # Compile $ make -j4 make bzImage make modules # Install $ sudo -i $ make modules_install make install -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Tue, Feb 17, 2015 at 07:07:21PM +0100, Jan Kiszka wrote: On 2015-02-17 19:00, Bandan Das wrote: Kashyap Chamarthy kcham...@redhat.com writes: .. Does enable_apicv make a difference? Actually, I did perform a test (on Paolo's suggestion on IRC) with enable_apicv=0 on physical host, and it didn't make any difference: $ cat /proc/cmdline BOOT_IMAGE=/vmlinuz-3.20.0-0.rc0.git5.1.fc23.x86_64 root=/dev/mapper/fedora--server_dell--per910--02-root ro console=ttyS1,115200n81 rd.lvm.lv=fedora-server_dell-per910-02/swap rd.lvm.lv=fedora-server_dell-per910-02/root LANG=en_US.UTF-8 enable_apicv=0 I am not sure if this works ? enable_apicv is a kvm_intel module parameter Good point. Has to be kvm_intel.enable_apicv=0 (if the module is built in). Hmm, yeah, I should have just added options kvm-intel enable_apicv=n (without quotes) to /etc/modprobe.d/dist.conf. I just rebooted the host with kvm_intel.enable_apicv=0 on Kernel command line: $ dmesg | grep apicv [0.00] Command line: BOOT_IMAGE=/vmlinuz-3.20.0-0.rc0.git5.1.fc23.x86_64 root=/dev/mapper/fedora--server_dell--per910--02-root ro console=ttyS1,115200n81 rd.lvm.lv=fedora-server_dell-per910-02/swap rd.lvm.lv=fedora-server_dell-per910-02/root LANG=en_US.UTF-8 kvm_intel.enable_apicv=0 [0.00] Kernel command line: BOOT_IMAGE=/vmlinuz-3.20.0-0.rc0.git5.1.fc23.x86_64 root=/dev/mapper/fedora--server_dell--per910--02-root ro console=ttyS1,115200n81 rd.lvm.lv=fedora-server_dell-per910-02/swap rd.lvm.lv=fedora-server_dell-per910-02/root LANG=en_US.UTF-8 kvm_intel.enable_apicv=0 $ cat /sys/module/kvm_intel/parameters/enable_apicv N Then, booted an L2 guest over L1's serial console, while observing host's `dmesg -w`, I can see the same traceback: . . . [ 918.327553] [ cut here ] [ 918.332196] WARNING: CPU: 13 PID: 2201 at arch/x86/kvm/vmx.c:9190 nested_vmx_vmexit+0x96e/0xb00 [kvm_intel]() [ 918.342162] Modules linked in: vhost_net vhost macvtap macvlan xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack tun bridge stp llc ip6table_filter ip6_tables cfg80211 rfkill iTCO_wdt ipmi_devintf iTCO_vendor_support gpio_ich dcdbas coretemp kvm_intel kvm crc32c_intel serio_raw ipmi_ssif ipmi_si tpm_tis tpm ipmi_msghandler acpi_power_meter i7core_edac lpc_ich edac_core acpi_cpufreq mfd_core shpchp wmi mgag200 i2c_algo_bit drm_kms_helper ttm drm ata_generic pata_acpi megaraid_sas bnx2 [ 918.396548] CPU: 13 PID: 2201 Comm: qemu-system-x86 Not tainted 3.20.0-0.rc0.git5.1.fc23.x86_64 #1 [ 918.405605] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2 10/25/2012 [ 918.413138] 651bc665 883f619ebc18 818760f7 [ 918.420790] 883f619ebc58 810ab80a [ 918.428336] 887f5b838000 883f3f8c8000 0014 [ 918.435865] Call Trace: [ 918.438390] [818760f7] dump_stack+0x4c/0x65 [ 918.443590] [810ab80a] warn_slowpath_common+0x8a/0xc0 [ 918.449596] [810ab93a] warn_slowpath_null+0x1a/0x20 [ 918.455494] [a100857e] nested_vmx_vmexit+0x96e/0xb00 [kvm_intel] [ 918.462455] [a100b5f7] ? vmx_handle_exit+0x1e7/0xcb2 [kvm_intel] [ 918.469444] [a0236972] ? kvm_arch_vcpu_ioctl_run+0x6d2/0x1b50 [kvm] [ 918.476731] [a100892a] vmx_queue_exception+0x10a/0x150 [kvm_intel] [ 918.483870] [a023730b] kvm_arch_vcpu_ioctl_run+0x106b/0x1b50 [kvm] [ 918.491078] [a0236972] ? kvm_arch_vcpu_ioctl_run+0x6d2/0x1b50 [kvm] [ 918.498302] [8110760d] ? trace_hardirqs_on+0xd/0x10 [ 918.504203] [a021edf6] ? vcpu_load+0x26/0x70 [kvm] [ 918.510012] [81103c0f] ? lock_release_holdtime.part.29+0xf/0x200 [ 918.516973] [a021f203] kvm_vcpu_ioctl+0x383/0x7e0 [kvm] [ 918.523157] [81027b9d] ? native_sched_clock+0x2d/0xa0 [ 918.529167] [810d5fc6] ? creds_are_invalid.part.1+0x16/0x50 [ 918.535701] [810d6021] ? creds_are_invalid+0x21/0x30 [ 918.541678] [813a61da] ? inode_has_perm.isra.48+0x2a/0xa0 [ 918.548097] [8128c7b8] do_vfs_ioctl+0x2e8/0x530 [ 918.553579] [8128ca81] SyS_ioctl+0x81/0xa0 [ 918.558635] [8187f8e9] system_call_fastpath+0x12/0x17 [ 918.564640] ---[ end trace b07d41c569219c46 ]--- [ 1092.389383 173.824743] kvm [2168]: vcpu0 unhandled rdmsr: 0x1c9 [ 1092.394374 0.004991] kvm [2168]: vcpu0 unhandled rdmsr: 0x1a6 [ 1092.399399 0.005025] kvm [2168]: vcpu0 unhandled rdmsr: 0x3f6 . . . -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
On Tue, Feb 17, 2015 at 07:02:14AM +0100, Jan Kiszka wrote: On 2015-02-16 21:40, Kashyap Chamarthy wrote: I can observe this only one of the Intel Xeon machines (which has 48 CPUs and 1TB memory), but very reliably reproducible. Reproducer: - Just ensure physical host (L0) and guest hypervisor (L1) are running 3.20.0-0.rc0.git5.1 Kernel (I used from Fedora's Rawhide). Preferably on an Intel Xeon machine - as that's where I could reproduce this issue, not on a Haswell machine - Boot an L2 guest: Run `qemu-sanity-check --accel=kvm` in L1 (or your own preferred method to boot an L2 KVM guest). - On a different terminal, which has serial console for L1: observe L1 reboot The only thing I notice in `demsg` (on L0) is this trace. _However_ this trace does not occur when an L1 reboot is triggered while you watch `dmesg -w` (to wait for new messages) as I boot an L2 guest -- which means, the below trace is not the root cause of L1 being rebooted. When the L2 gets rebooted, what you observe is just one of these messages vcpu0 unhandled rdmsr: 0x1a6 below . . . [Feb16 13:44] [ cut here ] [ +0.004632] WARNING: CPU: 4 PID: 1837 at arch/x86/kvm/vmx.c:9190 nested_vmx_vmexit+0x96e/0xb00 [kvm_intel]() [ +0.009835] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack tun bridge stp llc ip6table_filter ip6_tables cfg80211 rfkill iTCO_wdt iTCO_vendor_support ipmi_devintf gpio_ich dcdbas coretemp kvm_intel kvm crc32c_intel ipmi_ssif serio_raw acpi_power_meter ipmi_si tpm_tis ipmi_msghandler tpm lpc_ich i7core_edac mfd_core edac_core acpi_cpufreq shpchp wmi mgag200 i2c_algo_bit drm_kms_helper ttm ata_generic drm pata_acpi megaraid_sas bnx2 [ +0.050289] CPU: 4 PID: 1837 Comm: qemu-system-x86 Not tainted 3.20.0-0.rc0.git5.1.fc23.x86_64 #1 [ +0.008902] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2 10/25/2012 [ +0.007469] ee6c0c54 88bf60bf7c18 818760f7 [ +0.007542] 88bf60bf7c58 810ab80a [ +0.007519] 88ff625b8000 883f55f9b000 0014 [ +0.007489] Call Trace: [ +0.002471] [818760f7] dump_stack+0x4c/0x65 [ +0.005152] [810ab80a] warn_slowpath_common+0x8a/0xc0 [ +0.006020] [810ab93a] warn_slowpath_null+0x1a/0x20 [ +0.005851] [a130957e] nested_vmx_vmexit+0x96e/0xb00 [kvm_intel] [ +0.006974] [a130c5f7] ? vmx_handle_exit+0x1e7/0xcb2 [kvm_intel] [ +0.006999] [a02ca972] ? kvm_arch_vcpu_ioctl_run+0x6d2/0x1b50 [kvm] [ +0.007239] [a130992a] vmx_queue_exception+0x10a/0x150 [kvm_intel] [ +0.007136] [a02cb30b] kvm_arch_vcpu_ioctl_run+0x106b/0x1b50 [kvm] [ +0.007162] [a02ca972] ? kvm_arch_vcpu_ioctl_run+0x6d2/0x1b50 [kvm] [ +0.007241] [8110760d] ? trace_hardirqs_on+0xd/0x10 [ +0.005864] [a02b2df6] ? vcpu_load+0x26/0x70 [kvm] [ +0.005761] [81103c0f] ? lock_release_holdtime.part.29+0xf/0x200 [ +0.006979] [a02c5f88] ? kvm_arch_vcpu_load+0x58/0x210 [kvm] [ +0.006634] [a02b3203] kvm_vcpu_ioctl+0x383/0x7e0 [kvm] [ +0.006197] [81027b9d] ? native_sched_clock+0x2d/0xa0 [ +0.006026] [810d5fc6] ? creds_are_invalid.part.1+0x16/0x50 [ +0.006537] [810d6021] ? creds_are_invalid+0x21/0x30 [ +0.005930] [813a61da] ? inode_has_perm.isra.48+0x2a/0xa0 [ +0.006365] [8128c7b8] do_vfs_ioctl+0x2e8/0x530 [ +0.005496] [8128ca81] SyS_ioctl+0x81/0xa0 [ +0.005065] [8187f8e9] system_call_fastpath+0x12/0x17 [ +0.006014] ---[ end trace 2f24e0820b44f686 ]--- [ +5.870886] kvm [1783]: vcpu0 unhandled rdmsr: 0x1c9 [ +0.004991] kvm [1783]: vcpu0 unhandled rdmsr: 0x1a6 [ +0.005020] kvm [1783]: vcpu0 unhandled rdmsr: 0x3f6 [Feb16 14:18] kvm [1783]: vcpu0 unhandled rdmsr: 0x1c9 [ +0.005020] kvm [1783]: vcpu0 unhandled rdmsr: 0x1a6 [ +0.004998] kvm [1783]: vcpu0 unhandled rdmsr: 0x3f6 . . . Version --- Exact below versions were used on L0 and L1: $ uname -r; rpm -q qemu-system-x86 3.20.0-0.rc0.git5.1.fc23.x86_64 qemu-system-x86-2.2.0-5.fc22.x86_64 Other info -- - Unpacking the kernel-3.20.0-0.rc0.git5.1.fc23.src.rpm and looking at this file, arch/x86/kvm/vmx.c, line 9190 is below, with contextual code: [. . .] 9178 * Emulate an exit from nested guest (L2) to L1, i.e., prepare to run L1 9179 * and modify vmcs12 to make it see what it would expect to see there if 9180 * L2 was its real guest. Must only be called when in L2 (is_guest_mode()) 9181 */ 9182 static void
Re: nSVM: Booting L2 results in L1 hang and a skip_emulated_instruction
On Thu, Feb 12, 2015 at 07:12:06AM +0100, Jan Kiszka wrote: On 2015-02-11 19:12, Kashyap Chamarthy wrote: Hi, This was tested with kernel-3.19.0-1.fc22) and QEMU (qemu-2.2.0-5.fc22) on L0 L1. Description --- Inside L1, boot a nested KVM guest (L2) . Instead of a full blown guest, let's use `qemu-sanity-check` with KVM: $ qemu-sanity-check --accel=kvm Wwich gives you this CLI (run from a different shell), that confirms that the L2 guest is indeed running on KVM (and not TCG): $ ps -ef | grep -i qemu root 763 762 35 11:49 ttyS000:00:00 qemu-system-x86_64 -nographic -nodefconfig -nodefaults -machine accel=kvm -no-reboot -serial file:/tmp/tmp.rl3naPaCkZ.out -kernel /boot/vmlinuz-3.19.0-1.fc21.x86_64 -initrd /usr/lib64/qemu-sanity-check/initrd -append console=ttyS0 oops=panic panic=-1 Which results in: (a) L1 (guest hypervisor) completely hangs and is unresponsive. But when I query libvirt, (`virsh list`) the guest is still reported as 'running' (b) On L0, I notice a ton of these messages: skip_emulated_instruction: ip 0xffec next 0x8105e964 I can get `dmesg`, `dmidecode` , `x86info -a` on L0 and L1 if it helps in narrowing down the issue. Related bug and reproducer details -- https://bugzilla.redhat.com/show_bug.cgi?id=1191665 -- Nested KVM with AMD: L2 (nested guest) fails with divide error: [#1] SMP Is this a regression (of the kernel)? If so, can you bisect to the commit that introduced it? [Sorry, I missed this reply and just noticed it.] I can't certainly say that it's a regression of Kernel. I also heard from Dan Berrange (CCed), that when an booting L2 guest it caused L1 to panic and reboot on AMD. I don't have the AMD physical machine that I tested this on, will try to find one this week and see if I can bisect. Thanks. -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[nVMX] With 3.20.0-0.rc0.git5.1 on L0, booting L2 guest results in L1 *rebooting*
I can observe this only one of the Intel Xeon machines (which has 48
CPUs and 1TB memory), but very reliably reproducible.
Reproducer:
- Just ensure physical host (L0) and guest hypervisor (L1) are running
3.20.0-0.rc0.git5.1 Kernel (I used from Fedora's Rawhide).
Preferably on an Intel Xeon machine - as that's where I could
reproduce this issue, not on a Haswell machine
- Boot an L2 guest: Run `qemu-sanity-check --accel=kvm` in L1 (or
your own preferred method to boot an L2 KVM guest).
- On a different terminal, which has serial console for L1: observe L1
reboot
The only thing I notice in `demsg` (on L0) is this trace. _However_ this
trace does not occur when an L1 reboot is triggered while you watch
`dmesg -w` (to wait for new messages) as I boot an L2 guest -- which
means, the below trace is not the root cause of L1 being rebooted. When
the L2 gets rebooted, what you observe is just one of these messages
vcpu0 unhandled rdmsr: 0x1a6 below
. . .
[Feb16 13:44] [ cut here ]
[ +0.004632] WARNING: CPU: 4 PID: 1837 at arch/x86/kvm/vmx.c:9190
nested_vmx_vmexit+0x96e/0xb00 [kvm_intel]()
[ +0.009835] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE
nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4
nf_defrag_ipv4 xt_conntrack nf_conntrack tun bridge stp llc ip6table_filter
ip6_tables cfg80211 rfkill iTCO_wdt iTCO_vendor_support ipmi_devintf gpio_ich
dcdbas coretemp kvm_intel kvm crc32c_intel ipmi_ssif serio_raw acpi_power_meter
ipmi_si tpm_tis ipmi_msghandler tpm lpc_ich i7core_edac mfd_core edac_core
acpi_cpufreq shpchp wmi mgag200 i2c_algo_bit drm_kms_helper ttm ata_generic drm
pata_acpi megaraid_sas bnx2
[ +0.050289] CPU: 4 PID: 1837 Comm: qemu-system-x86 Not tainted
3.20.0-0.rc0.git5.1.fc23.x86_64 #1
[ +0.008902] Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.8.2
10/25/2012
[ +0.007469] ee6c0c54 88bf60bf7c18
818760f7
[ +0.007542] 88bf60bf7c58
810ab80a
[ +0.007519] 88ff625b8000 883f55f9b000
0014
[ +0.007489] Call Trace:
[ +0.002471] [818760f7] dump_stack+0x4c/0x65
[ +0.005152] [810ab80a] warn_slowpath_common+0x8a/0xc0
[ +0.006020] [810ab93a] warn_slowpath_null+0x1a/0x20
[ +0.005851] [a130957e] nested_vmx_vmexit+0x96e/0xb00 [kvm_intel]
[ +0.006974] [a130c5f7] ? vmx_handle_exit+0x1e7/0xcb2 [kvm_intel]
[ +0.006999] [a02ca972] ? kvm_arch_vcpu_ioctl_run+0x6d2/0x1b50 [kvm]
[ +0.007239] [a130992a] vmx_queue_exception+0x10a/0x150 [kvm_intel]
[ +0.007136] [a02cb30b] kvm_arch_vcpu_ioctl_run+0x106b/0x1b50 [kvm]
[ +0.007162] [a02ca972] ? kvm_arch_vcpu_ioctl_run+0x6d2/0x1b50 [kvm]
[ +0.007241] [8110760d] ? trace_hardirqs_on+0xd/0x10
[ +0.005864] [a02b2df6] ? vcpu_load+0x26/0x70 [kvm]
[ +0.005761] [81103c0f] ? lock_release_holdtime.part.29+0xf/0x200
[ +0.006979] [a02c5f88] ? kvm_arch_vcpu_load+0x58/0x210 [kvm]
[ +0.006634] [a02b3203] kvm_vcpu_ioctl+0x383/0x7e0 [kvm]
[ +0.006197] [81027b9d] ? native_sched_clock+0x2d/0xa0
[ +0.006026] [810d5fc6] ? creds_are_invalid.part.1+0x16/0x50
[ +0.006537] [810d6021] ? creds_are_invalid+0x21/0x30
[ +0.005930] [813a61da] ? inode_has_perm.isra.48+0x2a/0xa0
[ +0.006365] [8128c7b8] do_vfs_ioctl+0x2e8/0x530
[ +0.005496] [8128ca81] SyS_ioctl+0x81/0xa0
[ +0.005065] [8187f8e9] system_call_fastpath+0x12/0x17
[ +0.006014] ---[ end trace 2f24e0820b44f686 ]---
[ +5.870886] kvm [1783]: vcpu0 unhandled rdmsr: 0x1c9
[ +0.004991] kvm [1783]: vcpu0 unhandled rdmsr: 0x1a6
[ +0.005020] kvm [1783]: vcpu0 unhandled rdmsr: 0x3f6
[Feb16 14:18] kvm [1783]: vcpu0 unhandled rdmsr: 0x1c9
[ +0.005020] kvm [1783]: vcpu0 unhandled rdmsr: 0x1a6
[ +0.004998] kvm [1783]: vcpu0 unhandled rdmsr: 0x3f6
. . .
Version
---
Exact below versions were used on L0 and L1:
$ uname -r; rpm -q qemu-system-x86
3.20.0-0.rc0.git5.1.fc23.x86_64
qemu-system-x86-2.2.0-5.fc22.x86_64
Other info
--
- Unpacking the kernel-3.20.0-0.rc0.git5.1.fc23.src.rpm and looking at
this file, arch/x86/kvm/vmx.c, line 9190 is below, with contextual
code:
[. . .]
9178 * Emulate an exit from nested guest (L2) to L1, i.e., prepare to run L1
9179 * and modify vmcs12 to make it see what it would expect to see there if
9180 * L2 was its real guest. Must only be called when in L2
(is_guest_mode())
9181 */
9182 static void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason,
9183 u32 exit_intr_info,
9184 unsigned long exit_qualification)
9185 {
9186 struct vcpu_vmx *vmx = to_vmx(vcpu);
9187 struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
9188
9189 /* trying
nSVM: Booting L2 results in L1 hang and a skip_emulated_instruction
Hi, This was tested with kernel-3.19.0-1.fc22) and QEMU (qemu-2.2.0-5.fc22) on L0 L1. Description --- Inside L1, boot a nested KVM guest (L2) . Instead of a full blown guest, let's use `qemu-sanity-check` with KVM: $ qemu-sanity-check --accel=kvm Wwich gives you this CLI (run from a different shell), that confirms that the L2 guest is indeed running on KVM (and not TCG): $ ps -ef | grep -i qemu root 763 762 35 11:49 ttyS000:00:00 qemu-system-x86_64 -nographic -nodefconfig -nodefaults -machine accel=kvm -no-reboot -serial file:/tmp/tmp.rl3naPaCkZ.out -kernel /boot/vmlinuz-3.19.0-1.fc21.x86_64 -initrd /usr/lib64/qemu-sanity-check/initrd -append console=ttyS0 oops=panic panic=-1 Which results in: (a) L1 (guest hypervisor) completely hangs and is unresponsive. But when I query libvirt, (`virsh list`) the guest is still reported as 'running' (b) On L0, I notice a ton of these messages: skip_emulated_instruction: ip 0xffec next 0x8105e964 I can get `dmesg`, `dmidecode` , `x86info -a` on L0 and L1 if it helps in narrowing down the issue. Related bug and reproducer details -- https://bugzilla.redhat.com/show_bug.cgi?id=1191665 -- Nested KVM with AMD: L2 (nested guest) fails with divide error: [#1] SMP -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [question] incremental backup a running vm
On Wed, Jan 21, 2015 at 11:39:44AM +0100, Paolo Bonzini wrote:
On 21/01/2015 11:32, Zhang Haoyu wrote:
Hi,
Does drive_mirror support incremental backup a running vm?
Or other mechanism does?
incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to
destination,
then a copied bitmap will be saved to a file, then the bitmap file will log
dirty for
the changed data.
Next time backup, only the dirty data will be mirrored to destination.
Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Drive-mirror is for storage migration. For backup there is another job,
drive-backup. drive-backup copies a point-in-time snapshot of one or
more disks corresponding to when the backup was started.
Zhang, I've been testing the `drive-backup` command via QMP for a little
while, and it works reasonably well. If you'd like to test it you can
quickly try as below, once you have a QEMU instance runing with QMP
(I invoke my QEMU instances with '-qmp tcp:localhost:,server').
The below script invokes the 'drive-backup' QMP command (Ensure you're
using the correct disk, your disk ID might be 'drive-virtio-disk1' :-) )
-
#!/bin/bash
set -x
exec 3/dev/tcp/localhost/
echo -e { 'execute': 'qmp_capabilities' } 3
read response 3
echo $response
echo -e { 'execute': 'drive-backup', 'arguments':
{ 'device': 'drive-virtio-disk0', 'sync': 'full', 'target':
'/export/backup-of-vm1.qcow2', 'mode': 'absolute-paths', 'format':
'qcow2' } } 3
read response 3
echo $response
-
Once the above is invoked succesfully, you can see the success of the
command on the shell where your QMP server is running:
-
[. . .]
char device redirected to /dev/pts/8 (label charserial0)
QEMU waiting for connection on: disconnected:tcp:localhost:,server
Formatting '/export/backup-of-vm1.qcow2', fmt=qcow2 size=53687091200
encryption=off cluster_size=65536 lazy_refcounts=off
-
Incremental backup is being worked on. You can see patches on the list.
Paolo
--
/kashyap
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [question] incremental backup a running vm
On Wed, Jan 21, 2015 at 11:39:44AM +0100, Paolo Bonzini wrote:
On 21/01/2015 11:32, Zhang Haoyu wrote:
Hi,
Does drive_mirror support incremental backup a running vm?
Or other mechanism does?
incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to
destination,
then a copied bitmap will be saved to a file, then the bitmap file will log
dirty for
the changed data.
Next time backup, only the dirty data will be mirrored to destination.
Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Drive-mirror is for storage migration. For backup there is another job,
drive-backup. drive-backup copies a point-in-time snapshot of one or
more disks corresponding to when the backup was started.
Zhang, I've been testing the `drive-backup` command via QMP for a little
while, and it works reasonably well. If you'd like to test it you can
quickly try as below, once you have a QEMU instance runing with QMP
(I invoke my QEMU instances with '-qmp tcp:localhost:,server').
The below script invokes the 'drive-backup' QMP command (Ensure you're
using the correct disk, your disk ID might be 'drive-virtio-disk1' :-) )
-
#!/bin/bash
set -x
exec 3/dev/tcp/localhost/
echo -e { 'execute': 'qmp_capabilities' } 3
read response 3
echo $response
echo -e { 'execute': 'drive-backup', 'arguments':
{ 'device': 'drive-virtio-disk0', 'sync': 'full', 'target':
'/export/backup-of-vm1.qcow2', 'mode': 'absolute-paths', 'format':
'qcow2' } } 3
read response 3
echo $response
-
Once the above is invoked succesfully, you can see the success of the
command on the shell where your QMP server is running:
-
[. . .]
char device redirected to /dev/pts/8 (label charserial0)
QEMU waiting for connection on: disconnected:tcp:localhost:,server
Formatting '/export/backup-of-vm1.qcow2', fmt=qcow2 size=53687091200
encryption=off cluster_size=65536 lazy_refcounts=off
-
Incremental backup is being worked on. You can see patches on the list.
--
/kashyap
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
With '-cpu host' in L1 nested guest fails: KVM: entry failed, hardware error 0x7
Hi, This error occurs when using using '-cpu host'. And, is consistently reproducible with Fedora 21's Kernels. One way to reproduce the issue -- a) Enable nested virt on host, I use this procedure[1] b) Boot a a guest (ensure /dev/kvm character device shows up) c) Invoke `libguestfs-test-tool` utility in L1 (which will run a QEMU appliance with -cpu host). You should see the failure in this log: ~/.cache/libvirt/qemu/log/guestfs-vegb4aor6a8d1r3e.log QEMU CLI: [. . .] 2014-11-26 12:16:02.449+: starting up LC_ALL=C PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/kashyapc/.local/bin:/home/kashyapc/bin:/usr/local/sbin:/usr/sbin:/sbin HOME=/home/kashyapc USER=kashyapc LOGNAME=kashyapc QEMU_AUDIO_DRV=none TMPDIR=/var/tmp /bin/qemu-kvm -name guestfs-vegb4aor6a8d1r3e -S -machine pc-i440fx-2.1,accel=kvm,usb=off -cpu host -m 500 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 27869b93-0b83-4b57-a909-9265c677d307 -nographic -no-user-config -nodefaults -device sga -chardev socket,id=charmonitor,path=/home/kashyapc/.config/libvirt/qemu/lib/guestfs-vegb4aor6a8d1r3e.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-reboot -no-acpi -boot strict=on -kernel /var/tmp/.guestfs-1000/appliance.d/kernel -initrd /var/tmp/.guestfs-1000/appliance.d/initrd -append panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 TERM=screen -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/tmp/libguestfsenp1an/devnull1,if=none,id=drive-scsi0-0-0-0,format=raw,cache=writeback -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/tmp/libguestfsenp1an/overlay2,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 -chardev socket,id=charserial0,path=/tmp/libguestfsenp1an/console.sock -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/tmp/libguestfsenp1an/guestfsd.sock -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on Domain id=2 is tainted: custom-argv Domain id=2 is tainted: host-cpu KVM: entry failed, hardware error 0x7 RAX=000f RBX= RCX=038f RDX= RSI=000f RDI=038f RBP=88001ee4faa0 RSP=88001ee4faa0 R8 = R9 =88001f00cf60 R10=88001ee4fa48 R11=0005 R12=038f R13=81809390 R14= R15=88001f00ca40 RIP=8105b8ca RFL=0046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES = 00c0 CS =0010 00a09b00 DPL=0 CS64 [-RA] SS = 00c0 DS = 00c0 FS = 00c0 GS = 88001f00 00c0 LDT= 00c0 TR =0040 88001f012080 2087 8b00 DPL=0 TSS64-busy GDT= 88001f00a000 007f IDT= ff56b000 0fff CR0=8005003b CR2= CR3=01c11000 CR4=07f0 DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER=0d01 Code=20 48 89 f8 48 09 f0 5d c3 90 55 89 f0 89 f9 48 89 e5 0f 30 31 c0 5d c3 66 90 55 89 f9 48 89 e5 0f 33 48 89 d7 89 c1 48 c1 e7 20 48 89 f8 48 09 c8 5d [. . .] I'll admit, with Fedora Rawhide Kernel, libvirt and QEMU versions I could not reproduce the issue: $ uname -r; rpm -q qemu-system-x86 libvirt-daemon-kvm 3.18.0-0.rc6.git0.1.fc22.x86_64 qemu-system-x86-2.2.0-0.1.rc1.fc22.x86_64 libvirt-daemon-kvm-1.2.10-3.fc22.x86_64 There's an existing bug for Fedora, here[2] [1] https://kashyapc.fedorapeople.org/virt/procedure-to-enable-nested-virt-on-intel-machines.txt [2] https://bugzilla.redhat.com/show_bug.cgi?id=1016748#c9 -- KVM: entry failed, hardware error 0x7 for nested kvm guest -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Howto connect to a terminal in an emalated linux-livecd?
On Fri, Sep 12, 2014 at 01:43:18PM +0100, Stefan Hajnoczi wrote: On Thu, Sep 11, 2014 at 01:48:51PM +0200, Oliver Rath wrote: after hours for searching in google-world, i didnt find any appropriate for this problem: I want to boot a live-cd (i.e. ubuntu 14.04.1-desktop) in qemu, which starts with an graphical interface, done i.e. by qemu-system-x86_64 -m 3G -smp 2 -drive file=ubuntu-14.04.1-desktop-i386.iso,media=cdrom,if=virtio --enable-kvm Now i want to access to the console of the ubuntu-livecd. At the moment i can do this over changing to text mode via sendkey ctrl-alt-f1 in qemu-console (Alt-2), then switching back to qemu-window (alt-1). Now i have access to tty1 of my livecd. But IMHO there should be a more simple way to access to such a console with qemu, i.e. through a pipe, a serial console etc., but i didnt found anything working. The best i got was with -chardev pty,id=myid, which resulted in a char device redirected to /dev/pts/0 (label myid). But with a screen /dev/pts/0 i wasnt able to see any input or output. ssh is unfortunatly not available at this time on the livecd (so i could connect i.e. via -net user,hostfwd:tcp:10022-:22) Any hints to connect directly to a console in an emulated linux? I use the serial console: $ qemu-system-x86_64 -serial stdio ... Make sure the guest has console=ttyS0 on the kernel command-line. Just to add a little more to what Stefan wrote, here's a working CLI (not be the most optimal) I use w/ serial console: $ /usr/bin/qemu-system-x86_64 -m 2048 \ -nographic -nodefconfig -nodefaults \ -machine accel=kvm -m 2048 \ -drive file=./snap1-f20vm.qcow2,if=ide,format=qcow2 \ -serial stdio And, a little more info here[1] [1] http://rwmj.wordpress.com/2011/07/08/setting-up-a-serial-console-in-qemu-and-libvirt/ -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Where to file this bug, please?
On Sat, Mar 22, 2014 at 08:07:42AM +, Prof. Dr. Michael Schefczyk wrote: Dear Recipients, As per http://www.linux-kvm.org/page/Bugs, I am seeking advice regarding where to file the following bug: If you're experiencing it on CentOS http://bugs.centos.org/ If you're experiencing it on RHEL, you could file it (a clear reproducer will increase the likelihood of developer attention) in Red Hat bugzilla, under Product Red Hat Enterprise Linux 6, Component qemu-kvm: https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%206 -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Is there any consumer of virtio-balloon now?
On Fri, Mar 21, 2014 at 12:22:54PM +0800, Kai Huang wrote: Hi, I see the virtio-balloon is designed for memory auto balloon between KVM host and guest, but from latest linux kernel mainline code, looks currently there's no consumer actually using it? Would you let me know who is the consumer if there's any? From a quick look up, I see at-least there are some users[1] on libvirt-users list. And, here's a nice explanation/usage of it[2] [1] https://www.redhat.com/archives/libvirt-users/2011-October/msg00059.html [2] http://rwmj.wordpress.com/2010/07/17/virtio-balloon/ -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [fedora-virt] 3.13 - Nested KVM (vmx) totally broken?
On Tue, Mar 04, 2014 at 09:13:40AM +0100, Paolo Bonzini wrote: Il 04/03/2014 03:40, Ian Pilcher ha scritto: Is this a known problem? I just tried using nested vmx for the first time since upgrading my system from F19 (3.12.?? at the time) to F20, and I cannot start any L2 guests. The L2 guest appears to hang almost immediately after starting, consuming 100% of one of the L1 guest's VCPUs. If I reboot with kernel-3.12.10-300.fc20.x86_64, the problem does not occur. Any known workaround? (Other than using 3.12.10?) If you want to try, I made a Fedora Kernel scratch build (i.e. not official) with fix Paolo pointed to below and this works for me: http://koji.fedoraproject.org/koji/taskinfo?taskID=6577700 (NOTE: Fedora Scratch build URLs won't last more than 10 days or so) There is a fix on the way to the 3.13 kernel. You can open a Fedora bug and ask them to include http://article.gmane.org/gmane.linux.kernel.stable/82043/raw in the kernel. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [fedora-virt] 3.13 - Nested KVM (vmx) totally broken?
On Tue, Mar 04, 2014 at 03:00:22PM +0530, Kashyap Chamarthy wrote: On Tue, Mar 04, 2014 at 09:13:40AM +0100, Paolo Bonzini wrote: Il 04/03/2014 03:40, Ian Pilcher ha scritto: Is this a known problem? I just tried using nested vmx for the first time since upgrading my system from F19 (3.12.?? at the time) to F20, and I cannot start any L2 guests. The L2 guest appears to hang almost immediately after starting, consuming 100% of one of the L1 guest's VCPUs. If I reboot with kernel-3.12.10-300.fc20.x86_64, the problem does not occur. Err, I missed to read this. Sorry about that. Any known workaround? (Other than using 3.12.10?) If you want to try, I made a Fedora Kernel scratch build (i.e. not official) with fix Paolo pointed to below and this works for me: http://koji.fedoraproject.org/koji/taskinfo?taskID=6577700 (NOTE: Fedora Scratch build URLs won't last more than 10 days or so) There is a fix on the way to the 3.13 kernel. You can open a Fedora bug and ask them to include http://article.gmane.org/gmane.linux.kernel.stable/82043/raw in the kernel. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- /kashyap -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: nVMX regression v3.13+, bisected
On Thu, Feb 27, 2014 at 02:14:23AM +0530, Kashyap Chamarthy wrote: On Wed, Feb 26, 2014 at 09:27:17PM +0100, Stefan Bader wrote: On 26.02.2014 21:25, Paolo Bonzini wrote: [. . .] I bisected this and ended up on the following commit which, when reverted made the launch work again: Author: Anthoine Bourgeois bourge...@bertin.fr Date: Wed Nov 13 11:45:37 2013 +0100 kvm, vmx: Fix lazy FPU on nested guest If a nested guest does a NM fault but its CR0 doesn't contain the TS flag (because it was already cleared by the guest with L1 aid) then we have to activate FPU ourselves in L0 and then continue to L2. If TS flag is set then we fallback on the previous behavior, forward the fault to L1 if it asked for. Signed-off-by: Anthoine Bourgeois bourge...@bertin.fr Signed-off-by: Paolo Bonzini pbonz...@redhat.com The condition to exit to L0 seems to be according to what the description says. Could it be that the handling in L0 is doing something wrong? Thanks, I'll look at it tomorrow or Friday. Paolo Great thanks. And maybe it helps if I actually add the link to the bug report as I had intended... :-P [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1278531 Yes, I'm seeing something similar[*] in a consistent manner with minimal Fedora installs on L0, L1 and L2 Ok, I just tried to debug an L2 guest (a libguestfs appliance) via gdb following this method[1]. This is how far I got: From shell on L1, launch the libguestfs appliance (note: here libguestfs is compiled with gdb debugging enabled, so QEMU won't start running the appliance): $ ./run libguestfs-test-tool [. . .] checking modpath /lib/modules/3.14.0-0.rc2.git0.1.fc21.x86_64 is a directory picked kernel vmlinuz-3.14.0-0.rc2.git0.1.fc21.x86_64 supermin helper [0ms] finished creating kernel [. . .] libguestfs: warning: qemu debugging is enabled, connect gdb to tcp::1234 to begin [. . .] From a different shell, I invoke gdb like that: (gdb) symbol-file /usr/lib/debug/lib/modules/3.14.0-0.rc4.git0.1.fc21.x86_64/vmlinux Reading symbols from /usr/lib/debug/lib/modules/3.14.0-0.rc4.git0.1.fc21.x86_64/vmlinux...done. (gdb) target remote tcp::1234 Remote debugging using tcp::1234 0xfff0 in ftrace_stack () (gdb) bt #0 0x0997 in irq_stack_union () #1 0x in ?? () (gdb) (gdb) c Continuing. Again, back to libguestfs-test-tool, it's just hung attempting to booting from ROM: [. . .] SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mockbuild@) Wed Aug 14 23:57:08 UTC 2013 Term: 80x24 4 0 SeaBIOS (version 1.7.4-20140106_154858-) Booting from ROM... Back to gdb, to find out _what_ file the above function is trying to be executed from: (gdb) c Continuing. ^C Program received signal SIGINT, Interrupt. 0x0997 in irq_stack_union () (gdb) bt #0 0x0997 in irq_stack_union () #1 0x in ?? () (gdb) list 1 /* 2* Copyright 2002, 2003 Andi Kleen, SuSE Labs. 3* 4* This file is subject to the terms and conditions of the GNU General Public 5* License. See the file COPYING in the main directory of this archive 6* for more details. No warranty for anything given at all. 7*/ 8 #include linux/linkage.h 9 #include asm/dwarf2.h 10 #include asm/errno.h (gdb) [. . .] (gdb) 241 ENDPROC(csum_partial_copy_generic) (gdb) Line number 242 out of range; arch/x86/lib/csum-copy_64.S has 241 lines. (gdb) PS: Paolo, I'll try to test with your new patch soon. Thanks. [1] https://github.com/libguestfs/libguestfs/blob/master/src/launch-direct.c#L404 [*] https://bugzilla.kernel.org/show_bug.cgi?id=69491#c7 -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: nVMX regression v3.13+, bisected
On Thu, Feb 27, 2014 at 05:40:56PM +0530, Kashyap Chamarthy wrote: On Thu, Feb 27, 2014 at 02:14:23AM +0530, Kashyap Chamarthy wrote: On Wed, Feb 26, 2014 at 09:27:17PM +0100, Stefan Bader wrote: On 26.02.2014 21:25, Paolo Bonzini wrote: appliance): $ ./run libguestfs-test-tool [. . .] checking modpath /lib/modules/3.14.0-0.rc2.git0.1.fc21.x86_64 is a directory picked kernel vmlinuz-3.14.0-0.rc2.git0.1.fc21.x86_64 supermin helper [0ms] finished creating kernel [. . .] libguestfs: warning: qemu debugging is enabled, connect gdb to tcp::1234 to begin [. . .] From a different shell, I invoke gdb like that: (gdb) symbol-file /usr/lib/debug/lib/modules/3.14.0-0.rc4.git0.1.fc21.x86_64/vmlinux Disregard me here. I loaded wrong symbol file (Thanks to David Gilbert for spotting that on IRC) :-( Just issued a Fedora Kernel test build[1] with Paolo's patch, will see how it goes. [1] http://koji.fedoraproject.org/koji/taskinfo?taskID=6577135 -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: nVMX regression v3.13+, bisected
On Thu, Feb 27, 2014 at 05:58:46PM +0100, Paolo Bonzini wrote: Il 27/02/2014 18:01, anthoine.bourge...@bertin.fr ha scritto: OK, so your patch works perfectly well with both of my test machines (a Ubuntu guest or a ChorusOS guest). I join the patch, can you signof it ? I'll post it tonight or tomorrow, thanks. I tested two cases: 1. Built a Fedora Kernel[a] and installed on both L0 and L1, booted an L2 guest successfully[b]. - I tried two things as L2 guest: (1) Run 'libguestfs-test-tool' that boots a minimal Kernel/initrd via 'libvirt' backend; (2) A regular Fedora-20 guest with QEMU command-line produced by libvirt. 2. Build KVM git with Paolo's patch, install the Kernel on L0 and L1; (and reboot both): $ git log | head -1 commit 404381c5839d67aa0c275ad1da96ef3d3928ca2c In this case, booting an L1 (guest hypervisor) itself results in the below stack trace for me: [. . .] [0.039000] task: 880216078000 ti: 880216056000 task.ti: 880216056000 [0.039000] RIP: 0010:[81ceab25] [81ceab25] intel_pmu_init+0x2d9/0x8e6 [0.039000] RSP: :880216057e40 EFLAGS: 0202 [0.039000] RAX: 0003 RBX: RCX: 0345 [0.039000] RDX: 0003 RSI: 0730 RDI: [0.039000] RBP: 880216057e48 R08: 0001 R09: 0007 [0.039000] R10: 81cc0c60 R11: 880216057c16 R12: 81ce9a4f [0.039000] R13: R14: R15: [0.039000] FS: () GS:88021fc0() knlGS: [0.039000] CS: 0010 DS: ES: CR0: 80050033 [0.039000] CR2: 88021000 CR3: 01c0b000 CR4: 001406f0 [0.039000] Stack: [0.039000] 880216057e98 81ce9a88 [0.039000] 8809b000 880216057e98 81ce9a4f [0.039000] 880216057f08 [0.039000] Call Trace: [0.039000] [81ce9a88] init_hw_perf_events+0x39/0x51a [0.039000] [81ce9a4f] ? check_bugs+0x2d/0x2d [0.039000] [81000332] do_one_initcall+0xf2/0x140 [0.039000] [81cef22f] ? native_smp_prepare_cpus+0x30c/0x32a [0.039000] [81ce2f19] kernel_init_freeable+0xc4/0x1ec [0.039000] [817aa230] ? rest_init+0x80/0x80 [0.039000] [817aa239] kernel_init+0x9/0xf0 [0.039000] [817c307c] ret_from_fork+0x7c/0xb0 [0.039000] [817aa230] ? rest_init+0x80/0x80 [0.039000] Code: 61 fd ff 44 89 0d e4 61 fd ff 89 0d 9e 62 fd ff 7e 2b 83 e2 1f b8 03 00 00 00 b9 45 03 00 00 83 fa 02 0f 4f c2 89 05 ab 61 fd ff 0f 32 48 c1 e2 20 89 c0 48 09 c2 48 89 15 49 62 fd ff e8 64 10 [0.039000] RIP [81ceab25] intel_pmu_init+0x2d9/0x8e6 [0.039000] RSP 880216057e40 [0.039013] ---[ end trace 6a1e6dd839222f3e ]--- [0.040007] swapper/0 (1) used greatest stack depth: 5720 bytes left [0.041008] Kernel panic - not syncing: Attempted to kill init! exitcode=0x000b Thanks Paolo, for the fix. [a] http://koji.fedoraproject.org/koji/taskinfo?taskID=6577700 [b] http://kashyapc.fedorapeople.org/temp/stdout-libguestfs-test-tool-in-L1-28FEB2014.txt -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: nVMX regression v3.13+, bisected
On Wed, Feb 26, 2014 at 09:27:17PM +0100, Stefan Bader wrote: On 26.02.2014 21:25, Paolo Bonzini wrote: [. . .] I bisected this and ended up on the following commit which, when reverted made the launch work again: Author: Anthoine Bourgeois bourge...@bertin.fr Date: Wed Nov 13 11:45:37 2013 +0100 kvm, vmx: Fix lazy FPU on nested guest If a nested guest does a NM fault but its CR0 doesn't contain the TS flag (because it was already cleared by the guest with L1 aid) then we have to activate FPU ourselves in L0 and then continue to L2. If TS flag is set then we fallback on the previous behavior, forward the fault to L1 if it asked for. Signed-off-by: Anthoine Bourgeois bourge...@bertin.fr Signed-off-by: Paolo Bonzini pbonz...@redhat.com The condition to exit to L0 seems to be according to what the description says. Could it be that the handling in L0 is doing something wrong? Thanks, I'll look at it tomorrow or Friday. Paolo Great thanks. And maybe it helps if I actually add the link to the bug report as I had intended... :-P [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1278531 Yes, I'm seeing something similar[*] in a consistent manner with minimal Fedora installs on L0, L1 and L2, but couldn't manage time to do the bisecting. I thought this would be my first bisecting exercise, but you already beat me to it. [*] https://bugzilla.kernel.org/show_bug.cgi?id=69491#c7 -- /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: nested EPT
On Fri, Jan 17, 2014 at 2:51 AM, duy hai nguyen hain...@gmail.com wrote:
Now I can run an L2 guest (nested guest) using the kvm kernel module
of kernel 3.12
However, I am facing a new problem when trying to build and use kvm
kernel module from git://git.kiszka.org/kvm-kmod.git: L1 (nested
hypervisor) cannot boot L2 and the graphic console of virt-manager
hangs displaying 'Booting from Hard Disk...'. L1 still runs fine.
Loading kvm_intel with 'emulate_invalid_guest_state=0' in L0 does not
solve the problem. I have also tried with different kernel versions:
3.12.0, 3.12.8 and 3.13.0 without success.
Can you give me some suggestions?
Maybe you can try without graphical managers and enable serial console
('console=ttyS0') to your Kernel command-line of L2 guest, so you can
see where it's stuck.
/kashyap
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
On Tue, Oct 8, 2013 at 8:18 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 08/10/2013 15:03, Paolo Bonzini ha scritto: Il 08/10/2013 07:38, Kashyap Chamarthy ha scritto: On Mon, Oct 7, 2013 at 6:29 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Gleb, so I just did a trace of KVM MMU to try to understand why L2 is stuck with shadow on EPT Paolo, were you able to reproduce this again? Yesterday, on #qemu you mentioned you'll test it again :-) Yes, I could reproduce it too. Boot L2 guest: Here L2 doesn't go past the second instruction. It gets a page fault even though the spte is present, and KVM then loops on a page fault for 0xfe05b. Here is an annotated function_graph trace of L1. It's possible that L0 is injecting the same fault repeatedly, i.e. they are not different faults from the processor. I'll get an L0 trace next. The L0 trace is not particularly helpful (and probably would not be particularly helpful even if there were a specific tracepoint for VMREAD): 287.534156: kvm_exit: reason VMRESUME rip 0xa021f8d1 info 0 0 287.534160: kvm_mmu_get_page: sp gfn 0 0/4 q0 direct --- !pge !nxe root 0sync 287.534161: kvm_entry:vcpu 0 287.534162: kvm_exit: reason EXCEPTION_NMI rip 0xe05b info fe05b 8b0e 287.534170: kvm_mmu_get_page: sp gfn 0 0/4 q0 direct --- !pge !nxe root 0sync 287.534171: kvm_entry:vcpu 0 287.534172: kvm_exit: reason VMREAD rip 0xa021f97d info 0 0 287.534173: kvm_entry:vcpu 0 287.534174: kvm_exit: reason VMREAD rip 0xa021f996 info 0 0 287.534174: kvm_entry:vcpu 0 287.534175: kvm_exit: reason VMREAD rip 0xa021f9b5 info 0 0 287.534175: kvm_entry:vcpu 0 287.534177: kvm_exit: reason VMREAD rip 0xa021b377 info 0 0 287.534177: kvm_entry:vcpu 0 287.534178: kvm_exit: reason VMREAD rip 0xa021b5ce info 0 0 287.534179: kvm_entry:vcpu 0 287.534180: kvm_exit: reason VMREAD rip 0xa0222c95 info 0 0 287.534180: kvm_entry:vcpu 0 287.534181: kvm_exit: reason VMREAD rip 0xa0222e1c info 0 0 287.534182: kvm_entry:vcpu 0 287.534185: kvm_exit: reason MSR_READ rip 0x8104c2b6 info 0 0 287.534185: kvm_msr: msr_read 1d9 = 0x0 287.534185: kvm_entry:vcpu 0 And then it repeats: 287.534186: kvm_exit: reason VMRESUME rip 0xa021f8d1 info 0 0 287.534191: kvm_mmu_get_page: sp gfn 0 0/4 q0 direct --- !pge !nxe root 0sync 287.534192: kvm_entry:vcpu 0 Trying to add function_graph loses a lot of events. Paolo Paolo, here's L0's trace with shadow-on-EPT. I invoked with the below command line on L0 $ /usr/bin/trace-cmd record -b 1 -e kvmmmu then, started the L2 guest. Report: $ trace-cmd report 21 | less The trace data file was large (150MB), just pasting the repeating message here: - . . . qemu-system-x86-2006 [000] 57931.272466: kvm_mmu_pagetable_walk: addr 813d8404 pferr 10 F qemu-system-x86-2006 [000] 57931.272468: kvm_mmu_paging_element: pte 1c0f067 level 4 qemu-system-x86-2006 [000] 57931.272469: kvm_mmu_paging_element: pte 1c10063 level 3 qemu-system-x86-2006 [000] 57931.272469: kvm_mmu_paging_element: pte 12001e1 level 2 qemu-system-x86-2006 [000] 57931.272497: kvm_mmu_pagetable_walk: addr 813d8404 pferr 10 F qemu-system-x86-2006 [000] 57931.272497: kvm_mmu_paging_element: pte 1c0f067 level 4 qemu-system-x86-2006 [000] 57931.272497: kvm_mmu_paging_element: pte 1c10063 level 3 qemu-system-x86-2006 [000] 57931.272497: kvm_mmu_paging_element: pte 12001e1 level 2 qemu-system-x86-2006 [000] 57931.272516: kvm_mmu_pagetable_walk: addr 813d8404 pferr 10 F qemu-system-x86-2006 [000] 57931.272516: kvm_mmu_paging_element: pte 1c0f067 level 4 qemu-system-x86-2006 [000] 57931.272516: kvm_mmu_paging_element: pte 1c10063 level 3 qemu-system-x86-2006 [000] 57931.272517: kvm_mmu_paging_element: pte 12001e1 level 2 qemu-system-x86-2006 [000] 57931.272525: kvm_mmu_pagetable_walk: addr 813d8404 pferr 10 F qemu-system-x86-2006 [000] 57931.272525: kvm_mmu_paging_element: pte 1c0f067 level 4 qemu-system-x86-2006 [000] 57931.272525: kvm_mmu_paging_element: pte 1c10063 level 3 qemu-system-x86-2006 [000] 57931.272525: kvm_mmu_paging_element: pte 12001e1 level 2 qemu-system-x86-2006 [000] 57931.272540: kvm_mmu_pagetable_walk: addr 813d8404 pferr 10 F qemu-system-x86-2006 [000] 57931.272540: kvm_mmu_paging_element: pte 1c0f067 level 4 qemu-system-x86-2006 [000] 57931.272540: kvm_mmu_paging_element: pte 1c10063 level 3 qemu-system-x86-2006 [000] 57931.272541: kvm_mmu_paging_element: pte 12001e1 level 2 qemu-system-x86-2006 [000] 57931.272580: kvm_mmu_pagetable_walk: addr
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
On Wed, Oct 9, 2013 at 1:46 PM, Gleb Natapov g...@redhat.com wrote: On Wed, Oct 09, 2013 at 11:52:29AM +0530, Kashyap Chamarthy wrote: On Tue, Oct 8, 2013 at 8:18 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 08/10/2013 15:03, Paolo Bonzini ha scritto: Il 08/10/2013 07:38, Kashyap Chamarthy ha scritto: On Mon, Oct 7, 2013 at 6:29 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Gleb, so I just did a trace of KVM MMU to try to understand why L2 is stuck with shadow on EPT Paolo, were you able to reproduce this again? Yesterday, on #qemu you mentioned you'll test it again :-) Yes, I could reproduce it too. 72f857950f6f19cba42a9ded078bbc99f10aa667 causes it and I ran kernel without this commit. Will look into it. Thank you, this fixed it. I tested by applying your patch from the other email.[1] I'm now able to boot L2 guest with shadow-on-EPT with the above fix. Thank you. /kashyap [1] http://www.mail-archive.com/kvm@vger.kernel.org/msg96484.html -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Arbitrary L0 crash with kvm.git queue (8a3c1a33476f6bfebd07954e2277dbc88003bd37)
I've seen this crash at-least twice on an Intel Haswell machine. I can't pin-point to an exact trigger. As of now, it crashed when I tried to reboot a linux guest. Thought I'll post the trace here, just in case someone wants to take a look. I compiled kernel from kvm.git queue on both L0 and L1: $ git describe for-linus-12017-g8a3c1a33 $ git log | head -1 commit 8a3c1a33476f6bfebd07954e2277dbc88003bd37 Crash: (Found via serial console) = [177558.023358] BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:2:6155] [177558.030845] Modules linked in: vhost_net vhost macvtap macvlan kvm_intel kvm ebtable_nat xt_CHECKSUM tun nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables bridge stp llc snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_page_alloc iwlwifi snd_timer snd cfg80211 e1000e iTCO_wdt iTCO_vendor_support mei_me lpc_ich mfd_core soundcore ptp mei pps_core serio_raw pcspkr rfkill i2c_i801 binfmt_misc i915 i2c_algo_bit drm_kms_helper drm i2c_core video [177558.107179] CPU: 0 PID: 6155 Comm: kworker/0:2 Tainted: G D W 3.12.0-rc2+ #7 [177558.116045] Hardware name: Intel Corporation Shark Bay Client platform/Flathead Creek Crb, BIOS HSWLPTU1.86C.0131.R03.1307262359 07/26/2013 [177558.130323] Workqueue: events jump_label_update_timeout [177558.136344] task: 8804379d5dc0 ti: 88007c8a8000 task.ti: 88007c8a8000 [177558.144913] RIP: 0010:[810cdf5e] [810cdf5e] smp_call_function_many+0x25e/0x2c0 [177558.155263] RSP: 0018:88007c8a9c68 EFLAGS: 0202 [177558.161374] RAX: 0003 RBX: RCX: 88044e2d73e8 [177558.169550] RDX: 0003 RSI: 0080 RDI: [177558.177726] RBP: 88007c8a9cc8 R08: 88044e214c88 R09: 0002 [177558.185902] R10: 88044e214c88 R11: 1f17 R12: dd50 [177558.194077] R13: 00fc R14: 0286 R15: 0286 [177558.202254] FS: () GS:88044e20() knlGS: [177558.211514] CS: 0010 DS: ES: CR0: 80050033 [177558.218115] CR2: 7fd4757f6000 CR3: 01c0c000 CR4: 001427e0 [177558.226291] DR0: DR1: DR2: [177558.234467] DR3: DR6: fffe0ff0 DR7: 0400 [177558.242642] Stack: [177558.245012] 88044e214c98 000181050e73 00014c40 [177558.253524] 81018be0 88044e2d4c40 0202 a040e705 [177558.262032] 81018be0 a040e706 a042d560 [177558.270542] Call Trace: [177558.273406] [81018be0] ? setup_data_read+0x90/0x90 [177558.280014] [a040e705] ? apic_has_pending_timer+0x15/0x90 [kvm] [177558.287895] [81018be0] ? setup_data_read+0x90/0x90 [177558.294501] [a040e706] ? apic_has_pending_timer+0x16/0x90 [kvm] [177558.302384] [810ce0fd] on_each_cpu+0x2d/0x60 [177558.308398] [a040e705] ? apic_has_pending_timer+0x15/0x90 [kvm] [177558.316280] [8101938b] text_poke_bp+0x5b/0xc0 [177558.322393] [a040e705] ? apic_has_pending_timer+0x15/0x90 [kvm] [177558.330274] [81016912] __jump_label_transform.isra.0+0x112/0x140 [177558.338255] [81016977] arch_jump_label_transform+0x37/0x50 [177558.345646] [8113c2bf] __jump_label_update+0x5f/0x80 [177558.352446] [8113c37d] jump_label_update+0x9d/0xb0 [177558.359049] [8113c58d] __static_key_slow_dec+0x4d/0xb0 [177558.366044] [8113c646] jump_label_update_timeout+0x16/0x20 [177558.373436] [81082185] process_one_work+0x175/0x430 [177558.380137] [81082dab] worker_thread+0x11b/0x3a0 [177558.386544] [81082c90] ? rescuer_thread+0x340/0x340 [177558.393245] [810897a0] kthread+0xc0/0xd0 [177558.398866] [810896e0] ? insert_kthread_work+0x40/0x40 [177558.405862] [816662bc] ret_from_fork+0x7c/0xb0 [177558.412070] [810896e0] ? insert_kthread_work+0x40/0x40 [177558.419066] Code: 3b 05 af 44 c2 00 89 c2 0f 8d 35 fe ff ff 48 98 49 8b 4d 00 48 03 0c c5 e0 fb ce 81 f6 41 20 01 74 cb 0f 1f 00 f3 90 f6 41 20 01 75 f8 eb be 0f b6 4d ac 48 8b 55 b8 44 89 ef 48 8b 75 c0 e8 fa [177586.007771] BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:2:6155] [177586.015259] Modules linked in: vhost_net vhost macvtap macvlan kvm_intel kvm ebtable_nat xt_CHECKSUM tun nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
Gleb, so I just did a trace of KVM MMU to try to understand why L2 is stuck with shadow on EPT Ensure, EPT is enabled on L0 disabled on L1 On L0: - $ cat /sys/module/kvm_intel/parameters/ept Y On L1 - $ cat /sys/module/kvm_intel/parameters/ept N Build and install trace-cmd on L1: $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git trace-cmd $ make sudo make-install Run trace command (on L1) to capture kvmmmu event: $ trace-cmd record -b 2000 -e kvmmmu /sys/kernel/debug/tracing/events/kvmmmu/filter /sys/kernel/debug/tracing/events/*/kvmmmu/filter Hit Ctrl^C to stop recording Boot L2 guest: $ virsh start nguest-01 --console After a minute or two, ctl-C the trace-cmd, generate the report: $ trace-cmd report version = 6 trace-cmd: No such file or directory function is_writable_pte not defined CPU 1 is empty CPU 2 is empty CPU 3 is empty cpus=4 qemu-system-x86-2554 [000] 90667.995345: kvm_mmu_invalidate_zap_all_pages: kvm-mmu-valid-gen 0 used_pages 0 qemu-system-x86-2557 [000] 90668.078978: kvm_mmu_get_page: new sp gfn 0 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.078980: kvm_mmu_get_page: new sp gfn 4 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.078980: kvm_mmu_get_page: new sp gfn 8 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.078981: kvm_mmu_get_page: new sp gfn c 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.079025: kvm_mmu_get_page: new sp gfn 0 0/1 q0 direct --- !pge !nxe root 0 sync Side question: I wonder why would trace-cmd say trace-cmd: No such file or directory? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
On Mon, Oct 7, 2013 at 6:29 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Gleb, so I just did a trace of KVM MMU to try to understand why L2 is stuck with shadow on EPT Paolo, were you able to reproduce this again? Yesterday, on #qemu you mentioned you'll test it again :-) I was using kvm.git queue on both L0 and L1: $ git describe ; git log | head -1 for-linus-11227-g8a3c1a33 commit 8a3c1a33476f6bfebd07954e2277dbc88003bd37 /kashyap Ensure, EPT is enabled on L0 disabled on L1 On L0: - $ cat /sys/module/kvm_intel/parameters/ept Y On L1 - $ cat /sys/module/kvm_intel/parameters/ept N Build and install trace-cmd on L1: $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git trace-cmd $ make sudo make-install Run trace command (on L1) to capture kvmmmu event: $ trace-cmd record -b 2000 -e kvmmmu /sys/kernel/debug/tracing/events/kvmmmu/filter /sys/kernel/debug/tracing/events/*/kvmmmu/filter Hit Ctrl^C to stop recording Boot L2 guest: $ virsh start nguest-01 --console After a minute or two, ctl-C the trace-cmd, generate the report: $ trace-cmd report version = 6 trace-cmd: No such file or directory function is_writable_pte not defined CPU 1 is empty CPU 2 is empty CPU 3 is empty cpus=4 qemu-system-x86-2554 [000] 90667.995345: kvm_mmu_invalidate_zap_all_pages: kvm-mmu-valid-gen 0 used_pages 0 qemu-system-x86-2557 [000] 90668.078978: kvm_mmu_get_page: new sp gfn 0 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.078980: kvm_mmu_get_page: new sp gfn 4 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.078980: kvm_mmu_get_page: new sp gfn 8 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.078981: kvm_mmu_get_page: new sp gfn c 0/2 q0 direct --- !pge !nxe root 0 sync qemu-system-x86-2557 [000] 90668.079025: kvm_mmu_get_page: new sp gfn 0 0/1 q0 direct --- !pge !nxe root 0 sync Side question: I wonder why would trace-cmd say trace-cmd: No such file or directory? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
I have EPT enabled on L0, disabled on L1, and when attempted to boot an L2 guest, it results in cursor just blinking on the serial console. (All of them are Fedora 19 minimal guests.) To be clear, by disable I mean, the below KVM parameter is N $ cat /sys/module/kvm_intel/parameters/ept N I don't see any boot messages on serial console either. Running $ virsh start nguest-01 --console results in cursor just waiting there and hung. I thought I'll see what's going on in L2 logs by mounting the disk image read-only via guestfish $ guestfish --ro -d nguest-01 -i This is hung too. NOTE: If I enable EPT in L1, I'm able to boot L2 guests just fine and can see all the boot messages. Any hints on what might be going on here? Details about test environment below. Setup info: L0 (Intel Haswell): - 4 pCPU 16G pMEM. - Version (same for both L0 and L1): $ uname -r; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.12.0-0.rc3.git1.2.fc21.x86_64 qemu-kvm-1.4.2-7.fc19.x86_64 libvirt-daemon-kvm-1.0.5.5-1.fc19.x86_64 libguestfs-1.22.5-1.fc19.x86_64 KVM Parameters in L0: $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y $ cat /sys/module/kvm_intel/parameters/ept Y KVM Parameters in L1 (Note - EPT is disabled below): $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N $ cat /sys/module/kvm_intel/parameters/ept N L1 qemu-kvm command line: $ ps -ef | grep qemu qemu 4329 1 30 03:35 ?00:13:47 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu host -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot c -usb -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 L2 qemu-kvm command line: $ ps -ef | grep qemu qemu 1174 1 99 04:13 ?00:06:12 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name nguest-01 -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid b47c5cbb-b320-ce9d-c595-4e083b0e465d -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/nguest-01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/nguest-01.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:be:d5:8e,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
Discussed with Gleb on IRC: I'll have to try with kvm.git and a simple qemu-kvm command line. Thanks, Gleb. On Fri, Oct 4, 2013 at 2:01 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: I have EPT enabled on L0, disabled on L1, and when attempted to boot an L2 guest, it results in cursor just blinking on the serial console. (All of them are Fedora 19 minimal guests.) To be clear, by disable I mean, the below KVM parameter is N $ cat /sys/module/kvm_intel/parameters/ept N I don't see any boot messages on serial console either. Running $ virsh start nguest-01 --console results in cursor just waiting there and hung. I thought I'll see what's going on in L2 logs by mounting the disk image read-only via guestfish $ guestfish --ro -d nguest-01 -i This is hung too. NOTE: If I enable EPT in L1, I'm able to boot L2 guests just fine and can see all the boot messages. Any hints on what might be going on here? Details about test environment below. Setup info: L0 (Intel Haswell): - 4 pCPU 16G pMEM. - Version (same for both L0 and L1): $ uname -r; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.12.0-0.rc3.git1.2.fc21.x86_64 qemu-kvm-1.4.2-7.fc19.x86_64 libvirt-daemon-kvm-1.0.5.5-1.fc19.x86_64 libguestfs-1.22.5-1.fc19.x86_64 KVM Parameters in L0: $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y $ cat /sys/module/kvm_intel/parameters/ept Y KVM Parameters in L1 (Note - EPT is disabled below): $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N $ cat /sys/module/kvm_intel/parameters/ept N L1 qemu-kvm command line: $ ps -ef | grep qemu qemu 4329 1 30 03:35 ?00:13:47 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu host -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot c -usb -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 L2 qemu-kvm command line: $ ps -ef | grep qemu qemu 1174 1 99 04:13 ?00:06:12 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name nguest-01 -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid b47c5cbb-b320-ce9d-c595-4e083b0e465d -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/nguest-01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/nguest-01.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:be:d5:8e,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
On Fri, Oct 4, 2013 at 3:09 PM, Gleb Natapov g...@redhat.com wrote: On Fri, Oct 04, 2013 at 03:03:17PM +0530, Kashyap Chamarthy wrote: Discussed with Gleb on IRC: I'll have to try with kvm.git I just compiled kernel from kvm.git queue on both L0 and L1. For completeness' sake, this how I did: $ git describe for-linus-12017-g8a3c1a33 $ git log | head -1 commit 8a3c1a33476f6bfebd07954e2277dbc88003bd37 $ git branch -d test_ept1 origin/queue $ make -j6 make bzImage make modules $ sudo make modules_install make install (Rebooted both L0 and L1 into the new kernel) and a simple qemu-kvm command line. And try to disable shadow vmcs on the host just to rule it out. $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N The above didn't have any effect. No, booting L2 is still stuck. Not sure what's broken (if at all) here. Gleb, you mentioned you were able to test the MMU combination 'shadow on EPT' just fine with an F19 liveCD without libvirt in-place. If you have still have that test environment, can you please post your qemu-kvm command line? Thanks. Thanks, Gleb. On Fri, Oct 4, 2013 at 2:01 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: I have EPT enabled on L0, disabled on L1, and when attempted to boot an L2 guest, it results in cursor just blinking on the serial console. (All of them are Fedora 19 minimal guests.) To be clear, by disable I mean, the below KVM parameter is N $ cat /sys/module/kvm_intel/parameters/ept N I don't see any boot messages on serial console either. Running $ virsh start nguest-01 --console results in cursor just waiting there and hung. I thought I'll see what's going on in L2 logs by mounting the disk image read-only via guestfish $ guestfish --ro -d nguest-01 -i This is hung too. NOTE: If I enable EPT in L1, I'm able to boot L2 guests just fine and can see all the boot messages. Any hints on what might be going on here? Details about test environment below. Setup info: L0 (Intel Haswell): - 4 pCPU 16G pMEM. - Version (same for both L0 and L1): $ uname -r; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.12.0-0.rc3.git1.2.fc21.x86_64 qemu-kvm-1.4.2-7.fc19.x86_64 libvirt-daemon-kvm-1.0.5.5-1.fc19.x86_64 libguestfs-1.22.5-1.fc19.x86_64 KVM Parameters in L0: $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y $ cat /sys/module/kvm_intel/parameters/ept Y KVM Parameters in L1 (Note - EPT is disabled below): $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N $ cat /sys/module/kvm_intel/parameters/ept N L1 qemu-kvm command line: $ ps -ef | grep qemu qemu 4329 1 30 03:35 ?00:13:47 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu host -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot c -usb -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 L2 qemu-kvm command line: $ ps -ef | grep qemu qemu 1174 1 99 04:13 ?00:06:12 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name nguest-01 -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid b47c5cbb-b320-ce9d-c595-4e083b0e465d -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/nguest-01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/nguest-01.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac
Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2 stuck on boot
On Fri, Oct 4, 2013 at 6:38 PM, Gleb Natapov g...@redhat.com wrote: On Fri, Oct 04, 2013 at 04:05:50PM +0300, Gleb Natapov wrote: On Fri, Oct 04, 2013 at 06:03:33PM +0530, Kashyap Chamarthy wrote: On Fri, Oct 4, 2013 at 3:09 PM, Gleb Natapov g...@redhat.com wrote: On Fri, Oct 04, 2013 at 03:03:17PM +0530, Kashyap Chamarthy wrote: Discussed with Gleb on IRC: I'll have to try with kvm.git I just compiled kernel from kvm.git queue on both L0 and L1. For completeness' sake, this how I did: $ git describe for-linus-12017-g8a3c1a33 $ git log | head -1 commit 8a3c1a33476f6bfebd07954e2277dbc88003bd37 $ git branch -d test_ept1 origin/queue $ make -j6 make bzImage make modules $ sudo make modules_install make install (Rebooted both L0 and L1 into the new kernel) and a simple qemu-kvm command line. And try to disable shadow vmcs on the host just to rule it out. $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N The above didn't have any effect. No, booting L2 is still stuck. Not sure what's broken (if at all) here. Gleb, you mentioned you were able to test the MMU combination 'shadow on EPT' just fine with an F19 liveCD without libvirt in-place. If you have still have that test environment, can you please post your qemu-kvm command line? Something as basic as qemu -m 2G -cdrom /path/to/image -cpu host Thanks. I'll see if I can figure further what's going on here, otherwise I'll just write a bug for now with all the details, just so it's tracked. If it turns out to be a false alarm, I can just close it. And Live CD I use is XFCE based, not sure if it matters. I'm using no X. So, it shouldn't matter :-) -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: nested migration issues
On Tue, Aug 27, 2013 at 2:48 PM, Jan Kiszka jan.kis...@siemens.com wrote: On 2013-08-27 10:29, 缪天翔 wrote: Hi all I am doing a research project similar to Turtles using the nested vmx feature of kvm. In detail, I launch a guest kvm (donated as L1) on the underlying kvm (donated as L0) and then set up a nested guest (donated as L2) upon the L1 guest kvm. The problem I come up with is that I want to migrate the whole L1 guest kvm from source machine to target (that is keeping the L2 guest running normally). And this action causes the L2 kernl to panic. Can anybody tell me whether kvm supports this kind of nested migration or anybody is working on the relating field? No, KVM does not support migration while nested VMX (or SVM) is enabled. We lack userspace interfaces to export/import the required state, and that mostly because we do not really know yet what state needs to be made available. Good to know, you saved some time :-). I was about to try to test if it works or not. Specifically nVMX is still under development. Migration support will officially follow once we feel we reached a mature state with nesting support. That said, if you want to play with adding such an interface yourself, no one will stop you. It may serve as a reference for us what is required, but you should not expect it to be merged soon. BTW, not only migration is broken so far. Even reset does not work while VMX is enabled. For the same reason: userspace cannot load a VCPU state that resets all VMX states properly. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kvm/queue still ahead of kvm/next
On Mon, Aug 19, 2013 at 2:56 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 19/08/2013 06:36, Kashyap Chamarthy ha scritto: On Sat, Aug 10, 2013 at 12:56 AM, Paolo Bonzini pbonz...@redhat.com wrote: Hi all, I'm seeing some breakage of shadow-on-shadow and shadow-on-EPT nested VMX. Until I can track more precisely whether it is a regression, and on which hosts I can reproduce it, I'm going to leave the patches out of kvm/next. The good news is that nested EPT works pretty well. :) Paolo/others, I'm trying to test nEPT, so I'm trying to start this way: $ git clone git://git.kernel.org/pub/scm/virt/kvm/kvm.git $ git branch --all $ git checkout remotes/origin/queue Compile, and proceed. Or would you suggest to use http://git.kiszka.org/?p=kvm-kmod.git;a=blob;f=README to test latest stuff? If you prefer not to build your own kernel, kvm-kmod works too. However, please build it with the latest 3.10.x kernel to make the environment as similar as possible to what you'd get with kvm.git. I'm fine building news Kernels. I just started this way: $ git remote -v origin git://git.kernel.org/pub/scm/virt/kvm/kvm.git (fetch) origin git://git.kernel.org/pub/scm/virt/kvm/kvm.git (push) $ git checkout -b test_nept origin/queue $ make defconfig $ make -j8 make modules $ make install make modules_install And, booting into the just built 3.11.0-rc1+ kernel (this is on bare-metal), hangs at: - . . . [5.437220] systemd[1]: Failed to mount /dev: No such device [5.523997] usb 2-1: New USB device found, idVendor=8087, idProduct=8000 [5.531600] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [5.540188] hub 2-1:1.0: USB hub found [5.544573] hub 2-1:1.0: 8 ports detected [5.757452] Switched to clocksource tsc [5.879287] [drm] Enabling RC6 states: RC6 on, RC6p off, RC6pp off - (This is on a Haswell machine). Am I missing anything here? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kvm/queue still ahead of kvm/next
On Mon, Aug 19, 2013 at 3:12 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: On Mon, Aug 19, 2013 at 2:56 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 19/08/2013 06:36, Kashyap Chamarthy ha scritto: On Sat, Aug 10, 2013 at 12:56 AM, Paolo Bonzini pbonz...@redhat.com wrote: Hi all, I'm seeing some breakage of shadow-on-shadow and shadow-on-EPT nested VMX. Until I can track more precisely whether it is a regression, and on which hosts I can reproduce it, I'm going to leave the patches out of kvm/next. The good news is that nested EPT works pretty well. :) Paolo/others, I'm trying to test nEPT, so I'm trying to start this way: $ git clone git://git.kernel.org/pub/scm/virt/kvm/kvm.git $ git branch --all $ git checkout remotes/origin/queue Compile, and proceed. Or would you suggest to use http://git.kiszka.org/?p=kvm-kmod.git;a=blob;f=README to test latest stuff? If you prefer not to build your own kernel, kvm-kmod works too. However, please build it with the latest 3.10.x kernel to make the environment as similar as possible to what you'd get with kvm.git. I'm fine building news Kernels. I just started this way: $ git remote -v origin git://git.kernel.org/pub/scm/virt/kvm/kvm.git (fetch) origin git://git.kernel.org/pub/scm/virt/kvm/kvm.git (push) $ git checkout -b test_nept origin/queue $ make defconfig $ make -j8 make modules $ make install make modules_install And, booting into the just built 3.11.0-rc1+ kernel (this is on bare-metal), hangs at: - . . . [5.437220] systemd[1]: Failed to mount /dev: No such device [5.523997] usb 2-1: New USB device found, idVendor=8087, idProduct=8000 [5.531600] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [5.540188] hub 2-1:1.0: USB hub found [5.544573] hub 2-1:1.0: 8 ports detected [5.757452] Switched to clocksource tsc [5.879287] [drm] Enabling RC6 states: RC6 on, RC6p off, RC6pp off - After that, it goes one step beyond after the above, and gets hung at: --- [ 305.591298] kworker/u16:1 (24) used greatest stack depth: 5488 bytes left --- (This is on a Haswell machine). Am I missing anything here? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kvm/queue still ahead of kvm/next
On Sat, Aug 10, 2013 at 12:56 AM, Paolo Bonzini pbonz...@redhat.com wrote: Hi all, I'm seeing some breakage of shadow-on-shadow and shadow-on-EPT nested VMX. Until I can track more precisely whether it is a regression, and on which hosts I can reproduce it, I'm going to leave the patches out of kvm/next. The good news is that nested EPT works pretty well. :) Paolo/others, I'm trying to test nEPT, so I'm trying to start this way: $ git clone git://git.kernel.org/pub/scm/virt/kvm/kvm.git $ git branch --all $ git checkout remotes/origin/queue Compile, and proceed. Or would you suggest to use http://git.kiszka.org/?p=kvm-kmod.git;a=blob;f=README to test latest stuff? Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX w/ Haswell] KVM unit-tests in L1 - eventinj test fails trying to send NMI
On Wed, Jun 26, 2013 at 10:49 PM, Jan Kiszka jan.kis...@web.de wrote: On 2013-06-26 10:03, Kashyap Chamarthy wrote: Thanks for the note, it's very helpful! This test actually fails on older CPUs as well, and I can finally reproduce the issue that Jay also reported. I'm not able to cure it by going back to 3b656cf764^, Ok, you tried w/o this commit.. commit 3b656cf764cbc43d3efb9bf5f45c618d4cf0989f Author: Jan Kiszka jan.kis...@siemens.com Date: Sun Apr 14 12:12:45 2013 +0200 KVM: nVMX: Fix injection of PENDING_INTERRUPT and NMI_WINDOW exits to L1 Check if the interrupt or NMI window exit is for L1 by testing if it has the corresponding controls enabled. This is required when we allow direct injection from L0 to L2 I first tried by reverting to the commit before this one, just like Jay reported for https://bugzilla.kernel.org/show_bug.cgi?id=58941. But this just varied the error (kvm reports an internal error), didn't solve the issue. Now I simply reverted the commit on top of next, but without an effect. Looks like those problems are not directly related. Kashyap, you can do us a favor and try to find out if there was a commit in the recent history (roughly before I started to hack on nVMX this year) where this test cases succeeded. Ok, will try this week by trial and error. I'm currently heads down on a couple of other things. By this test cases succeeded - I interpret it as NMI tests succeeded too. Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX w/ Haswell] KVM unit-tests in L1 - eventinj test fails trying to send NMI
Thanks for the note, it's very helpful! This test actually fails on older CPUs as well, and I can finally reproduce the issue that Jay also reported. I'm not able to cure it by going back to 3b656cf764^, Ok, you tried w/o this commit.. commit 3b656cf764cbc43d3efb9bf5f45c618d4cf0989f Author: Jan Kiszka jan.kis...@siemens.com Date: Sun Apr 14 12:12:45 2013 +0200 KVM: nVMX: Fix injection of PENDING_INTERRUPT and NMI_WINDOW exits to L1 Check if the interrupt or NMI window exit is for L1 by testing if it has the corresponding controls enabled. This is required when we allow direct injection from L0 to L2 just alter the error report. Anyway, a start. Now I just need time to debug it... Great, would you prefer a bug to track this? Or will that be ignored? :) Don't hesitate to let me know if you need any further testing help or want me to try something specific. Thanks. Jan /kashyap On Tue, Jun 4, 2013 at 6:14 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Heya, So, I invoked this in L1 with: === [test@foo kvm-unit-tests]$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio -nographic -no-user-config -nodefaults -device isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./x86/eventinj.flat | tee /var/tmp/eventinj-test.txt enabling apic paging enabled cr0 = 80010011 cr3 = 7fff000 cr4 = 20 Try to divide by 0 DE isr running divider is 0 Result is 150 DE exception: PASS Try int 3 BP isr running After int 3 BP exception: PASS Try send vec 33 to itself irq1 running After vec 33 to itself vec 33: PASS Try int $33 irq1 running After int $33 int $33: PASS Try send vec 32 and 33 to itself irq1 running irq0 running After vec 32 and 33 to itself vec 32/33: PASS Try send vec 32 and int $33 irq1 running irq0 running After vec 32 and int $33 vec 32/int $33: PASS Try send vec 33 and 62 and mask one with TPR irq1 running After 33/62 TPR test TPR: PASS irq0 running Try send NMI to itself After NMI to itself NMI: FAIL Try int 33 with shadowed stack irq1 running After int 33 with shadowed stack int 33 with shadowed stack: PASS summary: 9 tests, 1 failures real0m0.647s user0m0.164s sys 0m0.146s [test@foo kvm-unit-tests]$ === Any hints on further debugging this ? Other info: -- - L1's qemu-kvm CLI === # ps -ef | grep -i qemu qemu 5455 1 94 Jun02 ?1-07:14:29 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu Haswell,+vmx -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 root 12255 5419 0 08:41 pts/200:00:00 grep --color=auto -i qemu === - Setup details -- https://github.com/kashyapc/nvmx-haswell/blob/master/SETUP-nVMX.rst /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX w/ Haswell] KVM unit-tests in L1 - eventinj test fails trying to send NMI
Adding Jan, Jun, to see if they have any inputs here. /kashyap On Tue, Jun 4, 2013 at 6:14 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Heya, So, I invoked this in L1 with: === [test@foo kvm-unit-tests]$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio -nographic -no-user-config -nodefaults -device isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./x86/eventinj.flat | tee /var/tmp/eventinj-test.txt enabling apic paging enabled cr0 = 80010011 cr3 = 7fff000 cr4 = 20 Try to divide by 0 DE isr running divider is 0 Result is 150 DE exception: PASS Try int 3 BP isr running After int 3 BP exception: PASS Try send vec 33 to itself irq1 running After vec 33 to itself vec 33: PASS Try int $33 irq1 running After int $33 int $33: PASS Try send vec 32 and 33 to itself irq1 running irq0 running After vec 32 and 33 to itself vec 32/33: PASS Try send vec 32 and int $33 irq1 running irq0 running After vec 32 and int $33 vec 32/int $33: PASS Try send vec 33 and 62 and mask one with TPR irq1 running After 33/62 TPR test TPR: PASS irq0 running Try send NMI to itself After NMI to itself NMI: FAIL Try int 33 with shadowed stack irq1 running After int 33 with shadowed stack int 33 with shadowed stack: PASS summary: 9 tests, 1 failures real0m0.647s user0m0.164s sys 0m0.146s [test@foo kvm-unit-tests]$ === Any hints on further debugging this ? Other info: -- - L1's qemu-kvm CLI === # ps -ef | grep -i qemu qemu 5455 1 94 Jun02 ?1-07:14:29 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu Haswell,+vmx -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 root 12255 5419 0 08:41 pts/200:00:00 grep --color=auto -i qemu === - Setup details -- https://github.com/kashyapc/nvmx-haswell/blob/master/SETUP-nVMX.rst /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[nVMX w/ Haswell] KVM unit-tests in L1 - eventinj test fails trying to send NMI
Heya, So, I invoked this in L1 with: === [test@foo kvm-unit-tests]$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio -nographic -no-user-config -nodefaults -device isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./x86/eventinj.flat | tee /var/tmp/eventinj-test.txt enabling apic paging enabled cr0 = 80010011 cr3 = 7fff000 cr4 = 20 Try to divide by 0 DE isr running divider is 0 Result is 150 DE exception: PASS Try int 3 BP isr running After int 3 BP exception: PASS Try send vec 33 to itself irq1 running After vec 33 to itself vec 33: PASS Try int $33 irq1 running After int $33 int $33: PASS Try send vec 32 and 33 to itself irq1 running irq0 running After vec 32 and 33 to itself vec 32/33: PASS Try send vec 32 and int $33 irq1 running irq0 running After vec 32 and int $33 vec 32/int $33: PASS Try send vec 33 and 62 and mask one with TPR irq1 running After 33/62 TPR test TPR: PASS irq0 running Try send NMI to itself After NMI to itself NMI: FAIL Try int 33 with shadowed stack irq1 running After int 33 with shadowed stack int 33 with shadowed stack: PASS summary: 9 tests, 1 failures real0m0.647s user0m0.164s sys 0m0.146s [test@foo kvm-unit-tests]$ === Any hints on further debugging this ? Other info: -- - L1's qemu-kvm CLI === # ps -ef | grep -i qemu qemu 5455 1 94 Jun02 ?1-07:14:29 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu Haswell,+vmx -m 10240 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 root 12255 5419 0 08:41 pts/200:00:00 grep --color=auto -i qemu === - Setup details -- https://github.com/kashyapc/nvmx-haswell/blob/master/SETUP-nVMX.rst /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[nVMX ] A couple more tests
Heya,
So, Gleb suggested (thank you) to try kvm unit tests in L1, here's some
(incomprehensive) detail. I also ran a defconfig in a 20 loop iteration
(in L2) where subsequent runs after the first one are from page cache.
1. A couple of unit tests w/ 64 bit compiled:
$ ./configure
$ time make
[...]
gcc -g -D__x86_64__ -I../include/x86 -m64 -O1 -MMD -MF x86/.init.d -g
-fomit-frame-pointer -Wall -fno-stack-protector -I. -std=gnu99
-ffreestanding -I lib -I lib/x86 -c -o x86/init.o x86/init.c
x86/init.c: In function ‘main’:
x86/init.c:110:1: warning: control reaches end of non-void function
[-Wreturn-type]
}
^
gcc -g -D__x86_64__ -I../include/x86 -m64 -O1 -MMD -MF x86/.ini
real0m14.358s
user0m6.990s
sys 0m6.639s
= MSR test =
$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio \
-nographic -no-user-config -nodefaults -device \
isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./x86/msr.flat \
enabling apic
MSR_IA32_APICBASE: PASS
MSR_IA32_APICBASE: PASS
IA32_SYSENTER_CS: PASS
MSR_IA32_SYSENTER_ESP: PASS
IA32_SYSENTER_EIP: PASS
MSR_IA32_MISC_ENABLE: PASS
MSR_IA32_CR_PAT: PASS
MSR_FS_BASE: PASS
MSR_GS_BASE: PASS
MSR_KERNEL_GS_BASE: PASS
MSR_EFER: PASS
MSR_LSTAR: PASS
MSR_CSTAR: PASS
MSR_SYSCALL_MASK: PASS
MSR_*STAR eager loading: PASS
15 tests, 0 failures
real0m0.525s
user0m0.147s
sys 0m0.121s
= eventinj test =
$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio \
-nographic -no-user-config -nodefaults -device \
isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./x86/eventinj.flat
enabling apic
paging enabled
cr0 = 80010011
cr3 = 7fff000
cr4 = 20
Try to divide by 0
DE isr running divider is 0
Result is 150
DE exception: PASS
Try int 3
BP isr running
After int 3
BP exception: PASS
Try send vec 33 to itself
irq1 running
After vec 33 to itself
vec 33: PASS
Try int $33
irq1 running
After int $33
int $33: PASS
Try send vec 32 and 33 to itself
irq1 running
irq0 running
After vec 32 and 33 to itself
vec 32/33: PASS
Try send vec 32 and int $33
irq1 running
irq0 running
After vec 32 and int $33
vec 32/int $33: PASS
Try send vec 33 and 62 and mask one with TPR
irq1 running
After 33/62 TPR test
TPR: PASS
irq0 running
Try send NMI to itself
After NMI to itself
NMI: FAIL
Try int 33 with shadowed stack
irq1 running
After int 33 with shadowed stack
int 33 with shadowed stack: PASS
summary: 9 tests, 1 failures
real0m0.586s
user0m0.159s
sys 0m0.163s
2. A couple of kvm unit tests w/ 32-bit compiled:
$ yum install '*/stubs-32.h'
[...]
$ make ARCH=i386 clean all
[...]
gcc -g -D__i386__ -I
/usr/src/kernels/3.10.0-0.rc1.git5.2.fc20.x86_64/include
-I../include/x86 -m32 -O1 -MMD -MF x86/.eventinj.d -g
-fomit-frame-pointer -Wall -fno-stack-protector -I. -nostdlib -o
x86/eventin
j.elf -Wl,-T,flat.lds x86/eventinj.o x86/cstart.o lib/libcflat.a
/usr/lib/gcc/x86_64-redhat-linux/4.8.0/32/libgcc.a
objcopy -O elf32-i386 x86/eventinj.elf x86/eventinj.flat
[...]
= eventinj test =
$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio \
-nographic \ -no-user-config -nodefaults -device \
isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel \ ./x86/eventinj.flat \
[...]
After NMI to itself
NMI: FAIL
Try to divide by 0
PF running
DE isr running divider is 0
Result is 150
DE PF exceptions: PASS
Before NP test
PF running
NP isr running 400b5f err=18
irq1 running
After int33
NP PF exceptions: PASS
Try int 33 with shadowed stack
irq1 running
After int 33 with shadowed stack
int 33 with shadowed stack: PASS
summary: 14 tests, 1 failures
real0m0.589s
user0m0.188s
sys 0m0.127s
= MSR test =
$ time qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio
-nographic \
-no-user-config -nodefaults -device isa-debug-exit,iobase=0xf4,iosize=0x4
-kernel \
./x86/msr.flat
enabling apic
MSR_IA32_APICBASE: PASS
MSR_IA32_APICBASE: PASS
IA32_SYSENTER_CS: PASS
MSR_IA32_SYSENTER_ESP: PASS
IA32_SYSENTER_EIP: PASS
MSR_IA32_MISC_ENABLE: PASS
MSR_IA32_CR_PAT: PASS
MSR_FS_BASE: PASS
MSR_GS_BASE: PASS
MSR_KERNEL_GS_BASE: PASS
MSR_LSTAR: PASS
MSR_CSTAR: PASS
MSR_SYSCALL_MASK: PASS
13 tests, 0 failures
real0m0.499s
user0m0.136s
sys 0m0.117s
3. 20 iterations of kernel compile time:
http://kashyapc.fedorapeople.org/virt/Haswell-VMCS-Shadowing-tests/kernel-build-times-20-iterations.txt
Average real (elapsed) time - 9m44.495s.
The result of of the above file is obtained after:
$ for i in {1..20}; do make clean; \
time make -j 3; done 21 | tee \
defconfig-make-output-20-iterations-l2.txt
in L2 (nested guest).
And, then
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
A small update: Yesterday, I also ran a simple libguestfs test[1], which creates a light weight appliance (i.e. an L2) guest , with and without nesting to see some over-head. Summary: - L0 (bare-metal): - guestfish run: ~ 3 seconds L1 (guest): - w/o nesting, guestfish run: ~ 52 seconds - w/ nesting, guestfish run: ~ 8 seconds (Almost 6 1/2 times speed improvement) To clarify, by w/ nesting what I meant is, L1's qemu-kvm cli will have: -cpu Core2Duo,+vmx # or whatever the model is. And, w/o nesting: No -cpu parameter in L1's qemu-kvm cli. Details: - On libguestfs mailing list -- https://www.redhat.com/archives/libguestfs/2013-May/msg00033.html [1] The baseline measurement mentioned there -- http://libguestfs.org/guestfs-performance.1.html Thanks. Details: https://www.redhat.com/archives/libguestfs/2013-May/msg00033.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
Thanks for the detailed configurations and steps about the nVMX test. I have some suggestions if you want to test more: 1. you can make L2 guest have 4vCPU and 4Mem Yeah, will try it. L0 has a max of 8G MEM. 2. as L2 is SMP system, you can try 'time make -j 4' instead of 'time make'. This is my next step :) . 3. some performance comparison when enable/disable VMCS shadowing Yes, as we speak, Kernel is building in L2 w/ VMCS Shadowing disabled on L0. I'll do two runs. 4. some other benchmarks (e.g. UnixBench, SuperPI) ? Yeah, I was researching some benchmark tools, and came across the above. I've never tried them before. If they provide some meaningful results, I'll give them a shot. Thanks for your response. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
On Mon, May 13, 2013 at 11:51 AM, Kashyap Chamarthy kashyap...@gmail.com wrote: Thanks for the detailed configurations and steps about the nVMX test. I have some suggestions if you want to test more: 1. you can make L2 guest have 4vCPU and 4Mem Yeah, will try it. L0 has a max of 8G MEM. 2. as L2 is SMP system, you can try 'time make -j 4' instead of 'time make'. This is my next step :) . 3. some performance comparison when enable/disable VMCS shadowing Yes, as we speak, Kernel is building in L2 w/ VMCS Shadowing disabled on L0. I'll do two runs. Interesting: 1/ With VMCS Shadowing *disabled* : - On L0 (Side note: I did reboot the host, once I disabled shadow VMCS) $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N - Building Kernel on L2: $ time make . real36m33.361s user18m19.003s sys 17m6.208s 2/ With VMCS Shadowing *enabled* : - On L0 (Side note: I did reboot the host, once I enabled shadow VMCS) $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y - Building Kernel on L2: $ time make . real62m53.853s user32m32.724s sys 32m16.430s Comments ? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
On Mon, May 13, 2013 at 12:46 PM, Abel Gordon ab...@il.ibm.com wrote: Kashyap Chamarthy kashyap...@gmail.com wrote on 13/05/2013 10:07:02 AM: 1/ With VMCS Shadowing *disabled* : - On L0 (Side note: I did reboot the host, once I disabled shadow VMCS) $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs N - Building Kernel on L2: $ time make . real36m33.361s user18m19.003s sys 17m6.208s 2/ With VMCS Shadowing *enabled* : - On L0 (Side note: I did reboot the host, once I enabled shadow VMCS) $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y - Building Kernel on L2: $ time make . real62m53.853s user32m32.724s sys 32m16.430s Comments ? wow... according to this info shadow-vmcs is *significantly* slowing nested while it should be the opposite. Indeed. Did you monitor the exit rate with and without shadow-vmcs ? I'm afraid, I'm not too familiar w/ the internals of KVM, still learning. By exit rate, I presume, VMX exit ? Can you please suggest a way to capture these exits, so I could get reliable information post the notes here ? qemu/scripts/kvm_stat ? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
On Mon, May 13, 2013 at 1:00 PM, Ren, Yongjie yongjie@intel.com wrote: -Original Message- From: Kashyap Chamarthy [mailto:kashyap...@gmail.com] Sent: Monday, May 13, 2013 3:07 PM To: Ren, Yongjie Cc: kvm@vger.kernel.org Subject: Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0 On Mon, May 13, 2013 at 11:51 AM, Kashyap Chamarthy kashyap...@gmail.com wrote: Comments ? My test result is different from yours. In my test several days ago, when enabling shadow VMCS, I can get ~4% performance improvement in kernel build test in L2. Hardware is also a Haswell machine. shadow VMCS disabled, build time: 649 seconds shadow VMCS enabled, build time: 627 seconds. Let me try it on a new L2 guest to see if I can reproduce the behavior I reported reliably. Meantime, I noted my entire procedure https://github.com/kashyapc/nvmx-haswell/blob/master/SETUP-nVMX.rst Do you see any anomalies ? /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
Yep, kvm_stat is the simplest way to monitor the exits frequency. Attached is kvm_stat results from L1, captured from several seconds of intervals. /kashyap kvm_entry 79572047 kvm_exit 79592045 kvm_page_fault62371649 kvm_inj_exception 2074 552 kvm_emulate_insn 1162 311 kvm_apic 983 258 vcpu_match_mmio972 258 kvm_mmio 971 257 kvm_cr 933 235 kvm_inj_virq 468 123 kvm_eoi455 120 kvm_apic_accept_irq450 120 kvm_pv_eoi 413 111 kvm_apic_ipi74 21 kvm_cpuid 64 16 kvm_fpu 21 5 kvm_msi_set_irq 1 0 kvm_pio 1 0 kvm_exit(EXCEPTION_NMI) 1115 0 kvm_exit(EXTERNAL_INTERRUPT) 223 0 kvm_exit(CR_ACCESS)177 0 kvm_exit(HLT) 24 0 kvm_exit(CPUID) 16 0 kvm_exit(PENDING_INTERRUPT) 16 0 kvm_exit(INVLPG)11 0 kvm_exit(TRIPLE_FAULT) 6 0 kvm_exit(APIC_ACCESS)6 0 kvm_exit(MWAIT_INSTRUCTION) 5 0 kvm_exit(EPT_VIOLATION) 5 0 kvm_exit(TASK_SWITCH)5 0 kvm_exit(VMREAD) 5 0 kvm_exit(RDPMC) 5 0 kvm_exit(VMRESUME) 5 0 kvm_exit(EPT_MISCONFIG) 5 0 kvm_exit(VMOFF) 5 0 kvm_exit(DR_ACCESS) 4 0 kvm_exit(IO_INSTRUCTION) 4 0 kvm_exit(VMCLEAR)4 0 kvm_exit(VMON) 4 0 kvm_exit(MCE_DURING_VMENTRY) 3 0 kvm_exit(INVALID_STATE) 3 0 kvm_exit(TPR_BELOW_THRESHOLD)2 0 kvm_exit(MONITOR_INSTRUCTION)2 0 kvm statistics kvm_exit 2310005 29825 kvm_entry 230 29823 kvm_page_fault 2082037 26953 kvm_inj_exception 874914 11517 kvm_emulate_insn2366062991 kvm_mmio2126062789 vcpu_match_mmio 2122472784 kvm_apic2118982779 kvm_inj_virq1101411419 kvm_eoi 1089061410 kvm_apic_accept_irq 1089031410 kvm_pv_eoi 1025771338 kvm_cr 71455 804 kvm_apic_ipi 13427 165 kvm_cpuid 9242 76 kvm_fpu 3103 23 kvm_userspace_exit 360 5 kvm_msi_set_irq244 1 kvm_pio210 0 kvm_exit(EXCEPTION_NMI) 1115 0 kvm_exit(EXTERNAL_INTERRUPT) 223 0 kvm_exit(CR_ACCESS)177 0 kvm_msr 36 0 kvm_exit(HLT) 24 0 kvm_exit(CPUID) 16 0 kvm_exit(PENDING_INTERRUPT) 16 0 kvm_exit(INVLPG)11 0 kvm_exit(TRIPLE_FAULT) 6 0 kvm_exit(APIC_ACCESS)6 0 kvm_exit(MWAIT_INSTRUCTION) 5 0 kvm_exit(EPT_VIOLATION) 5 0 kvm_exit(TASK_SWITCH)5 0 kvm_exit(VMREAD) 5 0 kvm_exit(RDPMC) 5 0 kvm_exit(VMRESUME)
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
Thanks Gleb. Here we go: This time results appear more closer to what Ren mentioned in his previous email - i.e. enabling VMCS Shadowing didn't degrade, the Kernel compile time in L2. 1] VMCS Shadowing *enabled* on L0; Kernel build on L2, 3.10.0-0.rc0.git26.1.fc20.x86_64 Kernel build time on L2 -- $ time make [...] real31m0.386s user16m43.655s sys 14m6.451s kvm_stat intervals on L1 --- - http://kashyapc.fedorapeople.org/virt/kvm_stat-VMCS-Shadowing/kvm_stat-L0-VMCS-Shadowing-enabled.txt - NOTE: The below statistics are from intervals ranging from 30 seconds to 6,10,13.. minutes. 2] VMCS Shadowing *disabled* on L0; Kernel build on L2, 3.10.0-0.rc0.git26.1.fc20.x86_64 Kernel build time on L2 -- $ time make [...] real40m56.010s user20m19.061s sys 19m57.723s kvm_stat intervals on L1 --- - http://kashyapc.fedorapeople.org/virt/kvm_stat-VMCS-Shadowing/kvm_stat-L0-VMCS-Shadowing-disabled.txt - NOTE: The below statistics are from intervals ranging from 30 seconds to 6,10,13.. minutes. Looks better ? (Note to self: Maybe I should run these more number of times to monitor consistency) Thanks, Kashyap. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
- NOTE: The below statistics are from intervals ranging from 30 seconds to 6,10,13.. minutes. Looks better ? Much better :) Now seems the results were improved by 25% (reduced 10m of 40m) :) I wonder how it differed the first time. (Note to self: Maybe I should run these more number of times to monitor consistency) Yes, that's a good point. You could start with a short make and repeat the experiments to verify the results are consistent. Once you have reproducible results with a short make you can move to back to a long make. Just to be clear: - Short make - Subsequent runs of 'make 'after the first run (i.e. from page-cache). - Long make - 1st run of 'make' , after a 'make clean' (i.e. from disk). I presume that above is what you meant. Note that if you just run make then the source files will be loaded from disk only the first run (lot of I/O, which is slow with nested). Next runs may load the files from the page-cache and you may notice a performance improvement. Glad that you mention it.. Maybe, that's why your results were not like we expected in the first time. ..I did perform a make clean before doing make on each run. I note it in the above kvm_stat files. Re-running make (on L2), from page-cache gives me the below statistics: -- [test@nguest-02 linux]$ time make make[1]: Nothing to be done for `all'. make[1]: Nothing to be done for `relocs'. CHK include/generated/uapi/linux/version.h CHK include/generated/utsrelease.h CALLscripts/checksyscalls.sh CC scripts/mod/devicetable-offsets.s GEN scripts/mod/devicetable-offsets.h HOSTCC scripts/mod/file2alias.o HOSTLD scripts/mod/modpost CHK include/generated/compile.h make[3]: `arch/x86/realmode/rm/realmode.bin' is up to date. Kernel: arch/x86/boot/bzImage is ready (#6) Building modules, stage 2. MODPOST 10 modules real1m38.276s user0m36.806s sys 0m55.493s [test@nguest-02 linux]$ -- 2nd-run: --- real1m37.392s user0m36.587s sys 0m55.296s --- I'll run a short script for longer duration. Thanks Abel. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
Not exactly. With short a make I was a referring to a run that does not take more than few minutes and with a long a make I was referring to a run that may take an hour. Ah, sorry for misinterpreting it. I'll try to spend more time on it. Note that make clean clears the output but the source files may still be in the page-cache (if they were loaded into the cache by a previous run). Noted. Thank you. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
Another point I totally forgot to note - these tests were performeddebug enabled kernels from Fedora's Rawhide. I'll also try w/ Nodebug ones -- http://fedoraproject.org/wiki/RawhideKernelNodebug /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
I tried to reproduce such a problem, and I found L2 (Linux) hangs in
SeaBIOS, after line iPXE (http://ipxe.org) It happens with or
w/o VMCS shadowing (and even without my virtual EPT patches). I didn't
realize this problem until I updated the L1 kernel to the latest (e.g.
3.9.0) from 3.7.0. L0 uses the kvm.git, next branch. It's possible
that the L1 kernel exposed a bug with the nested virtualization, as we
saw such cases before.
This is probably fixed by 8d76c49e9ffeee839bc0b7a3278a23f99101263e. Try
it please.
I don't see the above SeaBIOS hang, however I'm able to consistently
reproduce this stack trace when booting L1 guest:
[2.516894] VFS: Cannot open root device mapper/fedora-root or
unknown-block(0,0): error -6
[2.527636] Please append a correct root= boot option; here are
the available partitions:
[2.538792] Kernel panic - not syncing: VFS: Unable to mount root
fs on unknown-block(0,0)
[2.539716] Pid: 1, comm: swapper/0 Not tainted 3.8.11-200.fc18.x86_64 #1
[2.539716] Call Trace:
[2.539716] [81649c19] panic+0xc1/0x1d0
[2.539716] [81d010e0] mount_block_root+0x1fa/0x2ac
[2.539716] [81d011e9] mount_root+0x57/0x5b
[2.539716] [81d0132a] prepare_namespace+0x13d/0x176
[2.539716] [81d00e1c] kernel_init_freeable+0x1cf/0x1da
[2.539716] [81d00610] ? do_early_param+0x8c/0x8c
[2.539716] [81637ca0] ? rest_init+0x80/0x80
[2.539716] [81637cae] kernel_init+0xe/0xf0
[2.539716] [8165bd6c] ret_from_fork+0x7c/0xb0
[2.539716] [81637ca0] ? rest_init+0x80/0x80
[2.539716] Uhhuh. NMI received for unknown reason 30 on CPU 1.
[2.539716] Do you have a strange power saving mode enabled?
[2.539716] Dazed and confused, but trying to continue
[2.539716] Uhhuh. NMI received for unknown reason 20 on CPU 1.
Howver, L1 boots just fine.
When I try to boot L2, it throws this different stack trace.
[176092.303585] lock(dev-device_lock);
[176092.307947]
[176092.307947] *** DEADLOCK ***
[176092.307947]
[176092.314943] 2 locks held by systemd/1:
[176092.319283] #0: (misc_mtx){+.+.+.}, at: [814534b8]
misc_open+0x28/0x1d0
[176092.328104] #1: (wdd-lock){+.+...}, at: [81557f22]
watchdog_start+0x22/0x80
[176092.337532]
[176092.337532] stack backtrace:
[176092.342661] CPU: 1 PID: 1 Comm: systemd Not tainted
3.10.0-0.rc0.git23.1.fc20.x86_64 #1
[176092.351823] Hardware name: Intel Corporation Shark Bay Client
platform/Flathead Creek Crb, BIOS HSWLPTU1.86C.0109.R03.1301282055
01/28/2013
[176092.366101] 8257d070 880241b1b9c0 81719128
880241b1ba00
[176092.374617] 81714d75 880241b1ba50 880241b80960
880241b8
[176092.383130] 0002 0002 880241b80960
880241b1bac0
[176092.391647] Call Trace:
[176092.394514] [81719128] dump_stack+0x19/0x1b
2m OK ] Re[176092.400430] [81714d75] print_circular_bug+0x201/0x210
[176092.408898] [810db094] __lock_acquire+0x17c4/0x1b30
ached target Shu[176092.415602] [81720d7c] ?
_raw_spin_unlock_irq+0x2c/0x50
[176092.424276] [810dbbf2] lock_acquire+0xa2/0x1f0
tdown.
[176092.430489] [8149028d] ? mei_wd_ops_start+0x2d/0xf0
[176092.438070] [8171d590] mutex_lock_nested+0x80/0x400
[176092.444772] [8149028d] ? mei_wd_ops_start+0x2d/0xf0
[176092.451471] [8149028d] ? mei_wd_ops_start+0x2d/0xf0
[176092.458172] [81557f22] ? watchdog_start+0x22/0x80
[176092.464678] [81557f22] ? watchdog_start+0x22/0x80
[176092.471182] [8149028d] mei_wd_ops_start+0x2d/0xf0
[176092.477687] [81557f5d] watchdog_start+0x5d/0x80
[176092.483994] [81558168] watchdog_open+0x88/0xf0
[176092.490214] [81453547] misc_open+0xb7/0x1d0
[176092.496128] [811e15d2] chrdev_open+0x92/0x1d0
[176092.502240] [811da57b] do_dentry_open+0x24b/0x300
[176092.508745] [812e8e7c] ? security_inode_permission+0x1c/0x30
[176092.516330] [811e1540] ? cdev_put+0x30/0x30
[176092.522243] [811da670] finish_open+0x40/0x50
[176092.528256] [811ec139] do_last+0x4d9/0xe40
[176092.534071] [811ecb53] path_openat+0xb3/0x530
[176092.540193] [810acc1f] ? local_clock+0x5f/0x70
[176092.546403] [8101fcf5] ? native_sched_clock+0x15/0x80
[176092.553301] [810d5d9d] ? trace_hardirqs_off+0xd/0x10
[176092.560099] [811ed658] do_filp_open+0x38/0x80
[176092.566211] [81720c77] ? _raw_spin_unlock+0x27/0x40
[176092.572913] [811fc39f] ? __alloc_fd+0xaf/0x200
[176092.579123] [811db9a9] do_sys_open+0xe9/0x1c0
[176092.585235] [811dba9e] SyS_open+0x1e/0x20
[176092.590953] [8172a999] system_call_fastpath+0x16/0x1b
Sending SIGTERM to remaining processes...
[176092.622745] systemd-journald[338]: Received SIGTERM
Sending SIGKILL
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
On Sun, May 12, 2013 at 6:08 PM, Gleb Natapov g...@redhat.com wrote: On Sun, May 12, 2013 at 06:00:38PM +0530, Kashyap Chamarthy wrote: I tried to reproduce such a problem, and I found L2 (Linux) hangs in SeaBIOS, after line iPXE (http://ipxe.org) It happens with or w/o VMCS shadowing (and even without my virtual EPT patches). I didn't realize this problem until I updated the L1 kernel to the latest (e.g. 3.9.0) from 3.7.0. L0 uses the kvm.git, next branch. It's possible that the L1 kernel exposed a bug with the nested virtualization, as we saw such cases before. This is probably fixed by 8d76c49e9ffeee839bc0b7a3278a23f99101263e. Try it please. I don't see the above SeaBIOS hang, however I'm able to consistently reproduce this stack trace when booting L1 guest: You mean L2 here? Yes. (Sorry about that.) L2 guest cannot find root file system. Unlikely related to KVM. Yeah, fair enough. Howver, L1 boots just fine. When I try to boot L2, it throws this different stack trace. Who is it? The stack trace bellow is from L0 judging by hardware name. Again not KVM related. Again, sorry :(. I was just about to reply that this was physical host. I'm testing by disabling VMCS Shadowing per Jan Kiszka's suggestion, and retrying. But I doubt that's the reason my L2 is seg-faulting. If it still fails, I'll try to create a new L2 to see I can reproduce more consistently. Thanks for your response. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
On Sun, May 12, 2013 at 6:29 PM, Abel Gordon ab...@il.ibm.com wrote: Kashyap Chamarthy kashyap...@gmail.com wrote on 12/05/2013 03:42:33 PM: Again, sorry :(. I was just about to reply that this was physical host. I'm testing by disabling VMCS Shadowing per Jan Kiszka's suggestion, and retrying. But I doubt that's the reason my L2 is seg-faulting. If it still fails, I'll try to create a new L2 to see I can reproduce more consistently. I doubt shadow-vmcs is related to this issue. Indeed. I just re-tested w/o it, and it has no effect. I'm trying a guest w/ newer kernel in L2. Note shadow vmcs is disabled unless you have a processor that supports this feature. Do you ?! Yes, I noted this in my previous email. I'm using Intel Haswell. Here's the info from MSR bits on the machine(From `Table 35-3`, MSRs in Procesors Based on Intel Core Microarchitecture, `Volume 3C of the SDM ) # Read msr value $ rdmsr 0x48B 7cff # Check Shadow VMCS is enabled: $ rdmsr 0x0485 300481e5 And, on the Kernel command line: # nested $ cat /sys/module/kvm_intel/parameters/nested Y # shadow VMCS $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y Just for reference, here's the detailed procedure I noted while testing it on Haswell -- https://raw.github.com/kashyapc/nvmx-haswell/master/SETUP-nVMX.rst Also note you can disable shadow-vmcs using the kvm-intel kernel module parameter enable_shadow_vmcs. Yes, to test w/o shadow VMCS, I disabled it by adding options kvm-intel enable_shadow_vmcs=y to /etc/modprobe.d/dist.conf reboot the host. Anyway, if you conclude this is related to shadow-vmcs let me know and I'll try to help. So, from the above info shadow-vmcs is ruled-out. I'm trying to investigate further, will post details if I have new findings. Thank you for your help. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
So, from the above info shadow-vmcs is ruled-out. I'm trying to investigate further, will post details if I have new findings. Update: - I just tried to create L2 w/ Fedora-19 TC4 compose of 11MAY2013, I contibuously see the below fragment (F18/F19, whatever the L2 guest is): [ 217.938034] Uhhuh. NMI received for unknown reason 30 on CPU 0. [ 217.938034] Do you have a strange power saving mode enabled? .[ 222.523373] Uhhuh. NMI received for unknown reason 20 on CPU 0. [ 222.524073] Do you have a strange power saving mode enabled? [ 222.524073] Dazed and confused, but trying to continue [ 243.860319] Uhhuh. NMI received for unknown reason 30 on CPU 0 . At the moment, L2 guest creation stuck at the above message However I have neither HPET or NMI Watchdog enabled on L0/L1. I checked it by: $ cat /etc/grub2.cfg | egrep -i 'hpet|nmi' I wonder if I'm missing something trivial, or maybe this is some kind of bug that needs more deeper investigation. But Jan Kiszka reported he isn't seeing any probs (but he isn't using Haswell ). Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
[ 217.938034] Uhhuh. NMI received for unknown reason 30 on CPU 0. [ 217.938034] Do you have a strange power saving mode enabled? .[ 222.523373] Uhhuh. NMI received for unknown reason 20 on CPU 0. [ 222.524073] Do you have a strange power saving mode enabled? [ 222.524073] Dazed and confused, but trying to continue [ 243.860319] Uhhuh. NMI received for unknown reason 30 on CPU 0 . At the moment, L2 guest creation stuck at the above message Well, not entirely stuck, it's moving, but w/ intermittent spitting of the above message. . . . Installing nss-softokn-freebl (10/236) [ 716.751098] Uhhuh. NMI received for unknown reason 30 on CPU 1. [ 716.751098] Do you have a strange power saving mode enabled? Installing glibc-common (11/236)d, but trying to continue [ 735.785034] Uhhuh. NMI received for unknown reason 20 on CPU 0. [ 735.785034] Do you have a strange power saving mode enabled? [ 735.785034] Dazed and confused, but trying to continue [ 736.502032] Uhhuh. NMI received for unknown reason 30 on CPU 0. [ 736.502032] Do you have a strange power saving mode enabled? [ 736.502032] Dazed and confused, but trying to continue [ 737.204936] Uhhuh. NMI received for unknown reason 20 on CPU 1. [ 737.205051] Do you have a strange power saving mode enabled? Installing glibc (12/236)confused, but trying to continue . . . /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
Yep, shadow-vmcs enabled :) :) Good to clarify. Just for reference, here's the detailed procedure I noted while testing it on Haswell -- https://raw.github.com/kashyapc/nvmx-haswell/master/SETUP-nVMX.rst Also note you can disable shadow-vmcs using the kvm-intel kernel module parameter enable_shadow_vmcs. Yes, to test w/o shadow VMCS, I disabled it by adding options kvm-intel enable_shadow_vmcs=y to /etc/modprobe.d/dist.conf reboot the host. I assume you meant enable_shadow_vmcs=n :) Yes, oops, typo :) Small question: did you try to disable apicv/posted interrupts at L0 ? I don't have to explicitly disable. Like Gleb (correctly) noted in his response, APIC-V is not present on Haswell machines. So, it's disabled by default. $ cat /sys/module/kvm_intel/parameters/enable_apicv N (Side note: I did post the out o/p of the above parameters and more in the SETUP-nVMX.rst notes I pointed above. But, I understand, that document is a bit large :) ) (for L1 you can't enable these features because they are not emulated) Yes, Paolo clarified this to me on IRC, when I erroneously assumed so. Thanks Paolo ! Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
[ 217.938034] Uhhuh. NMI received for unknown reason 30 on CPU 0. [ 217.938034] Do you have a strange power saving mode enabled? .[ 222.523373] Uhhuh. NMI received for unknown reason 20 on CPU 0. [ 222.524073] Do you have a strange power saving mode enabled? [ 222.524073] Dazed and confused, but trying to continue [ 243.860319] Uhhuh. NMI received for unknown reason 30 on CPU 0 . At the moment, L2 guest creation stuck at the above message Are those in L2 dmesg or L1? L2 dmesg. $ cat /etc/grub2.cfg | egrep -i 'hpet|nmi' IIRC watchdog is enabled by default. Indeed, you're right. I disabled NMI on L1, and rebooted the newly created L2 guest starts just fine. I'm cloning it to run another instance of L2. And, later try some kernel compiles inside L2 to see if I can consistently get some measurable numbers. Thanks all for your help. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[nVMX Test] Build kernel (defconfig) on L2, w/ VMCS Shadowing enabled on L0
Heya, A simple test below: Setup Info: - L0: - 4 pCPU, 8G pMEM. - Version: $ uname -r; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.10.0-0.rc0.git26.1.fc20.x86_64 qemu-kvm-1.4.1-1.fc19.x86_64 libvirt-daemon-kvm-1.0.5-2.fc19.x86_64 libguestfs-1.21.35-1.fc19.x86_64 - KVM parameters: $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y $ cat /sys/module/kvm_intel/parameters/enable_apicv N $ cat /sys/module/kvm_intel/parameters/ept Y L1: - 4 vCPU, 6G vMEM. - Version: $ uname -r; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.10.0-0.rc0.git26.1.fc20.x86_64 qemu-kvm-1.4.1-1.fc19.x86_64 libvirt-daemon-kvm-1.0.5-2.fc19.x86_64 libguestfs-1.21.35-1.fc19.x86_64 L2: - 3 vCPU, 3G vMEM. - Version: $ uname -r 3.9.1-301.fc19.x86_64 qemu-kvm command line for L0 and L1) here: https://raw.github.com/kashyapc/nvmx-haswell/master/SETUP-nVMX.rst Build Kernel w/ defconfig on L2: $ yum install git gcc -y $ yum-builddep kernel $ git clone \ git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git $ git describe v3.10-rc1 $ cd linux $ time make defconfig [...] real0m30.027s user0m9.716s sys 0m9.686s $ time make [...] real63m16.320s user32m28.038s sys 32m21.190s I should perhaps try a few possibilities here for MEM, CPU intensive workloads: - Build Kernel w/ defconfig, VMCS Shadowing enabled on L0. - Build Kernel w/ defconfig, VMCS Shadowind disabled on L0. Some more for I/O ? If you have further suggestions, please advise. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
Side note: While testing nVMX, I was hitting a libvirt bug, and filed this one: -- https://bugzilla.redhat.com/show_bug.cgi?id=961665 -- [virsh] Attempt to force destroy a guest fails due to 'unknown' reason, leaves a defunct qemu process which I was told is possibly a Kernel/KVM bug. Any further insights here ? Also, others testing with Libvirt (versions mentioned in the above bug), are you also seeing this ? Thanks. /kashyap Also, others testing w/ Libvirt - do you also see this ? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
Heya, This is on Intel Haswell. First, some version info: L0, L1 -- both of them have same versions of kernel, qemu: = $ rpm -q kernel --changelog | head -2 * Thu May 09 2013 Josh Boyer - 3.10.0-0.rc0.git23.1 - Linux v3.9-11789-ge0fd9af = = $ uname -r ; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.10.0-0.rc0.git23.1.fc20.x86_64 qemu-kvm-1.4.1-1.fc19.x86_64 libvirt-daemon-kvm-1.0.5-2.fc19.x86_64 libguestfs-1.21.35-1.fc19.x86_64 = Additionally, neither nmi_watchdog, nor hpet enabled on L0 L1 kernels: = $ egrep -i 'nmi|hpet' /etc/grub2.cfg $ = KVM parameters on L0 : = $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y $ cat /sys/module/kvm_intel/parameters/enable_apicv N $ cat /sys/module/kvm_intel/parameters/ept Y = - That's the stack trace I'm seeing, when I start the L2 guest: ... [2.162235] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [2.163080] Pid: 1, comm: swapper/0 Not tainted 3.8.11-200.fc18.x86_64 #1 [2.163080] Call Trace: [2.163080] [81649c19] panic+0xc1/0x1d0 [2.163080] [81d010e0] mount_block_root+0x1fa/0x2ac [2.163080] [81d011e9] mount_root+0x57/0x5b [2.163080] [81d0132a] prepare_namespace+0x13d/0x176 [2.163080] [81d00e1c] kernel_init_freeable+0x1cf/0x1da [2.163080] [81d00610] ? do_early_param+0x8c/0x8c [2.163080] [81637ca0] ? rest_init+0x80/0x80 [2.163080] [81637cae] kernel_init+0xe/0xf0 [2.163080] [8165bd6c] ret_from_fork+0x7c/0xb0 [2.163080] [81637ca0] ? rest_init+0x80/0x80 [2.163080] Uhhuh. NMI received for unknown reason 30 on CPU 0. [2.163080] Do you have a strange power saving mode enabled? [2.163080] Dazed and confused, but trying to continue [2.163080] Uhhuh. NMI received for unknown reason 20 on CPU 0. [2.163080] Do you have a strange power saving mode enabled? [2.163080] Dazed and confused, but trying to continue [2.163080] Uhhuh. NMI received for unknown reason 30 on CPU 0. I'm able to reproduce to reproduce this consistently. L1 QEMU command-line: $ ps -ef | grep -i qemu qemu 4962 1 21 15:41 ?00:00:41 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu Haswell,+vmx -m 6144 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 L2 QEMU command-line: $ qemu 2042 1 0 May09 ?00:05:03 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name nested-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid 02ea8988-1054-b08b-bafe-cfbe9659976c -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/nested-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/nested-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:65:c4:e6,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 I attached the vmxcap script output. Before I debug further, anyone has hints here? Many thanks in advance. [1] Notes -- https://github.com/kashyapc/nested-virt-notes-intel-f18 /kashyap Basic VMX Information Revision 18 VMCS size1024 VMCS restricted to 32 bit addresses no Dual-monitor support yes VMCS memory type 6 INS/OUTS instruction information yes IA32_VMX_TRUE_*_CTLS
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
On Fri, May 10, 2013 at 8:03 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Also, I'm able to reproduce this consistently: When I create an L2 guest: -- [.] [ 463.655031] Dazed and confused, but trying to continue [ 463.975563] Uhhuh. NMI received for unknown reason 20 on CPU 1. [ 463.976040] Do you have a strange power saving mode enabled? [ 463.976040] Dazed and confused, but trying to continue 29 199M 29 58.7M0 0 136k 0 0:25:02 0:07:20 0:17:42 153k [ 465.136405] Uhhuh. NMI received for unknown reason 30 on CPU 1. [ 465.137042] Do you have a strange power saving mode enabled? [ 465.137042] Dazed and confused, but trying to continue [ 466.645818] Uhhuh. NMI received for unknown reason 20 on CPU 1. [ 466.646044] Do you have a strange power saving mode enabled? [ 466.646044] Dazed and confused, but trying to continue [ 466.907999] Uhhuh. NMI received for unknown reason 30 on CPU 1. [ 466.908033] Do you have a strange power saving mode enabled? -- On Fri, May 10, 2013 at 6:30 PM, Kashyap Chamarthy kashyap...@gmail.com wrote: Heya, This is on Intel Haswell. First, some version info: L0, L1 -- both of them have same versions of kernel, qemu: = $ rpm -q kernel --changelog | head -2 * Thu May 09 2013 Josh Boyer - 3.10.0-0.rc0.git23.1 - Linux v3.9-11789-ge0fd9af = = $ uname -r ; rpm -q qemu-kvm libvirt-daemon-kvm libguestfs 3.10.0-0.rc0.git23.1.fc20.x86_64 qemu-kvm-1.4.1-1.fc19.x86_64 libvirt-daemon-kvm-1.0.5-2.fc19.x86_64 libguestfs-1.21.35-1.fc19.x86_64 = Additionally, neither nmi_watchdog, nor hpet enabled on L0 L1 kernels: = $ egrep -i 'nmi|hpet' /etc/grub2.cfg $ = KVM parameters on L0 : = $ cat /sys/module/kvm_intel/parameters/nested Y $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs Y $ cat /sys/module/kvm_intel/parameters/enable_apicv N $ cat /sys/module/kvm_intel/parameters/ept Y = - That's the stack trace I'm seeing, when I start the L2 guest: ... [2.162235] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [2.163080] Pid: 1, comm: swapper/0 Not tainted 3.8.11-200.fc18.x86_64 #1 [2.163080] Call Trace: [2.163080] [81649c19] panic+0xc1/0x1d0 [2.163080] [81d010e0] mount_block_root+0x1fa/0x2ac [2.163080] [81d011e9] mount_root+0x57/0x5b [2.163080] [81d0132a] prepare_namespace+0x13d/0x176 [2.163080] [81d00e1c] kernel_init_freeable+0x1cf/0x1da [2.163080] [81d00610] ? do_early_param+0x8c/0x8c [2.163080] [81637ca0] ? rest_init+0x80/0x80 [2.163080] [81637cae] kernel_init+0xe/0xf0 [2.163080] [8165bd6c] ret_from_fork+0x7c/0xb0 [2.163080] [81637ca0] ? rest_init+0x80/0x80 [2.163080] Uhhuh. NMI received for unknown reason 30 on CPU 0. [2.163080] Do you have a strange power saving mode enabled? [2.163080] Dazed and confused, but trying to continue [2.163080] Uhhuh. NMI received for unknown reason 20 on CPU 0. [2.163080] Do you have a strange power saving mode enabled? [2.163080] Dazed and confused, but trying to continue [2.163080] Uhhuh. NMI received for unknown reason 30 on CPU 0. I'm able to reproduce to reproduce this consistently. L1 QEMU command-line: $ ps -ef | grep -i qemu qemu 4962 1 21 15:41 ?00:00:41 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name regular-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu Haswell,+vmx -m 6144 -smp 4,sockets=4,cores=1,threads=1 -uuid 4ed9ac0b-7f72-dfcf-68b3-e6fe2ac588b2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/regular-guest.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/home/test/vmimages/regular-guest.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:80:c1:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 L2 QEMU command-line: $ qemu 2042 1 0 May09 ?00:05:03 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name nested-guest -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid 02ea8988-1054-b08b-bafe-cfbe9659976c -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
On Fri, May 10, 2013 at 8:54 PM, Jan Kiszka jan.kis...@siemens.com wrote: On 2013-05-10 17:12, Jan Kiszka wrote: On 2013-05-10 15:00, Kashyap Chamarthy wrote: Heya, This is on Intel Haswell. First, some version info: L0, L1 -- both of them have same versions of kernel, qemu: = $ rpm -q kernel --changelog | head -2 * Thu May 09 2013 Josh Boyer - 3.10.0-0.rc0.git23.1 - Linux v3.9-11789-ge0fd9af Please recheck with kvm.git, next branch. Hmm, looks like your branch already contains the patch I was thinking of (03b28f8). Yes. You could try if leaving shadow VMCS off makes a difference, but I bet that is unrelated. Right. I could try. But, like you said, does it *really* make a difference. You get that backtrace in L1, correct? Yes. If you have any further tracing pointers, I could do some debugging. I'll have to see if I can reproduce it. Thanks. If you're looking for a clear reproducer, this is how I conducted my tests, and here's where I'm capturing all of the related work: [1] Setup -- https://raw.github.com/kashyapc/nvmx-haswell/master/SETUP-nVMX.rst [2] Simple scripts used to create L1 and L2 -- https://github.com/kashyapc/nvmx-haswell/tree/master/tests/scripts [3] Libvirt XMLs I used (for reference) -- https://github.com/kashyapc/nvmx-haswell/tree/master/tests/libvirt-xmls-for-l1-l2 Thanks in advance. /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
[3] Libvirt XMLs I used (for reference) -- https://github.com/kashyapc/nvmx-haswell/tree/master/tests/libvirt-xmls-for-l1-l2 Oops, forgot to add, here we go -- https://github.com/kashyapc/nvmx-haswell/tree/master/tests/libvirt-xmls-for-l1-l2 /kashyap -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
We know after you tried. I don't have access to a Haswell box, so we better exclude this beforehand. Fair enough. I'll try that too, and let you know. You get that backtrace in L1, correct? Yes. If you have any further tracing pointers, I could do some debugging. Thanks, I may come back to you if reproduction fails here. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is rebooted.
On Fri, May 10, 2013 at 11:39 PM, Jan Kiszka jan.kis...@siemens.com wrote: On 2013-05-10 19:40, Nakajima, Jun wrote: On Fri, May 10, 2013 at 9:33 AM, Jan Kiszka jan.kis...@siemens.com wrote: On 2013-05-10 17:39, Kashyap Chamarthy wrote: On Fri, May 10, 2013 at 8:54 PM, Jan Kiszka jan.kis...@siemens.com wrote: On 2013-05-10 17:12, Jan Kiszka wrote: On 2013-05-10 15:00, Kashyap Chamarthy wrote: Heya, This is on Intel Haswell. First, some version info: L0, L1 -- both of them have same versions of kernel, qemu: I tried to reproduce such a problem, and I found L2 (Linux) hangs in SeaBIOS, after line iPXE (http://ipxe.org) It happens with or w/o VMCS shadowing (and even without my virtual EPT patches). I didn't realize this problem until I updated the L1 kernel to the latest (e.g. 3.9.0) from 3.7.0. L0 uses the kvm.git, next branch. It's possible that the L1 kernel exposed a bug with the nested virtualization, as we saw such cases before. Hmm, no such issues here ATM although I'm on 3.9 for L1 as well. Interesting. But Jan, you're not using an Haswell machine, right ? Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Interested in working for GSoC project
On Sun, Apr 14, 2013 at 1:48 PM, Jan Kiszka jan.kis...@web.de wrote: On 2013-04-13 20:24, Mohit Dhingra wrote: Hi All, I am Mohit Dhingra, planning to work in GSoC project Improve nested virtualization support for x86. I got in touch with Jan (jan.kis...@web.de) few days back. As suggested by him, I am trying to write a simple unit test which tests the vmx functions. To start with, I have written a small code which checks whether vmx functionality is available on the current processor or not[1]. I have used cpuid to check the VMX functionality. Before executing cpuid, eax is set to 1. cpuid then puts the feature flags in ecx and edx registers. Bit5 in ecx register tells whether VMX feature is present or not. I am attaching the patch along with this mail. Hint regarding assembly usage: Try to minimize the number of instructions. Check, e.g., how the kernel implements cpuid. Result evaluation can perfectly be done in C. But there is a problem that I am facing. CPUID behaves differently when I run it using qemu-kvm, and when I run it directly in user mode. Please check the below output. It would be great if someone can please help me out. To enable nested VMX, you need to tweak two things: first, you need nested=1 as parameter for kvm_intel. And, second, you have to select a CPU type via QEMU (-cpu) that supports VMX or explicitly add that feature to the selected CPU (-cpu ...,+vmx). In addendum to Jan's above info, I posted some notes here while testing it on Fedora, see if its any helpful: - http://kashyapc.wordpress.com/tag/nested-virtualization/ - https://github.com/kashyapc/nested-virt-notes-intel-f18 By default, only core[2]duo and the synthetic KVM type host come with VMX enabled, not much more modern Intel CPU like Nehalem or SandyBridge. That's inconsistent, and if you want to fix something in QEMU, you could check which CPU types in qemu/target-i386/cpu.c lack the flag and post a patch to change this. Jan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm tools: fix SDL build error when libsdl isn't installed
On Mon, Sep 17, 2012 at 4:03 PM, Sasha Levin levinsasha...@gmail.com wrote:
We used wrong prototypes for sdl init/exit when libsdl wasn't installed when
building. This would cause build errors.
Reported-by: Kashyap Chamarthy kashyap...@gmail.com
Signed-off-by: Sasha Levin levinsasha...@gmail.com
---
tools/kvm/include/kvm/sdl.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/kvm/include/kvm/sdl.h b/tools/kvm/include/kvm/sdl.h
index b444333..19e1d74 100644
--- a/tools/kvm/include/kvm/sdl.h
+++ b/tools/kvm/include/kvm/sdl.h
@@ -9,11 +9,11 @@ struct framebuffer;
int sdl__init(struct kvm *kvm);
int sdl__exit(struct kvm *kvm);
#else
-static inline void sdl__init(struct kvm *kvm)
+static inline int sdl__init(struct kvm *kvm)
{
die(SDL support not compiled in. (install the SDL-dev[el] package));
}
-static inline void sdl__exit(struct kvm *kvm)
+static inline int sdl__exit(struct kvm *kvm)
{
die(SDL support not compiled in. (install the SDL-dev[el] package));
}
--
1.7.12
Just for info for to the list, this does not apply.
I'm testing the newer patch sent by Sasha.
Thanks.
/kashyap
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm tools: don't die if sdl wasn't compiled in and we don't try using it
On Mon, Sep 17, 2012 at 4:46 PM, Sasha Levin levinsasha...@gmail.com wrote:
If SDL isn't compiled in we shouldn't die unless we actually try using it.
Signed-off-by: Sasha Levin levinsasha...@gmail.com
---
tools/kvm/include/kvm/sdl.h | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/kvm/include/kvm/sdl.h b/tools/kvm/include/kvm/sdl.h
index 19e1d74..2f0c213 100644
--- a/tools/kvm/include/kvm/sdl.h
+++ b/tools/kvm/include/kvm/sdl.h
@@ -11,11 +11,17 @@ int sdl__exit(struct kvm *kvm);
#else
static inline int sdl__init(struct kvm *kvm)
{
- die(SDL support not compiled in. (install the SDL-dev[el] package));
+ if (kvm-cfg.sdl)
+ die(SDL support not compiled in. (install the SDL-dev[el]
package));
+
+ return 0;
}
static inline int sdl__exit(struct kvm *kvm)
{
- die(SDL support not compiled in. (install the SDL-dev[el] package));
+ if (kvm-cfg.sdl)
+ die(SDL support not compiled in. (install the SDL-dev[el]
package));
+
+ return 0;
}
#endif
--
1.7.12
Patch applies.
But when I run the 'lkvm' binary (after recompiling the kernel and
kvm tool) , I see this:
#---#
[kashyap@moon kvm]$ ./lkvm run -k ../../arch/x86/boot/bzImage
# lkvm run -k ../../arch/x86/boot/bzImage -m 448 -c 4 --name guest-17105
PPrroobbiinngg EE ((ee==oo ttoo ddiissaabbllee)).. ookk
.
.
.
[1.492530] host=192.168.33.15, domain=, nis-domain=(none)
[1.493443] bootserver=192.168.33.1, rootserver=0.0.0.0, rootpath=
[1.494458] ALSA device list:
[1.494901] No soundcards found.
[1.497106] VFS: Mounted root (9p filesystem) on device 0:13.
[1.498617] devtmpfs: mounted
[1.500555] Freeing unused kernel memory: 512k freed
[1.501981] Write protecting the kernel read-only data: 12288k
[1.511605] Freeing unused kernel memory: 1820k freed
[1.520470] Freeing unused kernel memory: 1576k freed
Mounting...
[1.533811] KGDB: BP remove failed: 8105212f
#---#
Any hints?
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm tools: don't die if sdl wasn't compiled in and we don't try using it
On Mon, Sep 17, 2012 at 5:14 PM, Kashyap Chamarthy kashyap...@gmail.com wrote:
On Mon, Sep 17, 2012 at 4:46 PM, Sasha Levin levinsasha...@gmail.com wrote:
If SDL isn't compiled in we shouldn't die unless we actually try using it.
Signed-off-by: Sasha Levin levinsasha...@gmail.com
---
tools/kvm/include/kvm/sdl.h | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/kvm/include/kvm/sdl.h b/tools/kvm/include/kvm/sdl.h
index 19e1d74..2f0c213 100644
--- a/tools/kvm/include/kvm/sdl.h
+++ b/tools/kvm/include/kvm/sdl.h
@@ -11,11 +11,17 @@ int sdl__exit(struct kvm *kvm);
#else
static inline int sdl__init(struct kvm *kvm)
{
- die(SDL support not compiled in. (install the SDL-dev[el]
package));
+ if (kvm-cfg.sdl)
+ die(SDL support not compiled in. (install the SDL-dev[el]
package));
+
+ return 0;
}
static inline int sdl__exit(struct kvm *kvm)
{
- die(SDL support not compiled in. (install the SDL-dev[el]
package));
+ if (kvm-cfg.sdl)
+ die(SDL support not compiled in. (install the SDL-dev[el]
package));
+
+ return 0;
}
#endif
--
1.7.12
Patch applies.
But when I run the 'lkvm' binary (after recompiling the kernel and
kvm tool) , I see this:
#---#
[kashyap@moon kvm]$ ./lkvm run -k ../../arch/x86/boot/bzImage
# lkvm run -k ../../arch/x86/boot/bzImage -m 448 -c 4 --name guest-17105
PPrroobbiinngg EE ((ee==oo ttoo ddiissaabbllee)).. ookk
.
.
.
[1.492530] host=192.168.33.15, domain=, nis-domain=(none)
[1.493443] bootserver=192.168.33.1, rootserver=0.0.0.0, rootpath=
[1.494458] ALSA device list:
[1.494901] No soundcards found.
[1.497106] VFS: Mounted root (9p filesystem) on device 0:13.
[1.498617] devtmpfs: mounted
[1.500555] Freeing unused kernel memory: 512k freed
[1.501981] Write protecting the kernel read-only data: 12288k
[1.511605] Freeing unused kernel memory: 1820k freed
[1.520470] Freeing unused kernel memory: 1576k freed
Mounting...
[1.533811] KGDB: BP remove failed: 8105212f
#---#
Any hints?
Discussed w/ Sasha on #pv. He pointed I had the directive
CONFIG_KGDB_TESTS_ON_BOOT set . (Silly me!)
Builds fine, once booted, guest shell access over a serial console is
presented successfully.
#---#
[kashyap@moon kvm]$ ./kvm run --balloon -k ../../arch/x86/boot/bzImage
# kvm run -k ../../arch/x86/boot/bzImage -m 448 -c 4 --name guest-4418
PPrroobbiinngg EE ((ee==oo ttoo ddiissaabbllee)).. ookk
Decompressing Linux... Parsing ELF... done.
Booting the kernel.
.
.
.
[1.466336] device=eth0, addr=192.168.33.15,
mask=255.255.255.0, gw=192.168.33.1
[1.467964] host=192.168.33.15, domain=, nis-domain=(none)
[1.469212] bootserver=192.168.33.1, rootserver=0.0.0.0, rootpath=
[1.470599] ALSA device list:
[1.471257] No soundcards found.
[1.473453] VFS: Mounted root (9p filesystem) on device 0:13.
[1.474961] devtmpfs: mounted
[1.477103] Freeing unused kernel memory: 512k freed
[1.478598] Write protecting the kernel read-only data: 12288k
[1.488213] Freeing unused kernel memory: 1820k freed
[1.497327] Freeing unused kernel memory: 1576k freed
Mounting...
sh-4.2#
#---#
ACK.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
On Wed, Apr 11, 2012 at 6:14 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: Hi, Nested virtualization on Intel does not work for me with qemu-kvm. As soon as the third layer OS (second virtualised) is starting the Linux kernel, the entire second layer freezes up. The last thing I can see console of the third layer system before it freezes is Decompressing Linux... . (no done, though). When starting without nofb option, the kernel still manages to set the screen resolution before freezing. Grub/Syslinux still work, but are extremely slow. Both the first layer OS (i.e. the one running on bare metal) and the second layer OS are 64-bit-Fedora 16 with Kernel 3.3.1-3.fc16.x86_64. On both the first and second layer OS, the kvm_intel modules are loaded with nested=Y parameter. (I've also tried with nested=N in the second layer. Didn't change anything.) Qemu-kvm was originally the Fedora-shipped 0.14, but I have since upgraded to 1.0. (Using rpmbuild with the specfile and patches from http://pkgs.fedoraproject.org/gitweb/?p=qemu.git;a=blob;f=qemu.spec;hb=HEAD) The second layer machine has this CPU specification in libvirt on the first layer OS: cpu mode='custom' match='exact' model fallback='allow'Nehalem/model feature policy='require' name='vmx'/ /cpu which results in this qemu commandline (from libvirt's logs): LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu- kvm -S -M pc-0.15 -cpu kvm64,+lahf_lm,+popcnt,+sse4.2,+sse4.1,+ssse3,+vmx - enable-kvm -m 8192 -smp 8,sockets=8,cores=1,threads=1 -name vshost1 -uuid 192b8c4b-0ded-07aa-2545-d7fef4cd897f -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vshost1.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown - no-acpi -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/data/vshost1.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio- disk0,bootindex=1 -drive file=/data/Fedora-16-x86_64- netinst.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw - device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=21,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:84:7d:46,bus=pci.0,addr=0x3 -netdev tap,fd=23,id=hostnet1,vhost=on,vhostfd=24 -device virtio-net- pci,netdev=hostnet1,id=net1,mac=52:54:00:84:8d:46,bus=pci.0,addr=0x4 -vnc 127.0.0.1:0,password -k de -vga cirrus -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x6 I have also tried some other combinations for the cpu element, like changing the model to core2duo and/or including all the features reported by libvirt's capabalities command. The third level machine does not have a cpu element in libvirt, and its commandline looks like this: LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu- kvm -S -M pc-0.14 -enable-kvm -m 8192 -smp 4,sockets=4,cores=1,threads=1 -name gentoo -uuid 3cdcc902-4520-df25-92ac-31ca5c707a50 -nodefconfig -nodefaults - chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/gentoo.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-acpi -drive file=/data/gentoo.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 - drive file=/data/install-amd64- minimal-20120223.iso,if=none,media=cdrom,id=drive- ide0-1-0,readonly=on,format=raw -device ide- drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:84:6d:46,bus=pci.0,addr=0x3 -usb -vnc 127.0.0.1:0,password -k de -vga cirrus -device virtio-balloon- pci,id=balloon0,bus=pci.0,addr=0x5 The third layer OS is a recent Gentoo minimal install (amd64), but somehow I don't think that matters at this point... The metal is a Dell PowerEdge R710 server with two Xeon E5520 CPUs. I've tried updating the machine's BIOS and other firmware to the latest version. That took a lot of time and a lot of searching on Dell websites, but didn't change anything. Does anyone have any idea what might be going wrong here or how I could debug this further? Interesting. I recently(a couple of months) ago with this configuration: == 1/ Physical Host (Host hypervisor/Bare metal) -- Config: Intel(R) Xeon(R) CPU(4 cores/socket); 10GB Memory; CPU Freq – 2GHz; Running latest Fedora-16(Minimal foot-print, @core only with Virt pkgs;x86_64; kernel-3.1.8-2.fc16.x86_64 2/ Regualr Guest (Or Guest Hypervisor) -- Config: 4GB Memory; 4vCPU; 20GB Raw disk image with cache =’none’ to have decent I/O; Minimal, @core F16; And same virt-packages as Physical Host; x86_64 3/ Nested Guest (Guest installed
Re: Nested virtualization on Intel does not work - second level freezes when third level is starting
On Wed, Apr 11, 2012 at 11:34 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: Am Mittwoch, 11. April 2012, 23:14:07 schrieb Kashyap Chamarthy: On Wed, Apr 11, 2012 at 6:14 PM, Guido Winkelmann guido-k...@thisisnotatest.de wrote: [...] Here is my complete notes on nested virtualization w/ Intel -- http://kashyapc.wordpress.com/2012/01/14/nested-virtualization-with-kvm-inte l/ I had already found you blog post via Google :). In fact, I used some of the things you wrote for setting up my system, in the hopes that I would have more luck. BTW, the line for modprobe.d/dist.conf should read options kvm_intel nested=1 Really? I wonder how it went that far then. Thanks for that, I'll make the correction and re-try it. And I see in your other email, it seemed to work for you (w/ the patch indicated in that email). Good to know. I'll give it a try sometime this weekend. In your blog post, the options keyword is missing. My result: I was able to ssh into the nested guest (guest installed inside the regular guest), but, after a reboot, the nested-guest loses the IP rendering it inaccessible.(Info: the regular-guest has a bridged IP, and nested-guest has a NATed IP) Refer the comments in the above post for some more discussion. Though I haven't tried the suggestion of 'updating your system firmware and disabling VT for Direct I/O Access if you are able in the firmware' . And I wonder how does turning it off can alleviate the prob. Yeah, I've seen that comment, that was what prompted me to update the server's firmware. The Dell BIOS does not offer the option to disable VT for Direct I/O Access, though. And my AMD notes is here(which was completely successful) -- http://kashyapc.wordpress.com/2012/01/18/nested-virtualization-with-kvm-and- amd/ Unfortunately, AMD is not an option for me, at least not in this particular context. Guido -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: virtio issues
On Wed, Aug 24, 2011 at 9:04 AM, arag...@dcsnow.com wrote: Hello all, I have just started working with KVM and virtualization in general. I have setup a client using Suse 9.3 x64 as the base and Fedora 15 x64 as the host. The kernel in the client has been upgraded to a custom 2.6.28.4 build. Everything seems to be working fine but I wanted to get some more disk performance so I'm trying to get virtio running. I don't care too much about the disk the base OS is running on but I have a drive that I'm trying to add that I want to use the virtio for. I can add the drive using IDE or SCSI drivers and it works fine (just slower had hoped). If, however, I switch to virtio, it doesn't show up in the client. If I used edit from virsh, I see: Just checking, after you edit the xml, did you redefine your xml to take effect ? virsh define /path/to/domainxml /kashyap domain type='kvm' nameMC1DB/name uuidf57c6236-2595-57ca-595b-aaaef4de87ea/uuid memory768/memory currentMemory768/currentMemory vcpu4/vcpu os type arch='x86_64' machine='pc-0.14'hvm/type boot dev='hd'/ bootmenu enable='no'/ /os features acpi/ apic/ pae/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashrestart/on_crash devices emulator/usr/bin/qemu-kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/img/mc1db.img'/ target dev='hda' bus='ide'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='block' device='cdrom' driver name='qemu' type='raw'/ source dev='/dev/sr0'/ target dev='hdc' bus='ide'/ readonly/ address type='drive' controller='0' bus='1' unit='0'/ /disk disk type='block' device='disk' driver name='qemu' type='raw'/ source dev='/dev/sda1'/ target dev='vda' bus='virtio'/ address type='pci' domain='0x' bus='0x00' slot='0x07' function='0x0'/ /disk controller type='ide' index='0' address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller controller type='scsi' index='0' address type='pci' domain='0x' bus='0x00' slot='0x06' function='0x0'/ /controller interface type='network' mac address='52:54:00:fc:de:e0'/ source network='default'/ address type='pci' domain='0x' bus='0x00' slot='0x03' function='0x0'/ /interface serial type='pty' target port='0'/ /serial console type='pty' target type='serial' port='0'/ /console input type='mouse' bus='ps2'/ graphics type='vnc' port='-1' autoport='yes'/ sound model='ich6' address type='pci' domain='0x' bus='0x00' slot='0x04' function='0x0'/ /sound video model type='cirrus' vram='9216' heads='1'/ address type='pci' domain='0x' bus='0x00' slot='0x02' function='0x0'/ /video memballoon model='virtio' address type='pci' domain='0x' bus='0x00' slot='0x05' function='0x0'/ /memballoon /devices /domain Can someone tell me what I'm doing wrong? I'm stumped. And would it make a difference if I use virtio for the base disk also? -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Can VM be copied to another PC
On Mon, Feb 1, 2010 at 9:32 AM, sati...@pacific.net.hk wrote: Hi folks, Host – Fedora 12 64bit KVM – virtualization software I have 8 VMs on this virtual machine running different OS. Can VMs be moved/copied to another PC of similar hardware config running Fedora 12 (64bit) as host and KVM The below worked for me between two Fedora 12 hosts. I used virsh(comes with libvirt-client) 1) 'virsh dumpxml domain(s) (or) id '(this will dump the guest machine info. into XML) 2) copy your qcow2(or whatever format) image files from host1 to host2(which of similar config as host1) 3) 'virsh define xml file' for the dumped xml files from step 1) (for each of the guests) 4) Now you should be good to start/run the guests on your host2 running KVM Your milage may vary. See virsh manpages for more. If Yes, whether copying following files, e.g. copying following files and paste them on the same /path/to/directory # ls -l /etc/libvirt/qemu/ total 52 ... -rw--- 1 root root 1293 2010-01-14 22:48 vm01.xml -rw--- 1 root root 1293 2010-01-11 17:19 vm02.xml -rw--- 1 root root 1302 2010-01-11 19:11 vm03_ub9164.xml -rw--- 1 root root 1273 2010-01-12 02:09 vm04_f1264.xml -rw--- 1 root root 1193 2010-01-23 02:21 vm05_wser08.xml -rw--- 1 root root 1368 2010-01-25 22:44 vm06_wser08.xml -rw--- 1 root root 1389 2010-01-27 11:04 vm07_wser0832.xml -rw--- 1 root root 1370 2010-01-27 22:23 vm08_vista32.xml ... # ls -l /home/satimis/ | grep vm -rwxr-xr-x 1 root root 12884901888 2010-01-22 07:56 vm03_9164.qcow2 # ls -l /home/satimis/VM/ | grep vm -rwxr-xr-x 1 root root 12884901888 2010-01-23 08:19 vm01.qcow2 -rwxr-xr-x 1 root root 12884901888 2010-01-23 08:16 vm02.qcow2 -rwxr-xr-x 1 root root 12884901888 2010-01-26 22:33 vm04_f1264.qcow2 # ls -l /home/satimis/VM/VMWIN/ | grep vm -rwxr-xr-x 1 root root 32212254720 2010-01-23 06:55 vm05_wser08.qcow2 -rwxr-xr-x 1 root root 32212254720 2010-01-27 23:07 vm06_wser08.qcow2 -rwxr-xr-x 1 root root 32212254720 2010-01-27 12:45 vm07_wser0832.qcow2 -rwxr-xr-x 1 root root 32212254720 2010-01-28 00:14 vm08_vista32.qcow2 What about if the host is running different OS, say Debian etc. can it also work? haven't tried this. /kashyap TIA B.R. Stephen L -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
