Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Jan Kiszka wrote:
-
- if (!npt_enabled event_injection)
- kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ else {
+ if (svm-vcpu.arch.interrupt.pending ||
+ svm-vcpu.arch.exception.pending)
+ kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ }
Without understanding yet why kvm_mmu_unprotect_page_virt is required
here,
That is needed in case interrupt injection failed because the process of
injecting the interrupt (usually pushing data on the stack) writes to a
write-protected page table.
I guess we need nmi_pending here as well.
--
Do not meddle in the internals of kernels, for they are subtle and quick to
panic.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Sat, Apr 18, 2009 at 07:28:20PM +0300, Gleb Natapov wrote:
So this patch may either expose a bug in the svm emulation of qemu or
comes with a subtle regression that only triggers due to qemu's timing.
This needs to be understood. Gleb, any progress on reproducing it on
your side?
I reproduced it and I am debugging it. In my case the boot hangs on sti;hlt
sequence. Instrumentation thus far shows that at this point interrupts no
longer
injected because ppr value is too big. Need to see why, but tpr handling
is not complete in qemu svm. May be this is the reason. Will know more
tomorrow.
I've looked into this and my conclusion is that if you are not going to
develop SVM in qemu don't use it just yet. QEMU doesn't handle exceptions
during event injection properly. Actually it does not handle it at all,
so if PF happens during interrupt injection interrupt is lost and, what
worse, is never acked. If interrupt was high prio it blocks all other
interrupts.
The patch below adds exception handling during event injection. Valid
flag removed from EVENTINJ only after successful injection and EVENTINJ
is copied to EXITINTINFO on exit. Can you give it a try?
And this is not the only problem I saw, but the one that caused my guest
to hang.
diff --git a/target-i386/op_helper.c b/target-i386/op_helper.c
index be09263..9264afd 100644
--- a/target-i386/op_helper.c
+++ b/target-i386/op_helper.c
@@ -1249,6 +1249,10 @@ void do_interrupt(int intno, int is_int, int error_code,
} else {
do_interrupt_real(intno, is_int, error_code, next_eip);
}
+if (env-hflags HF_SVMI_MASK) {
+ uint32_t event_inj = ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.event_inj));
+ stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj),
event_inj ~SVM_EVTINJ_VALID);
+}
}
/* This should come from sysemu.h - if we could include it here... */
@@ -4994,7 +4998,6 @@ void helper_vmrun(int aflag, int next_eip_addend)
uint8_t vector = event_inj SVM_EVTINJ_VEC_MASK;
uint16_t valid_err = event_inj SVM_EVTINJ_VALID_ERR;
uint32_t event_inj_err = ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.event_inj_err));
-stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj),
event_inj ~SVM_EVTINJ_VALID);
qemu_log_mask(CPU_LOG_TB_IN_ASM, Injecting(%#hx): , valid_err);
/* FIXME: need to implement valid_err */
@@ -5331,6 +5334,8 @@ void helper_vmexit(uint32_t exit_code, uint64_t
exit_info_1)
cpu_x86_set_cpl(env, 0);
stq_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_code),
exit_code);
stq_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_info_1),
exit_info_1);
+stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_int_info),
ldl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj)));
+stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_int_info_err),
ldl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj_err)));
env-hflags2 = ~HF2_GIF_MASK;
/* FIXME: Resets the current ASID register to zero (host ASID). */
--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote:
On Sat, Apr 18, 2009 at 07:28:20PM +0300, Gleb Natapov wrote:
So this patch may either expose a bug in the svm emulation of qemu or
comes with a subtle regression that only triggers due to qemu's timing.
This needs to be understood. Gleb, any progress on reproducing it on
your side?
I reproduced it and I am debugging it. In my case the boot hangs on sti;hlt
sequence. Instrumentation thus far shows that at this point interrupts no
longer
injected because ppr value is too big. Need to see why, but tpr handling
is not complete in qemu svm. May be this is the reason. Will know more
tomorrow.
I've looked into this and my conclusion is that if you are not going to
develop SVM in qemu don't use it just yet.
We had a resource conflict regarding SVM capable AMD boxes and a tight
schedule, so we decided to pick qemu as initial development platform.
Turns out that this has was a bit too optimistic. :)
QEMU doesn't handle exceptions
during event injection properly. Actually it does not handle it at all,
so if PF happens during interrupt injection interrupt is lost and, what
worse, is never acked. If interrupt was high prio it blocks all other
interrupts.
The patch below adds exception handling during event injection. Valid
flag removed from EVENTINJ only after successful injection and EVENTINJ
is copied to EXITINTINFO on exit. Can you give it a try?
Ah, great, thanks. Will test.
And this is not the only problem I saw, but the one that caused my guest
to hang.
OK, good to know. I added Alex (though he's said to be on vacation ATM)
and qemu to CC. Maybe you can quickly list the other issues you've
stumbled over, for the records and for motivating contributors...
diff --git a/target-i386/op_helper.c b/target-i386/op_helper.c
index be09263..9264afd 100644
--- a/target-i386/op_helper.c
+++ b/target-i386/op_helper.c
@@ -1249,6 +1249,10 @@ void do_interrupt(int intno, int is_int, int
error_code,
} else {
do_interrupt_real(intno, is_int, error_code, next_eip);
}
+if (env-hflags HF_SVMI_MASK) {
+ uint32_t event_inj = ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.event_inj));
+ stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj),
event_inj ~SVM_EVTINJ_VALID);
+}
}
/* This should come from sysemu.h - if we could include it here... */
@@ -4994,7 +4998,6 @@ void helper_vmrun(int aflag, int next_eip_addend)
uint8_t vector = event_inj SVM_EVTINJ_VEC_MASK;
uint16_t valid_err = event_inj SVM_EVTINJ_VALID_ERR;
uint32_t event_inj_err = ldl_phys(env-vm_vmcb + offsetof(struct
vmcb, control.event_inj_err));
-stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj),
event_inj ~SVM_EVTINJ_VALID);
qemu_log_mask(CPU_LOG_TB_IN_ASM, Injecting(%#hx): , valid_err);
/* FIXME: need to implement valid_err */
@@ -5331,6 +5334,8 @@ void helper_vmexit(uint32_t exit_code, uint64_t
exit_info_1)
cpu_x86_set_cpl(env, 0);
stq_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_code),
exit_code);
stq_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_info_1),
exit_info_1);
+stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_int_info),
ldl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj)));
+stl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.exit_int_info_err), ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.event_inj_err)));
env-hflags2 = ~HF2_GIF_MASK;
/* FIXME: Resets the current ASID register to zero (host ASID). */
--
Gleb.
Thanks again,
Jan
signature.asc
Description: OpenPGP digital signature
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Sun, Apr 19, 2009 at 04:05:21PM +0200, Jan Kiszka wrote:
And this is not the only problem I saw, but the one that caused my guest
to hang.
OK, good to know. I added Alex (though he's said to be on vacation ATM)
and qemu to CC. Maybe you can quickly list the other issues you've
stumbled over, for the records and for motivating contributors...
Another one that I remember (because this was my first suspect) is
interrupt shadow handling. HF_INHIBIT_IRQ_MASK is cleared on exit when
shadow bit is set in int_state and is not set on entry if hypervisor
set shadow bit by itself. I am not sure how real HW actually handles this,
but patch below demonstrates how I think it does it :) And of cause
comments like /* FIXME: this should respect TPR */ don't look promising.
diff --git a/target-i386/op_helper.c b/target-i386/op_helper.c
index be09263..691a7f0 100644
--- a/target-i386/op_helper.c
+++ b/target-i386/op_helper.c
@@ -4971,6 +4997,15 @@ void helper_vmrun(int aflag, int next_eip_addend)
env-dr[6] = ldq_phys(env-vm_vmcb + offsetof(struct vmcb, save.dr6));
cpu_x86_set_cpl(env, ldub_phys(env-vm_vmcb + offsetof(struct vmcb,
save.cpl)));
+{
+ uint32_t aaa;
+aaa = ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.int_state));
+ if (aaa SVM_INTERRUPT_SHADOW_MASK)
+ helper_set_inhibit_irq();
+ else
+ helper_reset_inhibit_irq();
+}
+
/* FIXME: guest state consistency checks */
switch(ldub_phys(env-vm_vmcb + offsetof(struct vmcb, control.tlb_ctl))) {
@@ -5243,7 +5280,6 @@ void helper_vmexit(uint32_t exit_code, uint64_t
exit_info_1)
if(env-hflags HF_INHIBIT_IRQ_MASK) {
stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.int_state),
SVM_INTERRUPT_SHADOW_MASK);
-env-hflags = ~HF_INHIBIT_IRQ_MASK;
} else {
stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.int_state), 0);
}
--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Jan Kiszka wrote:
Gleb Natapov wrote:
On Sat, Apr 18, 2009 at 07:28:20PM +0300, Gleb Natapov wrote:
So this patch may either expose a bug in the svm emulation of qemu or
comes with a subtle regression that only triggers due to qemu's timing.
This needs to be understood. Gleb, any progress on reproducing it on
your side?
I reproduced it and I am debugging it. In my case the boot hangs on sti;hlt
sequence. Instrumentation thus far shows that at this point interrupts no
longer
injected because ppr value is too big. Need to see why, but tpr handling
is not complete in qemu svm. May be this is the reason. Will know more
tomorrow.
I've looked into this and my conclusion is that if you are not going to
develop SVM in qemu don't use it just yet.
We had a resource conflict regarding SVM capable AMD boxes and a tight
schedule, so we decided to pick qemu as initial development platform.
Turns out that this has was a bit too optimistic. :)
QEMU doesn't handle exceptions
during event injection properly. Actually it does not handle it at all,
so if PF happens during interrupt injection interrupt is lost and, what
worse, is never acked. If interrupt was high prio it blocks all other
interrupts.
The patch below adds exception handling during event injection. Valid
flag removed from EVENTINJ only after successful injection and EVENTINJ
is copied to EXITINTINFO on exit. Can you give it a try?
Ah, great, thanks. Will test.
I can confirm: patch below makes my kvm-in-qemu test case happy, too.
Maybe you want to post this with changelog and signed-off to qemu-devel.
Jan
diff --git a/target-i386/op_helper.c b/target-i386/op_helper.c
index be09263..9264afd 100644
--- a/target-i386/op_helper.c
+++ b/target-i386/op_helper.c
@@ -1249,6 +1249,10 @@ void do_interrupt(int intno, int is_int, int
error_code,
} else {
do_interrupt_real(intno, is_int, error_code, next_eip);
}
+if (env-hflags HF_SVMI_MASK) {
+uint32_t event_inj = ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.event_inj));
+stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj),
event_inj ~SVM_EVTINJ_VALID);
+}
}
/* This should come from sysemu.h - if we could include it here... */
@@ -4994,7 +4998,6 @@ void helper_vmrun(int aflag, int next_eip_addend)
uint8_t vector = event_inj SVM_EVTINJ_VEC_MASK;
uint16_t valid_err = event_inj SVM_EVTINJ_VALID_ERR;
uint32_t event_inj_err = ldl_phys(env-vm_vmcb + offsetof(struct
vmcb, control.event_inj_err));
-stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj),
event_inj ~SVM_EVTINJ_VALID);
qemu_log_mask(CPU_LOG_TB_IN_ASM, Injecting(%#hx): , valid_err);
/* FIXME: need to implement valid_err */
@@ -5331,6 +5334,8 @@ void helper_vmexit(uint32_t exit_code, uint64_t
exit_info_1)
cpu_x86_set_cpl(env, 0);
stq_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_code),
exit_code);
stq_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_info_1),
exit_info_1);
+stl_phys(env-vm_vmcb + offsetof(struct vmcb, control.exit_int_info),
ldl_phys(env-vm_vmcb + offsetof(struct vmcb, control.event_inj)));
+stl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.exit_int_info_err), ldl_phys(env-vm_vmcb + offsetof(struct vmcb,
control.event_inj_err)));
env-hflags2 = ~HF2_GIF_MASK;
/* FIXME: Resets the current ASID register to zero (host ASID). */
--
Gleb.
signature.asc
Description: OpenPGP digital signature
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Sun, Apr 19, 2009 at 05:06:29PM +0200, Jan Kiszka wrote: Jan Kiszka wrote: Gleb Natapov wrote: On Sat, Apr 18, 2009 at 07:28:20PM +0300, Gleb Natapov wrote: So this patch may either expose a bug in the svm emulation of qemu or comes with a subtle regression that only triggers due to qemu's timing. This needs to be understood. Gleb, any progress on reproducing it on your side? I reproduced it and I am debugging it. In my case the boot hangs on sti;hlt sequence. Instrumentation thus far shows that at this point interrupts no longer injected because ppr value is too big. Need to see why, but tpr handling is not complete in qemu svm. May be this is the reason. Will know more tomorrow. I've looked into this and my conclusion is that if you are not going to develop SVM in qemu don't use it just yet. We had a resource conflict regarding SVM capable AMD boxes and a tight schedule, so we decided to pick qemu as initial development platform. Turns out that this has was a bit too optimistic. :) QEMU doesn't handle exceptions during event injection properly. Actually it does not handle it at all, so if PF happens during interrupt injection interrupt is lost and, what worse, is never acked. If interrupt was high prio it blocks all other interrupts. The patch below adds exception handling during event injection. Valid flag removed from EVENTINJ only after successful injection and EVENTINJ is copied to EXITINTINFO on exit. Can you give it a try? Ah, great, thanks. Will test. I can confirm: patch below makes my kvm-in-qemu test case happy, too. Maybe you want to post this with changelog and signed-off to qemu-devel. Yeah, I'll reformat and submit. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote:
Start to use interrupt/exception queues like VMX does.
This also fix the bug that if exit was caused by a guest
internal exception access to IDT the exception was not
reinjected.
Signed-off-by: Gleb Natapov g...@redhat.com
---
arch/x86/kvm/svm.c | 176
++--
1 files changed, 75 insertions(+), 101 deletions(-)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 52c41aa..053370d 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO);
static int nested = 0;
module_param(nested, int, S_IRUGO);
-static void kvm_reput_irq(struct vcpu_svm *svm);
static void svm_flush_tlb(struct kvm_vcpu *vcpu);
static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override);
@@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu,
unsigned nr,
static bool svm_exception_injected(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
-
- return !(svm-vmcb-control.exit_int_info SVM_EXITINTINFO_VALID);
+ return false;
}
static int is_external_interrupt(u32 info)
@@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct
kvm_guest_debug *dbg)
static int svm_get_irq(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
-
- if (is_external_interrupt(exit_int_info))
- return exit_int_info SVM_EVTINJ_VEC_MASK;
- return -1;
+ if (!vcpu-arch.interrupt.pending)
+ return -1;
+ return vcpu-arch.interrupt.nr;
}
static void load_host_msrs(struct kvm_vcpu *vcpu)
@@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr,
unsigned long value,
static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
{
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
- struct kvm *kvm = svm-vcpu.kvm;
u64 fault_address;
u32 error_code;
- bool event_injection = false;
-
- if (!irqchip_in_kernel(kvm)
- is_external_interrupt(exit_int_info)) {
- event_injection = true;
- kvm_push_irq(svm-vcpu, exit_int_info SVM_EVTINJ_VEC_MASK);
- }
fault_address = svm-vmcb-control.exit_info_2;
error_code = svm-vmcb-control.exit_info_1;
@@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run)
*/
if (npt_enabled)
svm_flush_tlb(svm-vcpu);
-
- if (!npt_enabled event_injection)
- kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ else {
+ if (svm-vcpu.arch.interrupt.pending ||
+ svm-vcpu.arch.exception.pending)
+ kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ }
Without understanding yet why kvm_mmu_unprotect_page_virt is required
here, this looks like it is lacking '|| svm-vcpu.arch.nmi_injected'.
Interrupts and exceptions are re-queued on fault-during-injection,
therefore they are now pending again, right?
Jan
signature.asc
Description: OpenPGP digital signature
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. OK. On real SVM HW this seems to work. However now i'm stumbled upon another problem wrt. NMI. See another mail. I can confirm that this series (probably this patch) breaks the following scenario: qemu-system-x86_64 (qemu trunk, emulation mode) linux (kvm.git) with kvm-amd + kvm-userspace linux (the same kvm.git) The second-level guest hangs in the mid of its boot, maybe no longer receiving some or any interrupt. So this patch may either expose a bug in the svm emulation of qemu or comes with a subtle regression that only triggers due to qemu's timing. This needs to be understood. Gleb, any progress on reproducing it on your side? Jan signature.asc Description: OpenPGP digital signature
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Sat, Apr 18, 2009 at 11:05:10AM +0200, Jan Kiszka wrote:
Gleb Natapov wrote:
Start to use interrupt/exception queues like VMX does.
This also fix the bug that if exit was caused by a guest
internal exception access to IDT the exception was not
reinjected.
Signed-off-by: Gleb Natapov g...@redhat.com
---
arch/x86/kvm/svm.c | 176
++--
1 files changed, 75 insertions(+), 101 deletions(-)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 52c41aa..053370d 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO);
static int nested = 0;
module_param(nested, int, S_IRUGO);
-static void kvm_reput_irq(struct vcpu_svm *svm);
static void svm_flush_tlb(struct kvm_vcpu *vcpu);
static int nested_svm_exit_handled(struct vcpu_svm *svm, bool
kvm_override);
@@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu,
unsigned nr,
static bool svm_exception_injected(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
-
- return !(svm-vmcb-control.exit_int_info SVM_EXITINTINFO_VALID);
+ return false;
}
static int is_external_interrupt(u32 info)
@@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu,
struct kvm_guest_debug *dbg)
static int svm_get_irq(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
-
- if (is_external_interrupt(exit_int_info))
- return exit_int_info SVM_EVTINJ_VEC_MASK;
- return -1;
+ if (!vcpu-arch.interrupt.pending)
+ return -1;
+ return vcpu-arch.interrupt.nr;
}
static void load_host_msrs(struct kvm_vcpu *vcpu)
@@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int
dr, unsigned long value,
static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
{
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
- struct kvm *kvm = svm-vcpu.kvm;
u64 fault_address;
u32 error_code;
- bool event_injection = false;
-
- if (!irqchip_in_kernel(kvm)
- is_external_interrupt(exit_int_info)) {
- event_injection = true;
- kvm_push_irq(svm-vcpu, exit_int_info SVM_EVTINJ_VEC_MASK);
- }
fault_address = svm-vmcb-control.exit_info_2;
error_code = svm-vmcb-control.exit_info_1;
@@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run)
*/
if (npt_enabled)
svm_flush_tlb(svm-vcpu);
-
- if (!npt_enabled event_injection)
- kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ else {
+ if (svm-vcpu.arch.interrupt.pending ||
+ svm-vcpu.arch.exception.pending)
+ kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ }
Without understanding yet why kvm_mmu_unprotect_page_virt is required
here, this looks like it is lacking '|| svm-vcpu.arch.nmi_injected'.
Interrupts and exceptions are re-queued on fault-during-injection,
therefore they are now pending again, right?
Yes right, and we discussed this with Avi already. I'll send another
patch after this series will be applied (exactly the same issue exists
for VMX btw).
--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Sat, Apr 18, 2009 at 11:16:47AM +0200, Jan Kiszka wrote: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. OK. On real SVM HW this seems to work. However now i'm stumbled upon another problem wrt. NMI. See another mail. I can confirm that this series (probably this patch) breaks the following scenario: qemu-system-x86_64 (qemu trunk, emulation mode) linux (kvm.git) with kvm-amd + kvm-userspace linux (the same kvm.git) The second-level guest hangs in the mid of its boot, maybe no longer receiving some or any interrupt. So this patch may either expose a bug in the svm emulation of qemu or comes with a subtle regression that only triggers due to qemu's timing. This needs to be understood. Gleb, any progress on reproducing it on your side? I reproduced it and I am debugging it. In my case the boot hangs on sti;hlt sequence. Instrumentation thus far shows that at this point interrupts no longer injected because ppr value is too big. Need to see why, but tpr handling is not complete in qemu svm. May be this is the reason. Will know more tomorrow. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote:
Start to use interrupt/exception queues like VMX does.
This also fix the bug that if exit was caused by a guest
internal exception access to IDT the exception was not
reinjected.
Signed-off-by: Gleb Natapov g...@redhat.com
---
arch/x86/kvm/svm.c | 176
++--
1 files changed, 75 insertions(+), 101 deletions(-)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 52c41aa..053370d 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO);
static int nested = 0;
module_param(nested, int, S_IRUGO);
-static void kvm_reput_irq(struct vcpu_svm *svm);
static void svm_flush_tlb(struct kvm_vcpu *vcpu);
static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override);
@@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu,
unsigned nr,
static bool svm_exception_injected(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
-
- return !(svm-vmcb-control.exit_int_info SVM_EXITINTINFO_VALID);
+ return false;
}
static int is_external_interrupt(u32 info)
@@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct
kvm_guest_debug *dbg)
static int svm_get_irq(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
-
- if (is_external_interrupt(exit_int_info))
- return exit_int_info SVM_EVTINJ_VEC_MASK;
- return -1;
+ if (!vcpu-arch.interrupt.pending)
+ return -1;
+ return vcpu-arch.interrupt.nr;
}
static void load_host_msrs(struct kvm_vcpu *vcpu)
@@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr,
unsigned long value,
static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
{
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
- struct kvm *kvm = svm-vcpu.kvm;
u64 fault_address;
u32 error_code;
- bool event_injection = false;
-
- if (!irqchip_in_kernel(kvm)
- is_external_interrupt(exit_int_info)) {
- event_injection = true;
- kvm_push_irq(svm-vcpu, exit_int_info SVM_EVTINJ_VEC_MASK);
- }
fault_address = svm-vmcb-control.exit_info_2;
error_code = svm-vmcb-control.exit_info_1;
@@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run)
*/
if (npt_enabled)
svm_flush_tlb(svm-vcpu);
-
- if (!npt_enabled event_injection)
- kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ else {
+ if (svm-vcpu.arch.interrupt.pending ||
+ svm-vcpu.arch.exception.pending)
+ kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ }
return kvm_mmu_page_fault(svm-vcpu, fault_address, error_code);
}
@@ -2187,7 +2174,6 @@ static int handle_exit(struct kvm_run *kvm_run, struct
kvm_vcpu *vcpu)
}
}
- kvm_reput_irq(svm);
if (svm-vmcb-control.exit_code == SVM_EXIT_ERR) {
kvm_run-exit_reason = KVM_EXIT_FAIL_ENTRY;
@@ -2289,98 +2275,47 @@ static int svm_interrupt_allowed(struct kvm_vcpu
*vcpu)
(svm-vcpu.arch.hflags HF_GIF_MASK);
}
-static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
+static void enable_irq_window(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
- struct vmcb *vmcb = svm-vmcb;
- int intr_vector = -1;
-
- if ((vmcb-control.exit_int_info SVM_EVTINJ_VALID)
- ((vmcb-control.exit_int_info SVM_EVTINJ_TYPE_MASK) == 0)) {
- intr_vector = vmcb-control.exit_int_info
- SVM_EVTINJ_VEC_MASK;
- vmcb-control.exit_int_info = 0;
- svm_inject_irq(svm, intr_vector);
- goto out;
- }
-
- if (vmcb-control.int_ctl V_IRQ_MASK)
- goto out;
-
- if (!kvm_cpu_has_interrupt(vcpu))
- goto out;
-
- if (nested_svm_intr(svm))
- goto out;
-
- if (!(svm-vcpu.arch.hflags HF_GIF_MASK))
- goto out;
-
- if (!(vmcb-save.rflags X86_EFLAGS_IF) ||
- (vmcb-control.int_state SVM_INTERRUPT_SHADOW_MASK) ||
- (vmcb-control.event_inj SVM_EVTINJ_VALID)) {
- /* unable to deliver irq, set pending irq */
- svm_set_vintr(svm);
- svm_inject_irq(svm, 0x0);
- goto out;
- }
- /* Okay, we can deliver the interrupt: grab it and update PIC state. */
- intr_vector = kvm_cpu_get_interrupt(vcpu);
- svm_inject_irq(svm, intr_vector);
-out:
- update_cr8_intercept(vcpu);
+ svm_set_vintr(to_svm(vcpu));
+ svm_inject_irq(to_svm(vcpu), 0x0);
}
-static void kvm_reput_irq(struct
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Jan Kiszka wrote:
Gleb Natapov wrote:
+switch (type) {
+case SVM_EXITINTINFO_TYPE_NMI:
+svm-vcpu.arch.nmi_injected = true;
+break;
+case SVM_EXITINTINFO_TYPE_EXEPT:
+if (exitintinfo SVM_EXITINTINFO_VALID_ERR) {
+u32 err = svm-vmcb-control.exit_int_info_err;
+kvm_queue_exception_e(svm-vcpu, vector, err);
+
stgit remarked trailing whitespace. checkpatch.pl would probably do
so, too. :)
And there is another one in patch 14.
Sorry, the latter one was due to /me using the old version.
Jan
--
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. OK. On real SVM HW this seems to work. However now i'm stumbled upon another problem wrt. NMI. See another mail. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Tue, Apr 14, 2009 at 10:41:03PM +0300, Gleb Natapov wrote: Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo I just noticed that my kernel is different. Will install 2.6.26 and retest, but provide me yours anyway. 2.6.26-2-amd64 works for me too. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. qemu-64:~# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 2 model name : QEMU Virtual CPU version 0.10.2 stepping: 3 cpu MHz : 1828.754 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow up pni svm bogomips: 3700.32 TLB size: 1024 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: I just noticed that my kernel is different. Will install 2.6.26 and retest, but provide me yours anyway. The whole serie doesn't work too (that's why I started bisecting). And BTW, I got the same results with -no-kvm-irqchip -- Gleb. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. qemu-64:~# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 2 model name : QEMU Virtual CPU version 0.10.2 stepping: 3 cpu MHz : 1828.754 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow up pni svm bogomips: 3700.32 TLB size: 1024 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: I need _host_ cpu info. Do the same on the host please. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. qemu-64:~# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 2 model name : QEMU Virtual CPU version 0.10.2 stepping: 3 cpu MHz : 1828.754 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow up pni svm bogomips: 3700.32 TLB size: 1024 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: I need _host_ cpu info. Do the same on the host please. That _is_ his host - qemu in emulation mode (ie. nested virtualization). Maybe there is an issue with qemu's emulation of svm or, rather, with the apic emulation. The fact that he has to boot the first-level guest with noapic is fairly suspicious. Dmitry, what is your first level-guest distro/kernel, also Lenny? And what is the top-level qemu command line? Let's focus on this first, leaving KVM and this patch series aside for a while. Jan -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Wed, Apr 15, 2009 at 12:22:34PM +0200, Jan Kiszka wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. qemu-64:~# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 2 model name : QEMU Virtual CPU version 0.10.2 stepping: 3 cpu MHz : 1828.754 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow up pni svm bogomips: 3700.32 TLB size: 1024 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: I need _host_ cpu info. Do the same on the host please. That _is_ his host - qemu in emulation mode (ie. nested virtualization). Ah, now I noticed svm in cpu flags. Does qemu support svm in TCG? Maybe there is an issue with qemu's emulation of svm or, rather, with the apic emulation. The fact that he has to boot the first-level guest with noapic is fairly suspicious. Dmitry, what is your first level-guest distro/kernel, also Lenny? And what is the top-level qemu command line? Let's focus on this first, leaving KVM and this patch series aside for a while. If KVM runs inside a guest that is definitely a good idea :) -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote: On Wed, Apr 15, 2009 at 12:22:34PM +0200, Jan Kiszka wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. qemu-64:~# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 2 model name : QEMU Virtual CPU version 0.10.2 stepping: 3 cpu MHz : 1828.754 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow up pni svm bogomips: 3700.32 TLB size: 1024 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: I need _host_ cpu info. Do the same on the host please. That _is_ his host - qemu in emulation mode (ie. nested virtualization). Ah, now I noticed svm in cpu flags. Does qemu support svm in TCG? Yes, and KVM seems to have been fine without the patch. But that may not exclude remaining bugs in QEMU (as first-level hypervisor here). On the other hand, it wouldn't be the first time QEMU, with its extreme delays, triggers some nasty race in its guest... Maybe there is an issue with qemu's emulation of svm or, rather, with the apic emulation. The fact that he has to boot the first-level guest with noapic is fairly suspicious. Dmitry, what is your first level-guest distro/kernel, also Lenny? And what is the top-level qemu command line? Let's focus on this first, leaving KVM and this patch series aside for a while. If KVM runs inside a guest that is definitely a good idea :) -- Gleb. Jan -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Wed, Apr 15, 2009 at 12:51:00PM +0200, Jan Kiszka wrote: I need _host_ cpu info. Do the same on the host please. That _is_ his host - qemu in emulation mode (ie. nested virtualization). Ah, now I noticed svm in cpu flags. Does qemu support svm in TCG? Yes, and KVM seems to have been fine without the patch. But that may not exclude remaining bugs in QEMU (as first-level hypervisor here). On the other hand, it wouldn't be the first time QEMU, with its extreme delays, triggers some nasty race in its guest... It doesn't look like race to me. The failure is 100% reproducible. I'll try to reproduce locally and see what is going on. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). system inside qemu: 64-bit debian lenny KVM userspace: debian 84+dfsg-2 inside kvm I run 32-bit debian lenny with plain debian 2.6.26 kernel. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Jan -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Dmitry Eremin-Solenikov пишет: Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log Moreover, using bios from 0.10.2 I can't boot linux even with noapic: ACPI: PM-Timer IO Port: 0xb008 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ACPI: Skipping IOAPIC probe due to 'noapic' option. Using ACPI for processor (LAPIC) configuration information ACPI: HPET id: 0x8086a201 base: 0xfed0 Intel MultiProcessor Specification v1.4 MPTABLE: OEM ID: QEMUCPU MPTABLE: Product ID: 0.1 MPTABLE: APIC at: 0xFEE0 I/O APIC #1 Version 17 at 0xFEC0. Processors: 1 SMP: Allowing 1 CPUs, 0 hotplug CPUs Allocating PCI resources starting at 2000 (gap: 1000:effc) NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 25 pages at 880001033000, static data 70880 bytes Built 1 zonelists in Zone order, mobility grouping on. Total pages: 62771 Kernel command line: root=/dev/sda1 ro console=ttyS0 noapic Initializing CPU#0 NR_IRQS:512 PID hash table entries: 1024 (order: 10, 8192 bytes) Fast TSC calibration using PIT Detected 1828.371 MHz processor. Console: colour VGA+ 80x25 console [ttyS0] enabled Dentry cache hash table entries: 32768 (order: 6, 262144 bytes) Inode-cache hash table entries: 16384 (order: 5, 131072 bytes) Checking aperture... No AGP bridge found Memory: 249848k/262080k available (4048k kernel code, 388k absent, 11528k reserved, 1626k data, 436k init) SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Calibrating delay loop (skipped), value calculated using timer frequency.. 3656.74 BogoMIPS (lpj=7313484) Mount-cache hash table entries: 256 CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 512K (64 bytes/line) SMP alternatives: switching to UP code Freeing SMP alternatives: 29k freed ACPI: Core revision 20081204 ACPI: setting ELCR to 0200 (from 0a00) Setting APIC routing to flat CPU0: AMD QEMU Virtual CPU version 0.10.2 stepping 03 And after that qemu stalls. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Wed, Apr 15, 2009 at 03:53:40PM +0400, Dmitry Eremin-Solenikov wrote: Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log Now it seems to be a problem with KVM bios. KVM will not work with upstream bochs or qemu bios only with its own version. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov пишет: On Wed, Apr 15, 2009 at 03:53:40PM +0400, Dmitry Eremin-Solenikov wrote: Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log Now it seems to be a problem with KVM bios. KVM will not work with upstream bochs or qemu bios only with its own version. I was talking about qemu-system_x86-64, not about KVM. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Dmitry Eremin-Solenikov wrote: Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. Bochsbios is typically lacking some patches qemu needs, therefore that bios patch queue in qemu. I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log ... init IO_APIC IRQs 1-0 (apicid-pin) not connected IOAPIC[0]: Set routing entry (1-1 - 0x31 - IRQ 1 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-2 - 0x30 - IRQ 0 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-3 - 0x33 - IRQ 3 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-4 - 0x34 - IRQ 4 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-5 - 0x35 - IRQ 5 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-6 - 0x36 - IRQ 6 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-7 - 0x37 - IRQ 7 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-8 - 0x38 - IRQ 8 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-9 - 0x39 - IRQ 9 Mode:1 Active:1) IOAPIC[0]: Set routing entry (1-10 - 0x3a - IRQ 10 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-11 - 0x3b - IRQ 11 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-12 - 0x3c - IRQ 12 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-13 - 0x3d - IRQ 13 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-14 - 0x3e - IRQ 14 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-15 - 0x3f - IRQ 15 Mode:0 Active:0) 1-16 1-17 1-18 1-19 1-20 1-21 1-22 1-23 (apicid-pin) not connected ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 ..MP-BIOS bug: 8254 timer not connected to IO-APIC ...trying to set up timer (IRQ0) through the 8259A ... . (found apic 0 pin 2) ... ... failed. ...trying to set up timer as Virtual Wire IRQ... . failed. ...trying to set up timer as ExtINT IRQ... . failed :( . Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. This looks a bit like [1, 2] on first glance... Jan [1] http://permalink.gmane.org/gmane.comp.emulators.qemu/41300 [2] http://permalink.gmane.org/gmane.comp.emulators.qemu/41433 -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log -- With best wishes Dmitry Linux version 2.6.29-06626-gb9d7dba (lu...@doriath) (gcc version 4.3.3 (Debian 4.3.3-3) ) #8 SMP Wed Apr 15 15:46:28 MSD 2009 Command line: root=/dev/sda1 ro console=ttyS0 apic=debug debug KERNEL supported cpus: Intel GenuineIntel AMD AuthenticAMD Centaur CentaurHauls BIOS-provided physical RAM map: BIOS-e820: - 0009f000 (usable) BIOS-e820: 0009f000 - 000a (reserved) BIOS-e820: 000e8000 - 0010 (reserved) BIOS-e820: 0010 - 0fff (usable) BIOS-e820: 0fff - 1000 (ACPI data) BIOS-e820: fffc - 0001 (reserved) DMI 2.4 present. last_pfn = 0xfff0 max_arch_pfn = 0x1 x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106 init_memory_mapping: -0fff 00 - 000fe0 page 2M 000fe0 - 000fff page 4k kernel direct mapping tables up to fff @ 8000-b000 last_map_addr: fff end: fff ACPI: RSDP 000FBB80, 0014 (r0 QEMU ) ACPI: RSDT 0FFF, 0034 (r1 QEMU QEMURSDT1 QEMU1) ACPI: FACP 0FFF0034, 0074 (r1 QEMU QEMUFACP1 QEMU1) FADT: X_PM1a_EVT_BLK.bit_width (16) does not match PM1_EVT_LEN (4) ACPI: DSDT 0FFF0100, 080D (r1 BXPC BXDSDT1 INTL 20061109) ACPI: FACS 0FFF00C0, 0040 ACPI: APIC 0FFF0948, 004A (r1 QEMU QEMUAPIC1 QEMU1) ACPI: SSDT 0FFF090D, 0037 (r1 QEMU QEMUSSDT1 QEMU1) ACPI: HPET 0FFF0998, 0038 (r1 QEMU QEMUHPET1 QEMU1) ACPI: Local APIC address 0xfee0 (5 early reservations) == bootmem [00 - 000fff] #0 [00 - 001000] BIOS data page == [00 - 001000] #1 [006000 - 008000] TRAMPOLINE == [006000 - 008000] #2 [20 - 876c54]TEXT DATA BSS == [20 - 876c54] #3 [09fc00 - 10]BIOS reserved == [09fc00 - 10] #4 [008000 - 009000] PGTABLE == [008000 - 009000] Scan SMP from 8800 for 1024 bytes. Scan SMP from 8809fc00 for 1024 bytes. Scan SMP from 880f for 65536 bytes. found SMP MP-table at [880fba60] fba60 [e200-e23f] PMD - [88000120-8800015f] on node 0 Zone PFN ranges: DMA 0x - 0x1000 DMA320x1000 - 0x0010 Normal 0x0010 - 0x0010 Movable zone start PFN for each node early_node_map[2] active PFN ranges 0: 0x - 0x009f 0: 0x0100 - 0xfff0 On node 0 totalpages: 65423 DMA zone: 56 pages used for memmap DMA zone: 1756 pages reserved DMA zone: 2187 pages, LIFO batch:0 DMA32 zone: 840 pages used for memmap DMA32 zone: 60584 pages, LIFO batch:15 ACPI: PM-Timer IO Port: 0xb008 ACPI: Local APIC address 0xfee0 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ACPI: IOAPIC (id[0x01] address[0xfec0] gsi_base[0]) IOAPIC[0]: apic_id 1, version 0, address 0xfec0, GSI 0-23 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) ACPI: IRQ0 used by override. ACPI: IRQ2 used by override. ACPI: IRQ9 used by override. Using ACPI (MADT) for SMP configuration information ACPI: HPET id: 0x8086a201 base: 0xfed0 SMP: Allowing 1 CPUs, 0 hotplug CPUs mapped APIC to ff5fc000 (fee0) mapped IOAPIC to ff5fb000 (fec0) nr_irqs_gsi: 24 Allocating PCI resources starting at 2000 (gap: 1000:effc) NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 25 pages at 880001033000, static data 70880 bytes Built 1 zonelists in Zone order, mobility grouping on. Total pages: 62771 Kernel command line:
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Jan Kiszka пишет: Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. Bochsbios is typically lacking some patches qemu needs, therefore that bios patch queue in qemu. Debian's bochsbios provides two bios versions: one for bochs and one patched with qemu (maybe not the latest patches though) I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log ... init IO_APIC IRQs 1-0 (apicid-pin) not connected IOAPIC[0]: Set routing entry (1-1 - 0x31 - IRQ 1 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-2 - 0x30 - IRQ 0 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-3 - 0x33 - IRQ 3 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-4 - 0x34 - IRQ 4 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-5 - 0x35 - IRQ 5 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-6 - 0x36 - IRQ 6 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-7 - 0x37 - IRQ 7 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-8 - 0x38 - IRQ 8 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-9 - 0x39 - IRQ 9 Mode:1 Active:1) IOAPIC[0]: Set routing entry (1-10 - 0x3a - IRQ 10 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-11 - 0x3b - IRQ 11 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-12 - 0x3c - IRQ 12 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-13 - 0x3d - IRQ 13 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-14 - 0x3e - IRQ 14 Mode:0 Active:0) IOAPIC[0]: Set routing entry (1-15 - 0x3f - IRQ 15 Mode:0 Active:0) 1-16 1-17 1-18 1-19 1-20 1-21 1-22 1-23 (apicid-pin) not connected ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 ..MP-BIOS bug: 8254 timer not connected to IO-APIC ...trying to set up timer (IRQ0) through the 8259A ... . (found apic 0 pin 2) ... ... failed. ...trying to set up timer as Virtual Wire IRQ... . failed. ...trying to set up timer as ExtINT IRQ... . failed :( . Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. This looks a bit like [1, 2] on first glance... Jan [1] http://permalink.gmane.org/gmane.comp.emulators.qemu/41300 [2] http://permalink.gmane.org/gmane.comp.emulators.qemu/41433 Looks like a part of this changes. However I don't quite understand: these patches should address non-ACPI OS, but linux is surely and ACPI os! -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Tue, Apr 14, 2009 at 05:55:36PM +0300, Gleb Natapov wrote: After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? And while you are at it can you try to run with -no-kvm-irqchip? -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Tue, Apr 14, 2009 at 07:10:40PM +0300, Avi Kivity wrote: Gleb Natapov wrote: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. Er, I'd much rather have a bisectable series. The next patch (use eveninj to inject interrupts) is really small. We can coalesce those two. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). The whole serie doesn't work too (that's why I started bisecting). And BTW, I got the same results with -no-kvm-irqchip -- With best wishes Dmitry -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: 2009/4/14 Gleb Natapov g...@redhat.com: On Tue, Apr 14, 2009 at 02:14:04PM +, Dmitry Eremin-Solenikov wrote: Gleb Natapov wrote: Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2. Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info. What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? cat /proc/cpuinfo I just noticed that my kernel is different. Will install 2.6.26 and retest, but provide me yours anyway. The whole serie doesn't work too (that's why I started bisecting). And BTW, I got the same results with -no-kvm-irqchip -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Start to use interrupt/exception queues like VMX does.
This also fix the bug that if exit was caused by a guest
internal exception access to IDT the exception was not
reinjected.
Signed-off-by: Gleb Natapov g...@redhat.com
---
arch/x86/kvm/svm.c | 176 ++--
1 files changed, 75 insertions(+), 101 deletions(-)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 52c41aa..053370d 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO);
static int nested = 0;
module_param(nested, int, S_IRUGO);
-static void kvm_reput_irq(struct vcpu_svm *svm);
static void svm_flush_tlb(struct kvm_vcpu *vcpu);
static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override);
@@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu,
unsigned nr,
static bool svm_exception_injected(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
-
- return !(svm-vmcb-control.exit_int_info SVM_EXITINTINFO_VALID);
+ return false;
}
static int is_external_interrupt(u32 info)
@@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct
kvm_guest_debug *dbg)
static int svm_get_irq(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
-
- if (is_external_interrupt(exit_int_info))
- return exit_int_info SVM_EVTINJ_VEC_MASK;
- return -1;
+ if (!vcpu-arch.interrupt.pending)
+ return -1;
+ return vcpu-arch.interrupt.nr;
}
static void load_host_msrs(struct kvm_vcpu *vcpu)
@@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr,
unsigned long value,
static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
{
- u32 exit_int_info = svm-vmcb-control.exit_int_info;
- struct kvm *kvm = svm-vcpu.kvm;
u64 fault_address;
u32 error_code;
- bool event_injection = false;
-
- if (!irqchip_in_kernel(kvm)
- is_external_interrupt(exit_int_info)) {
- event_injection = true;
- kvm_push_irq(svm-vcpu, exit_int_info SVM_EVTINJ_VEC_MASK);
- }
fault_address = svm-vmcb-control.exit_info_2;
error_code = svm-vmcb-control.exit_info_1;
@@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm, struct
kvm_run *kvm_run)
*/
if (npt_enabled)
svm_flush_tlb(svm-vcpu);
-
- if (!npt_enabled event_injection)
- kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ else {
+ if (svm-vcpu.arch.interrupt.pending ||
+ svm-vcpu.arch.exception.pending)
+ kvm_mmu_unprotect_page_virt(svm-vcpu, fault_address);
+ }
return kvm_mmu_page_fault(svm-vcpu, fault_address, error_code);
}
@@ -2187,7 +2174,6 @@ static int handle_exit(struct kvm_run *kvm_run, struct
kvm_vcpu *vcpu)
}
}
- kvm_reput_irq(svm);
if (svm-vmcb-control.exit_code == SVM_EXIT_ERR) {
kvm_run-exit_reason = KVM_EXIT_FAIL_ENTRY;
@@ -2289,98 +2275,47 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu)
(svm-vcpu.arch.hflags HF_GIF_MASK);
}
-static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
+static void enable_irq_window(struct kvm_vcpu *vcpu)
{
- struct vcpu_svm *svm = to_svm(vcpu);
- struct vmcb *vmcb = svm-vmcb;
- int intr_vector = -1;
-
- if ((vmcb-control.exit_int_info SVM_EVTINJ_VALID)
- ((vmcb-control.exit_int_info SVM_EVTINJ_TYPE_MASK) == 0)) {
- intr_vector = vmcb-control.exit_int_info
- SVM_EVTINJ_VEC_MASK;
- vmcb-control.exit_int_info = 0;
- svm_inject_irq(svm, intr_vector);
- goto out;
- }
-
- if (vmcb-control.int_ctl V_IRQ_MASK)
- goto out;
-
- if (!kvm_cpu_has_interrupt(vcpu))
- goto out;
-
- if (nested_svm_intr(svm))
- goto out;
-
- if (!(svm-vcpu.arch.hflags HF_GIF_MASK))
- goto out;
-
- if (!(vmcb-save.rflags X86_EFLAGS_IF) ||
- (vmcb-control.int_state SVM_INTERRUPT_SHADOW_MASK) ||
- (vmcb-control.event_inj SVM_EVTINJ_VALID)) {
- /* unable to deliver irq, set pending irq */
- svm_set_vintr(svm);
- svm_inject_irq(svm, 0x0);
- goto out;
- }
- /* Okay, we can deliver the interrupt: grab it and update PIC state. */
- intr_vector = kvm_cpu_get_interrupt(vcpu);
- svm_inject_irq(svm, intr_vector);
-out:
- update_cr8_intercept(vcpu);
+ svm_set_vintr(to_svm(vcpu));
+ svm_inject_irq(to_svm(vcpu), 0x0);
}
-static void kvm_reput_irq(struct vcpu_svm *svm)
+static
