Re: [PATCH 2/2] KVM: Create debugfs dir and stat files for each VM

[Tyler Baker]

On 27 November 2015 at 00:54, Christian Borntraeger
 wrote:
> On 11/26/2015 09:47 PM, Christian Borntraeger wrote:
>> On 11/26/2015 05:17 PM, Tyler Baker wrote:
>>> Hi Christian,
>>>
>>> The kernelci.org bot recently has been reporting kvm guest boot
>>> failures[1] on various arm64 platforms in next-20151126. The bot
>>> bisected[2] the failures to the commit in -next titled "KVM: Create
>>> debugfs dir and stat files for each VM". I confirmed by reverting this
>>> commit on top of next-20151126 it resolves the boot issue.
>>>
>>> In this test case the host and guest are booted with the same kernel.
>>> The host is booted over nfs, installs qemu (qemu-system arm64 2.4.0),
>>> and launches a guest. The host is booting fine, but when the guest is
>>> launched it errors with "Failed to retrieve host CPU features!". I
>>> checked the host logs, and found an "Unable to handle kernel paging
>>> request" splat[3] which occurs when the guest is attempting to start.
>>>
>>> I scanned the patch in question but nothing obvious jumped out at me,
>>> any thoughts?
>>
>> Not really.
>> Do you have processing running that do read the files in 
>> /sys/kernel/debug/kvm/* ?
>>
>> If I read the arm oops message correctly it oopsed inside
>> __srcu_read_lock. there is actually nothing in there that can oops,
>> except the access to the preempt count. I am just guessing right now,
>> but maybe the preempt variable is no longer available (as the process
>> is gone). As long as a debugfs file is open, we hold a reference to
>> the kvm, which holds a reference to the mm, so the mm might be killed
>> after the process. But this is supposed to work, so maybe its something
>> different. An objdump of __srcu_read_lock might help.
>
> Hmm, the preempt thing is done in srcu_read_lock, but the crash is in
> __srcu_read_lock. This function gets the srcu struct from mmu_notifier.c,
> which must be present and is initialized during boot.
>
>
> int __srcu_read_lock(struct srcu_struct *sp)
> {
> int idx;
>
> idx = READ_ONCE(sp->completed) & 0x1;
> __this_cpu_inc(sp->per_cpu_ref->c[idx]);
> smp_mb(); /* B */  /* Avoid leaking the critical section. */
> __this_cpu_inc(sp->per_cpu_ref->seq[idx]);
> return idx;
> }
>
> Looking at the code I have no clue why the patch does make a difference.
> Can you try to get an objdump -S for__Srcu_read_lock?

Using next-20151126 as the base, here is the objdump[1] I came up with
for__srcu_read_lock.

Cheers,

Tyler

[1] http://hastebin.com/bifiqobola.pl
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 13/21] arm64: KVM: Implement TLB handling

[Marc Zyngier]

Implement the TLB handling as a direct translation of the assembly
code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile |  1 +
 arch/arm64/kvm/hyp/tlb.c| 72 +
 2 files changed, 73 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/tlb.c

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 56238d0..1a529f5 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -10,3 +10,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
 obj-$(CONFIG_KVM_ARM_HOST) += switch.o
 obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
+obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
new file mode 100644
index 000..d4a07d0
--- /dev/null
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -0,0 +1,72 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include "hyp.h"
+
+void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
+{
+   dsb(ishst);
+
+   /* Switch to requested VMID */
+   kvm = kern_hyp_va(kvm);
+   write_sysreg(kvm->arch.vttbr, vttbr_el2);
+   isb();
+
+   /*
+* We could do so much better if we had the VA as well.
+* Instead, we invalidate Stage-2 for this IPA, and the
+* whole of Stage-1. Weep...
+*/
+   ipa >>= 12;
+   asm volatile("tlbi ipas2e1is, %0" : : "r" (ipa));
+   dsb(ish);
+   /*
+* We have to ensure completion of the invalidation at Stage-2,
+* since a table walk on another CPU could refill a TLB with a
+* complete (S1 + S2) walk based on the old Stage-2 mapping if
+* the Stage-1 invalidation happened first.
+*/
+   asm volatile("tlbi vmalle1is" : : );
+   dsb(ish);
+   isb();
+
+   write_sysreg(0, vttbr_el2);
+}
+
+void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
+{
+   dsb(ishst);
+
+   /* Switch to requested VMID */
+   kvm = kern_hyp_va(kvm);
+   write_sysreg(kvm->arch.vttbr, vttbr_el2);
+   isb();
+
+   asm volatile("tlbi vmalls12e1is" : : );
+   dsb(ish);
+   isb();
+
+   write_sysreg(0, vttbr_el2);
+}
+
+void __hyp_text __tlb_flush_vm_context(void)
+{
+   dsb(ishst);
+   asm volatile("tlbi alle1is  \n"
+"ic ialluis  ": : );
+   dsb(ish);
+}
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 14/21] arm64: KVM: HYP mode entry points

[Marc Zyngier]

Add the entry points for HYP mode (both for hypercalls and
exception handling).

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile|   1 +
 arch/arm64/kvm/hyp/hyp-entry.S | 198 +
 2 files changed, 199 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/hyp-entry.S

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 1a529f5..826032b 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -11,3 +11,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += entry.o
 obj-$(CONFIG_KVM_ARM_HOST) += switch.o
 obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
+obj-$(CONFIG_KVM_ARM_HOST) += hyp-entry.o
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
new file mode 100644
index 000..8334407
--- /dev/null
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -0,0 +1,198 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+   .text
+   .pushsection.hyp.text, "ax"
+
+el1_sync:  // Guest trapped into EL2
+   pushx0, x1
+   pushx2, x3
+
+   mrs x1, esr_el2
+   lsr x2, x1, #ESR_ELx_EC_SHIFT
+
+   cmp x2, #ESR_ELx_EC_HVC64
+   b.neel1_trap
+
+   mrs x3, vttbr_el2   // If vttbr is valid, the 64bit 
guest
+   cbnzx3, el1_trap// called HVC
+
+   /* Here, we're pretty sure the host called HVC. */
+   pop x2, x3
+   pop x0, x1
+
+   /* Check for __hyp_get_vectors */
+   cbnzx0, 1f
+   mrs x0, vbar_el2
+   b   2f
+
+1: pushlr, xzr
+
+   /*
+* Compute the function address in EL2, and shuffle the parameters.
+*/
+   kern_hyp_va x0
+   mov lr, x0
+   mov x0, x1
+   mov x1, x2
+   mov x2, x3
+   blr lr
+
+   pop lr, xzr
+2: eret
+
+el1_trap:
+   /*
+* x1: ESR
+* x2: ESR_EC
+*/
+
+   /* Guest accessed VFP/SIMD registers, save host, restore Guest */
+   cmp x2, #ESR_ELx_EC_FP_ASIMD
+   b.eq__fpsimd_guest_restore
+
+   cmp x2, #ESR_ELx_EC_DABT_LOW
+   mov x0, #ESR_ELx_EC_IABT_LOW
+   ccmpx2, x0, #4, ne
+   b.ne1f  // Not an abort we care about
+
+   /* This is an abort. Check for permission fault */
+alternative_if_not ARM64_WORKAROUND_834220
+   and x2, x1, #ESR_ELx_FSC_TYPE
+   cmp x2, #FSC_PERM
+   b.ne1f  // Not a permission fault
+alternative_else
+   nop // Use the permission fault path to
+   nop // check for a valid S1 translation,
+   nop // regardless of the ESR value.
+alternative_endif
+
+
+   /*
+* Check for Stage-1 page table walk, which is guaranteed
+* to give a valid HPFAR_EL2.
+*/
+   tbnzx1, #7, 1f  // S1PTW is set
+
+   /* Preserve PAR_EL1 */
+   mrs x3, par_el1
+   pushx3, xzr
+
+   /*
+* Permission fault, HPFAR_EL2 is invalid.
+* Resolve the IPA the hard way using the guest VA.
+* Stage-1 translation already validated the memory access rights.
+* As such, we can use the EL1 translation regime, and don't have
+* to distinguish between EL0 and EL1 access.
+*/
+   mrs x2, far_el2
+   at  s1e1r, x2
+   isb
+
+   /* Read result */
+   mrs x3, par_el1
+   pop x0, xzr // Restore PAR_EL1 from the stack
+   msr par_el1, x0
+   tbnzx3, #0, 3f  // Bail out if we failed the translation
+   ubfxx3, x3, #12, #36// Extract IPA
+   lsl x3, x3, #4  // and present it like HPFAR
+   b   2f
+
+1: mrs x3, hpfar_el2
+   mrs x2, far_el2
+
+2: mrs x0, tpidr_el2
+   str w1, [x0, #VCPU_ESR_EL2]
+   str x2, [x0, #VCPU_FAR_EL2]
+   str x3, [x0, #VCPU_HPFAR_EL2]
+
+   mov x1, #ARM_EXCEPTION_TRAP
+   b   __guest_exit
+
+   /*
+* Translation failed. Just return to the guest and
+* let it 

[PATCH v2 06/21] arm64: KVM: Implement system register save/restore

[Marc Zyngier]

Implement the system registe save restore as a direct translation of
the assembly code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile|  1 +
 arch/arm64/kvm/hyp/hyp.h   |  3 ++
 arch/arm64/kvm/hyp/sysreg-sr.c | 90 ++
 3 files changed, 94 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.c

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 455dc0a..ec94200 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -5,3 +5,4 @@
 obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 86aa5a2..087d3a5 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -36,5 +36,8 @@ void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
 void __timer_save_state(struct kvm_vcpu *vcpu);
 void __timer_restore_state(struct kvm_vcpu *vcpu);
 
+void __sysreg_save_state(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
+
 #endif /* __ARM64_KVM_HYP_H__ */
 
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
new file mode 100644
index 000..add8fcb
--- /dev/null
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -0,0 +1,90 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+#include 
+
+#include 
+
+#include "hyp.h"
+
+/* ctxt is already in the HYP VA space */
+void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
+{
+   ctxt->sys_regs[MPIDR_EL1]   = read_sysreg(vmpidr_el2);
+   ctxt->sys_regs[CSSELR_EL1]  = read_sysreg(csselr_el1);
+   ctxt->sys_regs[SCTLR_EL1]   = read_sysreg(sctlr_el1);
+   ctxt->sys_regs[ACTLR_EL1]   = read_sysreg(actlr_el1);
+   ctxt->sys_regs[CPACR_EL1]   = read_sysreg(cpacr_el1);
+   ctxt->sys_regs[TTBR0_EL1]   = read_sysreg(ttbr0_el1);
+   ctxt->sys_regs[TTBR1_EL1]   = read_sysreg(ttbr1_el1);
+   ctxt->sys_regs[TCR_EL1] = read_sysreg(tcr_el1);
+   ctxt->sys_regs[ESR_EL1] = read_sysreg(esr_el1);
+   ctxt->sys_regs[AFSR0_EL1]   = read_sysreg(afsr0_el1);
+   ctxt->sys_regs[AFSR1_EL1]   = read_sysreg(afsr1_el1);
+   ctxt->sys_regs[FAR_EL1] = read_sysreg(far_el1);
+   ctxt->sys_regs[MAIR_EL1]= read_sysreg(mair_el1);
+   ctxt->sys_regs[VBAR_EL1]= read_sysreg(vbar_el1);
+   ctxt->sys_regs[CONTEXTIDR_EL1]  = read_sysreg(contextidr_el1);
+   ctxt->sys_regs[TPIDR_EL0]   = read_sysreg(tpidr_el0);
+   ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
+   ctxt->sys_regs[TPIDR_EL1]   = read_sysreg(tpidr_el1);
+   ctxt->sys_regs[AMAIR_EL1]   = read_sysreg(amair_el1);
+   ctxt->sys_regs[CNTKCTL_EL1] = read_sysreg(cntkctl_el1);
+   ctxt->sys_regs[PAR_EL1] = read_sysreg(par_el1);
+   ctxt->sys_regs[MDSCR_EL1]   = read_sysreg(mdscr_el1);
+
+   ctxt->gp_regs.regs.sp   = read_sysreg(sp_el0);
+   ctxt->gp_regs.regs.pc   = read_sysreg(elr_el2);
+   ctxt->gp_regs.regs.pstate   = read_sysreg(spsr_el2);
+   ctxt->gp_regs.sp_el1= read_sysreg(sp_el1);
+   ctxt->gp_regs.elr_el1   = read_sysreg(elr_el1);
+   ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg(spsr_el1);
+}
+
+void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
+{
+   write_sysreg(ctxt->sys_regs[MPIDR_EL1],   vmpidr_el2);
+   write_sysreg(ctxt->sys_regs[CSSELR_EL1],  csselr_el1);
+   write_sysreg(ctxt->sys_regs[SCTLR_EL1],   sctlr_el1);
+   write_sysreg(ctxt->sys_regs[ACTLR_EL1],   actlr_el1);
+   write_sysreg(ctxt->sys_regs[CPACR_EL1],   cpacr_el1);
+   write_sysreg(ctxt->sys_regs[TTBR0_EL1],   ttbr0_el1);
+   write_sysreg(ctxt->sys_regs[TTBR1_EL1],   ttbr1_el1);
+   write_sysreg(ctxt->sys_regs[TCR_EL1], tcr_el1);
+   write_sysreg(ctxt->sys_regs[ESR_EL1], esr_el1);
+   write_sysreg(ctxt->sys_regs[AFSR0_EL1],   afsr0_el1);
+   write_sysreg(ctxt->sys_regs[AFSR1_EL1],   afsr1_el1);
+   write_sysreg(ctxt->sys_regs[FAR_EL1], far_el1);
+   write_sysreg(ctxt->sys_regs[MAIR_EL1],  

[PATCH v2 15/21] arm64: KVM: Add panic handling

[Marc Zyngier]

Add the panic handler, together with the small bits of assembly
code to call the kernel's panic implementation.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/hyp-entry.S | 11 ++-
 arch/arm64/kvm/hyp/hyp.h   |  1 +
 arch/arm64/kvm/hyp/switch.c| 30 ++
 3 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 8334407..39d6935 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -150,7 +150,16 @@ el1_irq:
mov x1, #ARM_EXCEPTION_IRQ
b   __guest_exit
 
-.macro invalid_vector  label, target = __kvm_hyp_panic
+ENTRY(__hyp_do_panic)
+   mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\
+ PSR_MODE_EL1h)
+   msr spsr_el2, lr
+   ldr lr, =panic
+   msr elr_el2, lr
+   eret
+ENDPROC(__hyp_do_panic)
+
+.macro invalid_vector  label, target = __hyp_panic
.align  2
 \label:
b \target
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 18365dd..87f16fa 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -74,6 +74,7 @@ static inline bool __fpsimd_enabled(void)
 }
 
 u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
+void __noreturn __hyp_do_panic(unsigned long, ...);
 
 #endif /* __ARM64_KVM_HYP_H__ */
 
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 8affc19..7b81089 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -140,3 +140,33 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
 
return exit_code;
 }
+
+static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx 
ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
+
+void __hyp_text __noreturn __hyp_panic(void)
+{
+   unsigned long str_va = (unsigned long)__hyp_panic_string;
+   u64 spsr = read_sysreg(spsr_el2);
+   u64 elr = read_sysreg(elr_el2);
+   u64 par = read_sysreg(par_el1);
+
+   if (read_sysreg(vttbr_el2)) {
+   struct kvm_vcpu *vcpu;
+   struct kvm_cpu_context *host_ctxt;
+
+   vcpu = (struct kvm_vcpu *)read_sysreg(tpidr_el2);
+   host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+   __deactivate_traps(vcpu);
+   __deactivate_vm(vcpu);
+   __sysreg_restore_state(host_ctxt);
+   }
+
+   /* Call panic for real */
+   __hyp_do_panic(str_va - HYP_PAGE_OFFSET + PAGE_OFFSET,
+  spsr,  elr,
+  read_sysreg(esr_el2),   read_sysreg(far_el2),
+  read_sysreg(hpfar_el2), par,
+  (void *)read_sysreg(tpidr_el2));
+
+   unreachable();
+}
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c

[Marc Zyngier]

As we've now rewritten most of our code-base in C, most of the
KVM-specific code in asm-offset.c is useless. Delete-time again!

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kernel/asm-offsets.c | 39 ---
 1 file changed, 39 deletions(-)

diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 4b72231..94090a6 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -108,50 +108,11 @@ int main(void)
   DEFINE(CPU_GP_REGS,  offsetof(struct kvm_cpu_context, gp_regs));
   DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
   DEFINE(CPU_FP_REGS,  offsetof(struct kvm_regs, fp_regs));
-  DEFINE(CPU_SP_EL1,   offsetof(struct kvm_regs, sp_el1));
-  DEFINE(CPU_ELR_EL1,  offsetof(struct kvm_regs, elr_el1));
-  DEFINE(CPU_SPSR, offsetof(struct kvm_regs, spsr));
-  DEFINE(CPU_SYSREGS,  offsetof(struct kvm_cpu_context, sys_regs));
   DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, 
arch.ctxt.sys_regs[FPEXC32_EL2]));
   DEFINE(VCPU_ESR_EL2, offsetof(struct kvm_vcpu, arch.fault.esr_el2));
   DEFINE(VCPU_FAR_EL2, offsetof(struct kvm_vcpu, arch.fault.far_el2));
   DEFINE(VCPU_HPFAR_EL2,   offsetof(struct kvm_vcpu, 
arch.fault.hpfar_el2));
-  DEFINE(VCPU_DEBUG_FLAGS, offsetof(struct kvm_vcpu, arch.debug_flags));
-  DEFINE(VCPU_DEBUG_PTR,   offsetof(struct kvm_vcpu, arch.debug_ptr));
-  DEFINE(DEBUG_BCR,offsetof(struct kvm_guest_debug_arch, dbg_bcr));
-  DEFINE(DEBUG_BVR,offsetof(struct kvm_guest_debug_arch, dbg_bvr));
-  DEFINE(DEBUG_WCR,offsetof(struct kvm_guest_debug_arch, dbg_wcr));
-  DEFINE(DEBUG_WVR,offsetof(struct kvm_guest_debug_arch, dbg_wvr));
-  DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2));
-  DEFINE(VCPU_MDCR_EL2,offsetof(struct kvm_vcpu, arch.mdcr_el2));
-  DEFINE(VCPU_IRQ_LINES,   offsetof(struct kvm_vcpu, arch.irq_lines));
   DEFINE(VCPU_HOST_CONTEXT,offsetof(struct kvm_vcpu, 
arch.host_cpu_context));
-  DEFINE(VCPU_HOST_DEBUG_STATE, offsetof(struct kvm_vcpu, 
arch.host_debug_state));
-  DEFINE(VCPU_TIMER_CNTV_CTL,  offsetof(struct kvm_vcpu, 
arch.timer_cpu.cntv_ctl));
-  DEFINE(VCPU_TIMER_CNTV_CVAL, offsetof(struct kvm_vcpu, 
arch.timer_cpu.cntv_cval));
-  DEFINE(KVM_TIMER_CNTVOFF,offsetof(struct kvm, arch.timer.cntvoff));
-  DEFINE(KVM_TIMER_ENABLED,offsetof(struct kvm, arch.timer.enabled));
-  DEFINE(VCPU_KVM, offsetof(struct kvm_vcpu, kvm));
-  DEFINE(VCPU_VGIC_CPU,offsetof(struct kvm_vcpu, 
arch.vgic_cpu));
-  DEFINE(VGIC_V2_CPU_HCR,  offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
-  DEFINE(VGIC_V2_CPU_VMCR, offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
-  DEFINE(VGIC_V2_CPU_MISR, offsetof(struct vgic_cpu, vgic_v2.vgic_misr));
-  DEFINE(VGIC_V2_CPU_EISR, offsetof(struct vgic_cpu, vgic_v2.vgic_eisr));
-  DEFINE(VGIC_V2_CPU_ELRSR,offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
-  DEFINE(VGIC_V2_CPU_APR,  offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
-  DEFINE(VGIC_V2_CPU_LR,   offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
-  DEFINE(VGIC_V3_CPU_SRE,  offsetof(struct vgic_cpu, vgic_v3.vgic_sre));
-  DEFINE(VGIC_V3_CPU_HCR,  offsetof(struct vgic_cpu, vgic_v3.vgic_hcr));
-  DEFINE(VGIC_V3_CPU_VMCR, offsetof(struct vgic_cpu, vgic_v3.vgic_vmcr));
-  DEFINE(VGIC_V3_CPU_MISR, offsetof(struct vgic_cpu, vgic_v3.vgic_misr));
-  DEFINE(VGIC_V3_CPU_EISR, offsetof(struct vgic_cpu, vgic_v3.vgic_eisr));
-  DEFINE(VGIC_V3_CPU_ELRSR,offsetof(struct vgic_cpu, vgic_v3.vgic_elrsr));
-  DEFINE(VGIC_V3_CPU_AP0R, offsetof(struct vgic_cpu, vgic_v3.vgic_ap0r));
-  DEFINE(VGIC_V3_CPU_AP1R, offsetof(struct vgic_cpu, vgic_v3.vgic_ap1r));
-  DEFINE(VGIC_V3_CPU_LR,   offsetof(struct vgic_cpu, vgic_v3.vgic_lr));
-  DEFINE(VGIC_CPU_NR_LR,   offsetof(struct vgic_cpu, nr_lr));
-  DEFINE(KVM_VTTBR,offsetof(struct kvm, arch.vttbr));
-  DEFINE(KVM_VGIC_VCTRL,   offsetof(struct kvm, arch.vgic.vctrl_base));
 #endif
 #ifdef CONFIG_CPU_PM
   DEFINE(CPU_SUSPEND_SZ,   sizeof(struct cpu_suspend_ctx));
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 11/21] arm64: KVM: Implement the core world switch

[Marc Zyngier]

Implement the core of the world switch in C. Not everything is there
yet, and there is nothing to re-enter the world switch either.

But this already outlines the code structure well enough.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile |   1 +
 arch/arm64/kvm/hyp/switch.c | 134 
 2 files changed, 135 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/switch.c

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 1e1ff06..9c11b0f 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -8,3 +8,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
+obj-$(CONFIG_KVM_ARM_HOST) += switch.o
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
new file mode 100644
index 000..d67ed9e
--- /dev/null
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -0,0 +1,134 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include "hyp.h"
+
+static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
+{
+   u64 val;
+
+   /*
+* We are about to set CPTR_EL2.TFP to trap all floating point
+* register accesses to EL2, however, the ARM ARM clearly states that
+* traps are only taken to EL2 if the operation would not otherwise
+* trap to EL1.  Therefore, always make sure that for 32-bit guests,
+* we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
+*/
+   val = vcpu->arch.hcr_el2;
+   if (!(val & HCR_RW)) {
+   write_sysreg(1 << 30, fpexc32_el2);
+   isb();
+   }
+   write_sysreg(val, hcr_el2);
+   write_sysreg(1 << 15, hstr_el2);
+   write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2);
+   write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+}
+
+static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
+{
+   write_sysreg(HCR_RW, hcr_el2);
+   write_sysreg(0, hstr_el2);
+   write_sysreg(read_sysreg(mdcr_el2) & MDCR_EL2_HPMN_MASK, mdcr_el2);
+   write_sysreg(0, cptr_el2);
+}
+
+static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
+{
+   struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+   write_sysreg(kvm->arch.vttbr, vttbr_el2);
+}
+
+static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
+{
+   write_sysreg(0, vttbr_el2);
+}
+
+static hyp_alternate_select(__vgic_call_save_state,
+   __vgic_v2_save_state, __vgic_v3_save_state,
+   ARM64_HAS_SYSREG_GIC_CPUIF);
+
+static hyp_alternate_select(__vgic_call_restore_state,
+   __vgic_v2_restore_state, __vgic_v3_restore_state,
+   ARM64_HAS_SYSREG_GIC_CPUIF);
+
+static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
+{
+   __vgic_call_save_state()(vcpu);
+   write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
+}
+
+static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
+{
+   u64 val;
+
+   val = read_sysreg(hcr_el2);
+   val |=  HCR_INT_OVERRIDE;
+   val |= vcpu->arch.irq_lines;
+   write_sysreg(val, hcr_el2);
+
+   __vgic_call_restore_state()(vcpu);
+}
+
+int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
+{
+   struct kvm_cpu_context *host_ctxt;
+   struct kvm_cpu_context *guest_ctxt;
+   u64 exit_code;
+
+   vcpu = kern_hyp_va(vcpu);
+   write_sysreg(vcpu, tpidr_el2);
+
+   host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+   guest_ctxt = >arch.ctxt;
+
+   __sysreg_save_state(host_ctxt);
+   __debug_cond_save_host_state(vcpu);
+
+   __activate_traps(vcpu);
+   __activate_vm(vcpu);
+
+   __vgic_restore_state(vcpu);
+   __timer_restore_state(vcpu);
+
+   /*
+* We must restore the 32-bit state before the sysregs, thanks
+* to Cortex-A57 erratum #852523.
+*/
+   __sysreg32_restore_state(vcpu);
+   __sysreg_restore_state(guest_ctxt);
+   __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), 
guest_ctxt);
+
+   /* Jump in the fire! */
+   exit_code = __guest_enter(vcpu, host_ctxt);
+   /* And we're baaack! */
+
+   __sysreg_save_state(guest_ctxt);
+   

[PATCH v2 08/21] arm64: KVM: Implement debug save/restore

[Marc Zyngier]

Implement the debug save restore as a direct translation of
the assembly code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile   |   1 +
 arch/arm64/kvm/hyp/debug-sr.c | 130 ++
 arch/arm64/kvm/hyp/hyp.h  |   9 +++
 3 files changed, 140 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/debug-sr.c

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index ec94200..ec14cac 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
new file mode 100644
index 000..a0b2b99
--- /dev/null
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -0,0 +1,130 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+#include 
+
+#include 
+
+#include "hyp.h"
+
+#define read_debug(r,n)read_sysreg(r##n##_el1)
+#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
+
+#define save_debug(ptr,reg,nr) \
+   switch (nr) {   \
+   case 15:ptr[15] = read_debug(reg, 15);  \
+   case 14:ptr[14] = read_debug(reg, 14);  \
+   case 13:ptr[13] = read_debug(reg, 13);  \
+   case 12:ptr[12] = read_debug(reg, 12);  \
+   case 11:ptr[11] = read_debug(reg, 11);  \
+   case 10:ptr[10] = read_debug(reg, 10);  \
+   case 9: ptr[9] = read_debug(reg, 9);\
+   case 8: ptr[8] = read_debug(reg, 8);\
+   case 7: ptr[7] = read_debug(reg, 7);\
+   case 6: ptr[6] = read_debug(reg, 6);\
+   case 5: ptr[5] = read_debug(reg, 5);\
+   case 4: ptr[4] = read_debug(reg, 4);\
+   case 3: ptr[3] = read_debug(reg, 3);\
+   case 2: ptr[2] = read_debug(reg, 2);\
+   case 1: ptr[1] = read_debug(reg, 1);\
+   default:ptr[0] = read_debug(reg, 0);\
+   }
+
+#define restore_debug(ptr,reg,nr)  \
+   switch (nr) {   \
+   case 15:write_debug(ptr[15], reg, 15);  \
+   case 14:write_debug(ptr[14], reg, 14);  \
+   case 13:write_debug(ptr[13], reg, 13);  \
+   case 12:write_debug(ptr[12], reg, 12);  \
+   case 11:write_debug(ptr[11], reg, 11);  \
+   case 10:write_debug(ptr[10], reg, 10);  \
+   case 9: write_debug(ptr[9], reg, 9);\
+   case 8: write_debug(ptr[8], reg, 8);\
+   case 7: write_debug(ptr[7], reg, 7);\
+   case 6: write_debug(ptr[6], reg, 6);\
+   case 5: write_debug(ptr[5], reg, 5);\
+   case 4: write_debug(ptr[4], reg, 4);\
+   case 3: write_debug(ptr[3], reg, 3);\
+   case 2: write_debug(ptr[2], reg, 2);\
+   case 1: write_debug(ptr[1], reg, 1);\
+   default:write_debug(ptr[0], reg, 0);\
+   }
+
+void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
+  struct kvm_guest_debug_arch *dbg,
+  struct kvm_cpu_context *ctxt)
+{
+   if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
+   u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
+   int brps, wrps;
+
+   brps = (aa64dfr0 >> 12) & 0xf;
+   wrps = (aa64dfr0 >> 20) & 0xf;
+
+  

[PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum

[Marc Zyngier]

Having the system register numbers as #defines has been a pain
since day one, as the ordering is pretty fragile, and moving
things around leads to renumbering and epic conflict resolutions.

Now that we're mostly acessing the sysreg file in C, an enum is
a much better type to use, and we can clean things up a bit.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/include/asm/kvm_asm.h | 76 -
 arch/arm64/include/asm/kvm_emulate.h |  1 -
 arch/arm64/include/asm/kvm_host.h| 81 +++-
 arch/arm64/include/asm/kvm_mmio.h|  1 -
 arch/arm64/kernel/asm-offsets.c  |  1 +
 arch/arm64/kvm/guest.c   |  1 -
 arch/arm64/kvm/handle_exit.c |  1 +
 arch/arm64/kvm/hyp/debug-sr.c|  1 +
 arch/arm64/kvm/hyp/entry.S   |  3 +-
 arch/arm64/kvm/hyp/sysreg-sr.c   |  1 +
 arch/arm64/kvm/sys_regs.c|  1 +
 virt/kvm/arm/vgic-v3.c   |  1 +
 12 files changed, 87 insertions(+), 82 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 5e37710..52b777b 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -20,82 +20,6 @@
 
 #include 
 
-/*
- * 0 is reserved as an invalid value.
- * Order *must* be kept in sync with the hyp switch code.
- */
-#defineMPIDR_EL1   1   /* MultiProcessor Affinity Register */
-#defineCSSELR_EL1  2   /* Cache Size Selection Register */
-#defineSCTLR_EL1   3   /* System Control Register */
-#defineACTLR_EL1   4   /* Auxiliary Control Register */
-#defineCPACR_EL1   5   /* Coprocessor Access Control */
-#defineTTBR0_EL1   6   /* Translation Table Base Register 0 */
-#defineTTBR1_EL1   7   /* Translation Table Base Register 1 */
-#defineTCR_EL1 8   /* Translation Control Register */
-#defineESR_EL1 9   /* Exception Syndrome Register */
-#defineAFSR0_EL1   10  /* Auxilary Fault Status Register 0 */
-#defineAFSR1_EL1   11  /* Auxilary Fault Status Register 1 */
-#defineFAR_EL1 12  /* Fault Address Register */
-#defineMAIR_EL113  /* Memory Attribute Indirection 
Register */
-#defineVBAR_EL114  /* Vector Base Address Register */
-#defineCONTEXTIDR_EL1  15  /* Context ID Register */
-#defineTPIDR_EL0   16  /* Thread ID, User R/W */
-#defineTPIDRRO_EL0 17  /* Thread ID, User R/O */
-#defineTPIDR_EL1   18  /* Thread ID, Privileged */
-#defineAMAIR_EL1   19  /* Aux Memory Attribute Indirection 
Register */
-#defineCNTKCTL_EL1 20  /* Timer Control Register (EL1) */
-#definePAR_EL1 21  /* Physical Address Register */
-#define MDSCR_EL1  22  /* Monitor Debug System Control Register */
-#define MDCCINT_EL123  /* Monitor Debug Comms Channel Interrupt Enable 
Reg */
-
-/* 32bit specific registers. Keep them at the end of the range */
-#defineDACR32_EL2  24  /* Domain Access Control Register */
-#defineIFSR32_EL2  25  /* Instruction Fault Status Register */
-#defineFPEXC32_EL2 26  /* Floating-Point Exception Control 
Register */
-#defineDBGVCR32_EL227  /* Debug Vector Catch Register */
-#defineNR_SYS_REGS 28
-
-/* 32bit mapping */
-#define c0_MPIDR   (MPIDR_EL1 * 2) /* MultiProcessor ID Register */
-#define c0_CSSELR  (CSSELR_EL1 * 2)/* Cache Size Selection Register */
-#define c1_SCTLR   (SCTLR_EL1 * 2) /* System Control Register */
-#define c1_ACTLR   (ACTLR_EL1 * 2) /* Auxiliary Control Register */
-#define c1_CPACR   (CPACR_EL1 * 2) /* Coprocessor Access Control */
-#define c2_TTBR0   (TTBR0_EL1 * 2) /* Translation Table Base Register 0 */
-#define c2_TTBR0_high  (c2_TTBR0 + 1)  /* TTBR0 top 32 bits */
-#define c2_TTBR1   (TTBR1_EL1 * 2) /* Translation Table Base Register 1 */
-#define c2_TTBR1_high  (c2_TTBR1 + 1)  /* TTBR1 top 32 bits */
-#define c2_TTBCR   (TCR_EL1 * 2)   /* Translation Table Base Control R. */
-#define c3_DACR(DACR32_EL2 * 2)/* Domain Access Control 
Register */
-#define c5_DFSR(ESR_EL1 * 2)   /* Data Fault Status Register */
-#define c5_IFSR(IFSR32_EL2 * 2)/* Instruction Fault Status 
Register */
-#define c5_ADFSR   (AFSR0_EL1 * 2) /* Auxiliary Data Fault Status R */
-#define c5_AIFSR   (AFSR1_EL1 * 2) /* Auxiliary Instr Fault Status R */
-#define c6_DFAR(FAR_EL1 * 2)   /* Data Fault Address Register 
*/
-#define c6_IFAR(c6_DFAR + 1)   /* Instruction Fault Address 
Register */
-#define c7_PAR (PAR_EL1 * 2)   /* Physical Address Register */
-#define c7_PAR_high(c7_PAR + 1)/* PAR top 32 bits */

[PATCH v2 00/21] arm64: KVM: world switch in C

[Marc Zyngier]

Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
and mean piece of hand-crafted assembly code. Over time, features have
crept in, the code has become harder to maintain, and the smallest
change is a pain to introduce. The VHE patches are a prime example of
why this doesn't work anymore.

This series rewrites most of the existing assembly code in C, but keeps
the existing code structure in place (most function names will look
familiar to the reader). The biggest change is that we don't have to
deal with a static register allocation (the compiler does it for us),
we can easily follow structure and pointers, and only the lowest level
is still in assembly code. Oh, and a negative diffstat.

There is still a healthy dose of inline assembly (system register
accessors, runtime code patching), but I've tried not to make it too
invasive. The generated code, while not exactly brilliant, doesn't
look too shaby. I do expect a small performance degradation, but I
believe this is something we can improve over time (my initial
measurements don't show any obvious regression though).

Eventually (and assuming people are happy with the general approach
taken here), it should be possible to make the 32bit converge with
this and reuse some parts of the code.

Patches are against 4.4-rc2 (mostly), and I've pushed a branch out
(kvm-arm64/wsinc). This has been tested on Juno, Seattle and the FVP
model. I also have pushed out kvm-arm64/vhe-wsinc that implements VHE
on top of these patches.

* From v1:
  - A number of bugs have been squashed: vgic, FP/SIMD, debug, panic
(thanks to Mark, Steve, Ard and Alex for their reviews)
  - Rebased on 4.4-rc2 plus KVM fixes aimed at -rc3 plus a couple
of gic-v3 fixes

Marc Zyngier (20):
  arm64: KVM: Add a HYP-specific header file
  arm64: KVM: Implement vgic-v2 save/restore
  arm64: KVM: Implement vgic-v3 save/restore
  arm64: KVM: Implement timer save/restore
  arm64: KVM: Implement system register save/restore
  arm64: KVM: Implement 32bit system register save/restore
  arm64: KVM: Implement debug save/restore
  arm64: KVM: Implement guest entry
  arm64: KVM: Add patchable function selector
  arm64: KVM: Implement the core world switch
  arm64: KVM: Implement fpsimd save/restore
  arm64: KVM: Implement TLB handling
  arm64: KVM: HYP mode entry points
  arm64: KVM: Add panic handling
  arm64: KVM: Add compatibility aliases
  arm64: KVM: Map the kernel RO section into HYP
  arm64: KVM: Move away from the assembly version of the world switch
  arm64: KVM: Turn system register numbers to an enum
  arm64: KVM: Cleanup asm-offset.c
  arm64: KVM: Remove weak attributes

Mark Rutland (1):
  arm64: Add macros to read/write system registers

 arch/arm/kvm/arm.c   |7 +
 arch/arm64/include/asm/kvm_asm.h |   76 ---
 arch/arm64/include/asm/kvm_emulate.h |1 -
 arch/arm64/include/asm/kvm_host.h|   81 ++-
 arch/arm64/include/asm/kvm_mmio.h|1 -
 arch/arm64/include/asm/sysreg.h  |   17 +
 arch/arm64/kernel/asm-offsets.c  |   40 +-
 arch/arm64/kvm/Makefile  |3 +-
 arch/arm64/kvm/guest.c   |1 -
 arch/arm64/kvm/handle_exit.c |1 +
 arch/arm64/kvm/hyp.S | 1081 +-
 arch/arm64/kvm/hyp/Makefile  |   14 +
 arch/arm64/kvm/hyp/debug-sr.c|  133 +
 arch/arm64/kvm/hyp/entry.S   |  184 ++
 arch/arm64/kvm/hyp/fpsimd.S  |   33 ++
 arch/arm64/kvm/hyp/hyp-entry.S   |  207 +++
 arch/arm64/kvm/hyp/hyp.h |   80 +++
 arch/arm64/kvm/hyp/switch.c  |  174 ++
 arch/arm64/kvm/hyp/sysreg-sr.c   |  132 +
 arch/arm64/kvm/hyp/timer-sr.c|   71 +++
 arch/arm64/kvm/hyp/tlb.c |   79 +++
 arch/arm64/kvm/hyp/vgic-v2-sr.c  |   89 +++
 arch/arm64/kvm/hyp/vgic-v3-sr.c  |  224 +++
 arch/arm64/kvm/sys_regs.c|1 +
 arch/arm64/kvm/vgic-v2-switch.S  |  134 -
 arch/arm64/kvm/vgic-v3-switch.S  |  269 -
 include/clocksource/arm_arch_timer.h |6 +
 virt/kvm/arm/vgic-v3.c   |1 +
 28 files changed, 1536 insertions(+), 1604 deletions(-)
 create mode 100644 arch/arm64/kvm/hyp/Makefile
 create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
 create mode 100644 arch/arm64/kvm/hyp/entry.S
 create mode 100644 arch/arm64/kvm/hyp/fpsimd.S
 create mode 100644 arch/arm64/kvm/hyp/hyp-entry.S
 create mode 100644 arch/arm64/kvm/hyp/hyp.h
 create mode 100644 arch/arm64/kvm/hyp/switch.c
 create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.c
 create mode 100644 arch/arm64/kvm/hyp/timer-sr.c
 create mode 100644 arch/arm64/kvm/hyp/tlb.c
 create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c
 create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
 delete mode 100644 arch/arm64/kvm/vgic-v2-switch.S
 delete mode 100644 arch/arm64/kvm/vgic-v3-switch.S

-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to 

[PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore

[Marc Zyngier]

Implement the vgic-v2 save restore (mostly) as a direct translation
of the assembly code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/Makefile |  1 +
 arch/arm64/kvm/hyp/Makefile |  5 +++
 arch/arm64/kvm/hyp/hyp.h|  3 ++
 arch/arm64/kvm/hyp/vgic-v2-sr.c | 89 +
 4 files changed, 98 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/Makefile
 create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c

diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 1949fe5..d31e4e5 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -10,6 +10,7 @@ KVM=../../../virt/kvm
 ARM=../../../arch/arm/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += kvm.o
+obj-$(CONFIG_KVM_ARM_HOST) += hyp/
 
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o 
$(KVM)/eventfd.o $(KVM)/vfio.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
new file mode 100644
index 000..d8d5968
--- /dev/null
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for Kernel-based Virtual Machine module, HYP part
+#
+
+obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index dac843e..78f25c4 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -27,5 +27,8 @@
 
 #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
 
+void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
+
 #endif /* __ARM64_KVM_HYP_H__ */
 
diff --git a/arch/arm64/kvm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
new file mode 100644
index 000..29a5c1d
--- /dev/null
+++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
@@ -0,0 +1,89 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+
+#include "hyp.h"
+
+/* vcpu is already in the HYP VA space */
+void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
+{
+   struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+   struct vgic_v2_cpu_if *cpu_if = >arch.vgic_cpu.vgic_v2;
+   struct vgic_dist *vgic = >arch.vgic;
+   void __iomem *base = kern_hyp_va(vgic->vctrl_base);
+   u32 __iomem *lr_base;
+   u32 eisr0, eisr1, elrsr0, elrsr1;
+   int i = 0, nr_lr;
+
+   if (!base)
+   return;
+
+   nr_lr = vcpu->arch.vgic_cpu.nr_lr;
+   cpu_if->vgic_vmcr = readl_relaxed(base + GICH_VMCR);
+   cpu_if->vgic_misr = readl_relaxed(base + GICH_MISR);
+   eisr0  = readl_relaxed(base + GICH_EISR0);
+   elrsr0 = readl_relaxed(base + GICH_ELRSR0);
+   if (unlikely(nr_lr > 32)) {
+   eisr1  = readl_relaxed(base + GICH_EISR1);
+   elrsr1 = readl_relaxed(base + GICH_ELRSR1);
+   } else {
+   eisr1 = elrsr1 = 0;
+   }
+#ifdef CONFIG_CPU_BIG_ENDIAN
+   cpu_if->vgic_eisr  = ((u64)eisr0 << 32) | eisr1;
+   cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
+#else
+   cpu_if->vgic_eisr  = ((u64)eisr1 << 32) | eisr0;
+   cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
+#endif
+   cpu_if->vgic_apr= readl_relaxed(base + GICH_APR);
+
+   writel_relaxed(0, base + GICH_HCR);
+
+   lr_base = base + GICH_LR0;
+   do {
+   cpu_if->vgic_lr[i++] = readl_relaxed(lr_base++);
+   } while (--nr_lr);
+}
+
+void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
+{
+   struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+   struct vgic_v2_cpu_if *cpu_if = >arch.vgic_cpu.vgic_v2;
+   struct vgic_dist *vgic = >arch.vgic;
+   void __iomem *base = kern_hyp_va(vgic->vctrl_base);
+   u32 __iomem *lr_base;
+   unsigned int i = 0, nr_lr;
+
+   if (!base)
+   return;
+
+   writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
+   writel_relaxed(cpu_if->vgic_vmcr, base + GICH_VMCR);
+   writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
+
+   lr_base = base + GICH_LR0;
+   nr_lr = vcpu->arch.vgic_cpu.nr_lr;
+   do {
+   writel_relaxed(cpu_if->vgic_lr[i++], lr_base++);
+   } while (--nr_lr);
+}
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org

[PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file

[Marc Zyngier]

In order to expose the various EL2 services that are private to
the hypervisor, add a new hyp.h file.

So far, it only contains mundane things such as section annotation
and VA manipulation.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/hyp.h | 31 +++
 1 file changed, 31 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/hyp.h

diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
new file mode 100644
index 000..dac843e
--- /dev/null
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#ifndef __ARM64_KVM_HYP_H__
+#define __ARM64_KVM_HYP_H__
+
+#include 
+#include 
+#include 
+#include 
+
+#define __hyp_text __section(.hyp.text) notrace
+
+#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
+
+#endif /* __ARM64_KVM_HYP_H__ */
+
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 01/21] arm64: Add macros to read/write system registers

[Marc Zyngier]

From: Mark Rutland 

Rather than crafting custom macros for reading/writing each system
register provide generics accessors, read_sysreg and write_sysreg, for
this purpose.

Unlike read_cpuid, calls to read_exception_reg are never expected
to be optimized away or replaced with synthetic values.

Signed-off-by: Mark Rutland 
Cc: Catalin Marinas 
Cc: Marc Zyngier 
Cc: Suzuki Poulose 
Cc: Will Deacon 
Signed-off-by: Marc Zyngier 
---
 arch/arm64/include/asm/sysreg.h | 17 +
 1 file changed, 17 insertions(+)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index d48ab5b..c9c283a 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -20,6 +20,8 @@
 #ifndef __ASM_SYSREG_H
 #define __ASM_SYSREG_H
 
+#include 
+
 #include 
 
 /*
@@ -208,6 +210,8 @@
 
 #else
 
+#include 
+
 asm(
 "  .irp
num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
 "  .equ__reg_num_x\\num, \\num\n"
@@ -232,6 +236,19 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
val |= set;
asm volatile("msr sctlr_el1, %0" : : "r" (val));
 }
+
+#define read_sysreg(r) ({  \
+   u64 __val;  \
+   asm volatile("mrs %0, " __stringify(r) : "=r" (__val)); \
+   __val;  \
+})
+
+#define write_sysreg(v, r) do {\
+   u64 __val = (u64)v; \
+   asm volatile("msr " __stringify(r) ", %0"   \
+: : "r" (__val));  \
+} while (0)
+
 #endif
 
 #endif /* __ASM_SYSREG_H */
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 09/21] arm64: KVM: Implement guest entry

[Marc Zyngier]

Contrary to the previous patch, the guest entry is fairly different
from its assembly counterpart, mostly because it is only concerned
with saving/restoring the GP registers, and nothing else.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile |   1 +
 arch/arm64/kvm/hyp/entry.S  | 155 
 arch/arm64/kvm/hyp/hyp.h|   2 +
 3 files changed, 158 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/entry.S

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index ec14cac..1e1ff06 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
new file mode 100644
index 000..2c4449a
--- /dev/null
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -0,0 +1,155 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define CPU_GP_REG_OFFSET(x)   (CPU_GP_REGS + x)
+#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
+
+   .text
+   .pushsection.hyp.text, "ax"
+
+.macro save_common_regs ctxt
+   stp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
+   stp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
+   stp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
+   stp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
+   stp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
+   stp x29, lr,  [\ctxt, #CPU_XREG_OFFSET(29)]
+.endm
+
+.macro restore_common_regs ctxt
+   ldp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
+   ldp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
+   ldp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
+   ldp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
+   ldp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
+   ldp x29, lr,  [\ctxt, #CPU_XREG_OFFSET(29)]
+.endm
+
+.macro save_host_regs reg
+   save_common_regs \reg
+.endm
+
+.macro restore_host_regs reg
+   restore_common_regs \reg
+.endm
+
+.macro save_guest_regs
+   // x0 is the vcpu address
+   // x1 is the return code, do not corrupt!
+   // x2 is the cpu context
+   // x3 is a tmp register
+   // Guest's x0-x3 are on the stack
+
+   add x2, x0, #VCPU_CONTEXT
+
+   // Compute base to save registers
+   stp x4, x5,   [x2, #CPU_XREG_OFFSET(4)]
+   stp x6, x7,   [x2, #CPU_XREG_OFFSET(6)]
+   stp x8, x9,   [x2, #CPU_XREG_OFFSET(8)]
+   stp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
+   stp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
+   stp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
+   stp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
+   str x18,  [x2, #CPU_XREG_OFFSET(18)]
+
+   pop x6, x7  // x2, x3
+   pop x4, x5  // x0, x1
+
+   stp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
+   stp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
+
+   save_common_regs x2
+.endm
+
+.macro restore_guest_regs
+   // Assume vcpu in x0, clobbers everything else
+
+   add x2, x0, #VCPU_CONTEXT
+
+   // Prepare x0-x3 for later restore
+   ldp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
+   ldp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
+   pushx4, x5  // Push x0-x3 on the stack
+   pushx6, x7
+
+   // x4-x18
+   ldp x4, x5,   [x2, #CPU_XREG_OFFSET(4)] 
+   ldp x6, x7,   [x2, #CPU_XREG_OFFSET(6)] 
+   ldp x8, x9,   [x2, #CPU_XREG_OFFSET(8)] 
+   ldp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
+   ldp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
+   ldp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
+   ldp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
+   ldr x18,  [x2, #CPU_XREG_OFFSET(18)]
+
+   // x19-x29, lr
+   restore_common_regs x2
+
+   // Last bits of the 64bit state
+   pop x2, x3
+   pop x0, x1
+
+   // Do not touch any register after this!
+.endm
+
+/*
+ * u64 __guest_enter(struct kvm_vcpu *vcpu,
+ *  struct kvm_cpu_context *host_ctxt);
+ */

[PATCH v2 05/21] arm64: KVM: Implement timer save/restore

[Marc Zyngier]

Implement the timer save restore as a direct translation of
the assembly code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile  |  1 +
 arch/arm64/kvm/hyp/hyp.h |  3 ++
 arch/arm64/kvm/hyp/timer-sr.c| 71 
 include/clocksource/arm_arch_timer.h |  6 +++
 4 files changed, 81 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/timer-sr.c

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index d1e38ce..455dc0a 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -4,3 +4,4 @@
 
 obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index a31cb6e..86aa5a2 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -33,5 +33,8 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
 void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
 
+void __timer_save_state(struct kvm_vcpu *vcpu);
+void __timer_restore_state(struct kvm_vcpu *vcpu);
+
 #endif /* __ARM64_KVM_HYP_H__ */
 
diff --git a/arch/arm64/kvm/hyp/timer-sr.c b/arch/arm64/kvm/hyp/timer-sr.c
new file mode 100644
index 000..8e2209c
--- /dev/null
+++ b/arch/arm64/kvm/hyp/timer-sr.c
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+
+#include "hyp.h"
+
+/* vcpu is already in the HYP VA space */
+void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
+{
+   struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+   struct arch_timer_cpu *timer = >arch.timer_cpu;
+
+   if (kvm->arch.timer.enabled) {
+   timer->cntv_ctl = read_sysreg(cntv_ctl_el0);
+   isb();
+   timer->cntv_cval = read_sysreg(cntv_cval_el0);
+   }
+
+   /* Disable the virtual timer */
+   write_sysreg(0, cntv_ctl_el0);
+
+   /* Allow physical timer/counter access for the host */
+   write_sysreg((read_sysreg(cnthctl_el2) | CNTHCTL_EL1PCTEN |
+ CNTHCTL_EL1PCEN),
+cnthctl_el2);
+
+   /* Clear cntvoff for the host */
+   write_sysreg(0, cntvoff_el2);
+}
+
+void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
+{
+   struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+   struct arch_timer_cpu *timer = >arch.timer_cpu;
+   u64 val;
+
+   /*
+* Disallow physical timer access for the guest
+* Physical counter access is allowed
+*/
+   val = read_sysreg(cnthctl_el2);
+   val &= ~CNTHCTL_EL1PCEN;
+   val |= CNTHCTL_EL1PCTEN;
+   write_sysreg(val, cnthctl_el2);
+
+   if (kvm->arch.timer.enabled) {
+   write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
+   write_sysreg(timer->cntv_cval, cntv_cval_el0);
+   isb();
+   write_sysreg(timer->cntv_ctl, cntv_ctl_el0);
+   }
+}
diff --git a/include/clocksource/arm_arch_timer.h 
b/include/clocksource/arm_arch_timer.h
index 9916d0e..25d0914 100644
--- a/include/clocksource/arm_arch_timer.h
+++ b/include/clocksource/arm_arch_timer.h
@@ -23,6 +23,12 @@
 #define ARCH_TIMER_CTRL_IT_MASK(1 << 1)
 #define ARCH_TIMER_CTRL_IT_STAT(1 << 2)
 
+#define CNTHCTL_EL1PCTEN   (1 << 0)
+#define CNTHCTL_EL1PCEN(1 << 1)
+#define CNTHCTL_EVNTEN (1 << 2)
+#define CNTHCTL_EVNTDIR(1 << 3)
+#define CNTHCTL_EVNTI  (0xF << 4)
+
 enum arch_timer_reg {
ARCH_TIMER_REG_CTRL,
ARCH_TIMER_REG_TVAL,
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP

[Marc Zyngier]

In order to run C code in HYP, we must make sure that the kernel's
RO section in mapped into HYP (otherwise things break badly).

Signed-off-by: Marc Zyngier 
---
 arch/arm/kvm/arm.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index e06fd29..d80aefe 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -44,6 +44,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #ifdef REQUIRES_VIRT
 __asm__(".arch_extension   virt");
@@ -1067,6 +1068,12 @@ static int init_hyp_mode(void)
goto out_free_mappings;
}
 
+   err = create_hyp_mappings(__start_rodata, __end_rodata);
+   if (err) {
+   kvm_err("Cannot map rodata section\n");
+   goto out_free_mappings;
+   }
+
/*
 * Map the Hyp stack pages
 */
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 16/21] arm64: KVM: Add compatibility aliases

[Marc Zyngier]

So far, we've implemented the new world switch with a completely
different namespace, so that we could have both implementation
compiled in.

Let's take things one step further by adding weak aliases that
have the same names as the original implementation. The weak
attributes allows the new implementation to be overriden by the
old one, and everything still work.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/debug-sr.c   | 3 +++
 arch/arm64/kvm/hyp/hyp-entry.S  | 3 +++
 arch/arm64/kvm/hyp/switch.c | 3 +++
 arch/arm64/kvm/hyp/tlb.c| 9 +
 arch/arm64/kvm/hyp/vgic-v3-sr.c | 3 +++
 5 files changed, 21 insertions(+)

diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index a0b2b99..afd0a53 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -128,3 +128,6 @@ u32 __hyp_text __debug_read_mdcr_el2(void)
 {
return read_sysreg(mdcr_el2);
 }
+
+__alias(__debug_read_mdcr_el2)
+u32 __weak __kvm_get_mdcr_el2(void);
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 39d6935..ace919b 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -184,6 +184,8 @@ ENDPROC(\label)
 
.align 11
 
+   .weak   __kvm_hyp_vector
+ENTRY(__kvm_hyp_vector)
 ENTRY(__hyp_vector)
ventry  el2t_sync_invalid   // Synchronous EL2t
ventry  el2t_irq_invalid// IRQ EL2t
@@ -205,3 +207,4 @@ ENTRY(__hyp_vector)
ventry  el1_fiq_invalid // FIQ 32-bit EL1
ventry  el1_error_invalid   // Error 32-bit EL1
 ENDPROC(__hyp_vector)
+ENDPROC(__kvm_hyp_vector)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 7b81089..c8ba370 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -141,6 +141,9 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
return exit_code;
 }
 
+__alias(__guest_run)
+int __weak __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
 static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx 
ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
 
 void __hyp_text __noreturn __hyp_panic(void)
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
index d4a07d0..2c279a8 100644
--- a/arch/arm64/kvm/hyp/tlb.c
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -47,6 +47,9 @@ void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, 
phys_addr_t ipa)
write_sysreg(0, vttbr_el2);
 }
 
+__alias(__tlb_flush_vmid_ipa)
+void __weak __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
+
 void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
 {
dsb(ishst);
@@ -63,6 +66,9 @@ void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
write_sysreg(0, vttbr_el2);
 }
 
+__alias(__tlb_flush_vmid)
+void __weak __kvm_tlb_flush_vmid(struct kvm *kvm);
+
 void __hyp_text __tlb_flush_vm_context(void)
 {
dsb(ishst);
@@ -70,3 +76,6 @@ void __hyp_text __tlb_flush_vm_context(void)
 "ic ialluis  ": : );
dsb(ish);
 }
+
+__alias(__tlb_flush_vm_context)
+void __weak __kvm_flush_vm_context(void);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index b490db5..1b0eedb 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -220,3 +220,6 @@ u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
 {
return read_gicreg(ICH_VTR_EL2);
 }
+
+__alias(__vgic_v3_read_ich_vtr_el2)
+u64 __weak __vgic_v3_get_ich_vtr_el2(void);
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore

[Marc Zyngier]

Implement the vgic-v3 save restore as a direct translation of
the assembly code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile |   1 +
 arch/arm64/kvm/hyp/hyp.h|   3 +
 arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 
 3 files changed, 226 insertions(+)
 create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index d8d5968..d1e38ce 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -3,3 +3,4 @@
 #
 
 obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 78f25c4..a31cb6e 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -30,5 +30,8 @@
 void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
 void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 
+void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+
 #endif /* __ARM64_KVM_HYP_H__ */
 
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
new file mode 100644
index 000..b490db5
--- /dev/null
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -0,0 +1,222 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+
+#include "hyp.h"
+
+/*
+ * We store LRs in reverse order to let the CPU deal with streaming
+ * access. Use this macro to make it look saner...
+ */
+#define LR_OFFSET(n)   (15 - n)
+
+#define read_gicreg(r) \
+   ({  \
+   u64 reg;\
+   asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
+   reg;\
+   })
+
+#define write_gicreg(v,r)  \
+   do {\
+   u64 __val = (v);\
+   asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
+   } while (0)
+
+/* vcpu is already in the HYP VA space */
+void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
+{
+   struct vgic_v3_cpu_if *cpu_if = >arch.vgic_cpu.vgic_v3;
+   u64 val;
+   u32 nr_lr, nr_pri;
+
+   /*
+* Make sure stores to the GIC via the memory mapped interface
+* are now visible to the system register interface.
+*/
+   dsb(st);
+
+   cpu_if->vgic_vmcr  = read_gicreg(ICH_VMCR_EL2);
+   cpu_if->vgic_misr  = read_gicreg(ICH_MISR_EL2);
+   cpu_if->vgic_eisr  = read_gicreg(ICH_EISR_EL2);
+   cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
+
+   write_gicreg(0, ICH_HCR_EL2);
+   val = read_gicreg(ICH_VTR_EL2);
+   nr_lr = val & 0xf;
+   nr_pri = ((u32)val >> 29) + 1;
+
+   switch (nr_lr) {
+   case 15:
+   cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
+   case 14:
+   cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
+   case 13:
+   cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
+   case 12:
+   cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
+   case 11:
+   cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
+   case 10:
+   cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
+   case 9:
+   cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
+   case 8:
+   cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
+   case 7:
+   cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
+   case 6:
+   cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
+   case 5:
+   cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
+   case 4:
+   cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
+   case 3:
+   cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
+   case 2:
+   cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
+   

[PATCH v2 07/21] arm64: KVM: Implement 32bit system register save/restore

[Marc Zyngier]

Implement the 32bit system register save restore as a direct
translation of the assembly code version.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/hyp.h   |  2 ++
 arch/arm64/kvm/hyp/sysreg-sr.c | 41 +
 2 files changed, 43 insertions(+)

diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 087d3a5..4639330 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -38,6 +38,8 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
 
 void __sysreg_save_state(struct kvm_cpu_context *ctxt);
 void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
+void __sysreg32_save_state(struct kvm_vcpu *vcpu);
+void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
 
 #endif /* __ARM64_KVM_HYP_H__ */
 
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index add8fcb..3f81a4d 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -88,3 +88,44 @@ void __hyp_text __sysreg_restore_state(struct 
kvm_cpu_context *ctxt)
write_sysreg(ctxt->gp_regs.elr_el1, elr_el1);
write_sysreg(ctxt->gp_regs.spsr[KVM_SPSR_EL1], spsr_el1);
 }
+
+void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
+{
+   if (!(read_sysreg(hcr_el2) & HCR_RW)) {
+   u64 *spsr = vcpu->arch.ctxt.gp_regs.spsr;
+   u64 *sysreg = vcpu->arch.ctxt.sys_regs;
+
+   spsr[KVM_SPSR_ABT] = read_sysreg(spsr_abt);
+   spsr[KVM_SPSR_UND] = read_sysreg(spsr_und);
+   spsr[KVM_SPSR_IRQ] = read_sysreg(spsr_irq);
+   spsr[KVM_SPSR_FIQ] = read_sysreg(spsr_fiq);
+
+   sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
+   sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
+
+   if (!(read_sysreg(cptr_el2) & CPTR_EL2_TFP))
+   sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
+
+   if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
+   sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
+   }
+}
+
+void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
+{
+   if (!(read_sysreg(hcr_el2) & HCR_RW)) {
+   u64 *spsr = vcpu->arch.ctxt.gp_regs.spsr;
+   u64 *sysreg = vcpu->arch.ctxt.sys_regs;
+
+   write_sysreg(spsr[KVM_SPSR_ABT], spsr_abt);
+   write_sysreg(spsr[KVM_SPSR_UND], spsr_und);
+   write_sysreg(spsr[KVM_SPSR_IRQ], spsr_irq);
+   write_sysreg(spsr[KVM_SPSR_FIQ], spsr_fiq);
+
+   write_sysreg(sysreg[DACR32_EL2], dacr32_el2);
+   write_sysreg(sysreg[IFSR32_EL2], ifsr32_el2);
+
+   if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
+   write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
+   }
+}
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore

[Marc Zyngier]

Implement the fpsimd save restore, keeping the lazy part in
assembler (as returning to C would be overkill).

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/Makefile|  1 +
 arch/arm64/kvm/hyp/entry.S | 32 +++-
 arch/arm64/kvm/hyp/fpsimd.S| 33 +
 arch/arm64/kvm/hyp/hyp.h   |  7 +++
 arch/arm64/kvm/hyp/switch.c|  8 
 arch/arm64/kvm/hyp/sysreg-sr.c |  2 +-
 6 files changed, 81 insertions(+), 2 deletions(-)
 create mode 100644 arch/arm64/kvm/hyp/fpsimd.S

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 9c11b0f..56238d0 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
 obj-$(CONFIG_KVM_ARM_HOST) += switch.o
+obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 2c4449a..7552922 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -27,6 +27,7 @@
 
 #define CPU_GP_REG_OFFSET(x)   (CPU_GP_REGS + x)
 #define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
+#define CPU_SYSREG_OFFSET(x)   (CPU_SYSREGS + 8*x)
 
.text
.pushsection.hyp.text, "ax"
@@ -152,4 +153,33 @@ ENTRY(__guest_exit)
ret
 ENDPROC(__guest_exit)
 
-   /* Insert fault handling here */
+ENTRY(__fpsimd_guest_restore)
+   pushx4, lr
+
+   mrs x2, cptr_el2
+   bic x2, x2, #CPTR_EL2_TFP
+   msr cptr_el2, x2
+   isb
+
+   mrs x3, tpidr_el2
+
+   ldr x0, [x3, #VCPU_HOST_CONTEXT]
+   kern_hyp_va x0
+   add x0, x0, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
+   bl  __fpsimd_save_state
+
+   add x2, x3, #VCPU_CONTEXT
+   add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
+   bl  __fpsimd_restore_state
+
+   mrs x1, hcr_el2
+   tbnzx1, #HCR_RW_SHIFT, 1f
+   ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
+   msr fpexc32_el2, x4
+1:
+   pop x4, lr
+   pop x2, x3
+   pop x0, x1
+
+   eret
+ENDPROC(__fpsimd_guest_restore)
diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S
new file mode 100644
index 000..da3f22c
--- /dev/null
+++ b/arch/arm64/kvm/hyp/fpsimd.S
@@ -0,0 +1,33 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include 
+
+#include 
+
+   .text
+   .pushsection.hyp.text, "ax"
+
+ENTRY(__fpsimd_save_state)
+   fpsimd_save x0, 1
+   ret
+ENDPROC(__fpsimd_save_state)
+
+ENTRY(__fpsimd_restore_state)
+   fpsimd_restore  x0, 1
+   ret
+ENDPROC(__fpsimd_restore_state)
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index f0427ee..18365dd 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -66,6 +66,13 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
 void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
 void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
 
+void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
+void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
+static inline bool __fpsimd_enabled(void)
+{
+   return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
+}
+
 u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
 
 #endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index d67ed9e..8affc19 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -88,6 +88,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
 {
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
+   bool fp_enabled;
u64 exit_code;
 
vcpu = kern_hyp_va(vcpu);
@@ -117,6 +118,8 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
exit_code = __guest_enter(vcpu, host_ctxt);
/* And we're baaack! */
 
+   fp_enabled = __fpsimd_enabled();
+
__sysreg_save_state(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_save_state(vcpu);
@@ -127,6 +130,11 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
 
__sysreg_restore_state(host_ctxt);
 
+   if (fp_enabled) {
+ 

[PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch

[Marc Zyngier]

This is it. We remove all of the code that has now been rewritten.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/Makefile |2 -
 arch/arm64/kvm/hyp.S| 1081 +--
 arch/arm64/kvm/vgic-v2-switch.S |  134 -
 arch/arm64/kvm/vgic-v3-switch.S |  269 --
 4 files changed, 1 insertion(+), 1485 deletions(-)
 delete mode 100644 arch/arm64/kvm/vgic-v2-switch.S
 delete mode 100644 arch/arm64/kvm/vgic-v3-switch.S

diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index d31e4e5..caee9ee 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -23,8 +23,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += guest.o debug.o reset.o 
sys_regs.o sys_regs_generi
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v2.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v2-emul.o
-kvm-$(CONFIG_KVM_ARM_HOST) += vgic-v2-switch.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v3.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v3-emul.o
-kvm-$(CONFIG_KVM_ARM_HOST) += vgic-v3-switch.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
index 86c2898..0ccdcbb 100644
--- a/arch/arm64/kvm/hyp.S
+++ b/arch/arm64/kvm/hyp.S
@@ -17,910 +17,7 @@
 
 #include 
 
-#include 
-#include 
 #include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#define CPU_GP_REG_OFFSET(x)   (CPU_GP_REGS + x)
-#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
-#define CPU_SPSR_OFFSET(x) CPU_GP_REG_OFFSET(CPU_SPSR + 8*x)
-#define CPU_SYSREG_OFFSET(x)   (CPU_SYSREGS + 8*x)
-
-   .text
-   .pushsection.hyp.text, "ax"
-   .align  PAGE_SHIFT
-
-.macro save_common_regs
-   // x2: base address for cpu context
-   // x3: tmp register
-
-   add x3, x2, #CPU_XREG_OFFSET(19)
-   stp x19, x20, [x3]
-   stp x21, x22, [x3, #16]
-   stp x23, x24, [x3, #32]
-   stp x25, x26, [x3, #48]
-   stp x27, x28, [x3, #64]
-   stp x29, lr, [x3, #80]
-
-   mrs x19, sp_el0
-   mrs x20, elr_el2// pc before entering el2
-   mrs x21, spsr_el2   // pstate before entering el2
-
-   stp x19, x20, [x3, #96]
-   str x21, [x3, #112]
-
-   mrs x22, sp_el1
-   mrs x23, elr_el1
-   mrs x24, spsr_el1
-
-   str x22, [x2, #CPU_GP_REG_OFFSET(CPU_SP_EL1)]
-   str x23, [x2, #CPU_GP_REG_OFFSET(CPU_ELR_EL1)]
-   str x24, [x2, #CPU_SPSR_OFFSET(KVM_SPSR_EL1)]
-.endm
-
-.macro restore_common_regs
-   // x2: base address for cpu context
-   // x3: tmp register
-
-   ldr x22, [x2, #CPU_GP_REG_OFFSET(CPU_SP_EL1)]
-   ldr x23, [x2, #CPU_GP_REG_OFFSET(CPU_ELR_EL1)]
-   ldr x24, [x2, #CPU_SPSR_OFFSET(KVM_SPSR_EL1)]
-
-   msr sp_el1, x22
-   msr elr_el1, x23
-   msr spsr_el1, x24
-
-   add x3, x2, #CPU_XREG_OFFSET(31)// SP_EL0
-   ldp x19, x20, [x3]
-   ldr x21, [x3, #16]
-
-   msr sp_el0, x19
-   msr elr_el2, x20// pc on return from el2
-   msr spsr_el2, x21   // pstate on return from el2
-
-   add x3, x2, #CPU_XREG_OFFSET(19)
-   ldp x19, x20, [x3]
-   ldp x21, x22, [x3, #16]
-   ldp x23, x24, [x3, #32]
-   ldp x25, x26, [x3, #48]
-   ldp x27, x28, [x3, #64]
-   ldp x29, lr, [x3, #80]
-.endm
-
-.macro save_host_regs
-   save_common_regs
-.endm
-
-.macro restore_host_regs
-   restore_common_regs
-.endm
-
-.macro save_fpsimd
-   // x2: cpu context address
-   // x3, x4: tmp regs
-   add x3, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
-   fpsimd_save x3, 4
-.endm
-
-.macro restore_fpsimd
-   // x2: cpu context address
-   // x3, x4: tmp regs
-   add x3, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
-   fpsimd_restore x3, 4
-.endm
-
-.macro save_guest_regs
-   // x0 is the vcpu address
-   // x1 is the return code, do not corrupt!
-   // x2 is the cpu context
-   // x3 is a tmp register
-   // Guest's x0-x3 are on the stack
-
-   // Compute base to save registers
-   add x3, x2, #CPU_XREG_OFFSET(4)
-   stp x4, x5, [x3]
-   stp x6, x7, [x3, #16]
-   stp x8, x9, [x3, #32]
-   stp x10, x11, [x3, #48]
-   stp x12, x13, [x3, #64]
-   stp x14, x15, [x3, #80]
-   stp x16, x17, [x3, #96]
-   str x18, [x3, #112]
-
-   pop x6, x7  // x2, x3
-   pop x4, x5  // x0, x1
-
-   add x3, x2, #CPU_XREG_OFFSET(0)
-   stp x4, x5, [x3]
-   stp x6, x7, [x3, #16]
-
-   save_common_regs
-.endm
-
-.macro restore_guest_regs
-   // x0 is the vcpu address.
-   // x2 is the cpu context
-   // x3 is 

[PATCH v2 21/21] arm64: KVM: Remove weak attributes

[Marc Zyngier]

As we've now switched to the new world switch implementation,
remove the weak attributes, as nobody is supposed to override
it anymore.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/debug-sr.c   |  5 ++---
 arch/arm64/kvm/hyp/hyp-entry.S  |  3 ---
 arch/arm64/kvm/hyp/switch.c |  5 ++---
 arch/arm64/kvm/hyp/tlb.c| 16 +++-
 arch/arm64/kvm/hyp/vgic-v3-sr.c |  5 ++---
 5 files changed, 13 insertions(+), 21 deletions(-)

diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index 774a3f69..747546b 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -125,10 +125,9 @@ void __hyp_text __debug_cond_restore_host_state(struct 
kvm_vcpu *vcpu)
}
 }
 
-u32 __hyp_text __debug_read_mdcr_el2(void)
+static u32 __hyp_text __debug_read_mdcr_el2(void)
 {
return read_sysreg(mdcr_el2);
 }
 
-__alias(__debug_read_mdcr_el2)
-u32 __weak __kvm_get_mdcr_el2(void);
+__alias(__debug_read_mdcr_el2) u32 __kvm_get_mdcr_el2(void);
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index ace919b..bbc0be1 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -184,9 +184,7 @@ ENDPROC(\label)
 
.align 11
 
-   .weak   __kvm_hyp_vector
 ENTRY(__kvm_hyp_vector)
-ENTRY(__hyp_vector)
ventry  el2t_sync_invalid   // Synchronous EL2t
ventry  el2t_irq_invalid// IRQ EL2t
ventry  el2t_fiq_invalid// FIQ EL2t
@@ -206,5 +204,4 @@ ENTRY(__hyp_vector)
ventry  el1_irq // IRQ 32-bit EL1
ventry  el1_fiq_invalid // FIQ 32-bit EL1
ventry  el1_error_invalid   // Error 32-bit EL1
-ENDPROC(__hyp_vector)
 ENDPROC(__kvm_hyp_vector)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index c8ba370..1154d66 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -84,7 +84,7 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu 
*vcpu)
__vgic_call_restore_state()(vcpu);
 }
 
-int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
+static int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
 {
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
@@ -141,8 +141,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
return exit_code;
 }
 
-__alias(__guest_run)
-int __weak __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+__alias(__guest_run) int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 
 static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx 
ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
 
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
index 2c279a8..250e06c 100644
--- a/arch/arm64/kvm/hyp/tlb.c
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -17,7 +17,7 @@
 
 #include "hyp.h"
 
-void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
+static void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
 {
dsb(ishst);
 
@@ -47,10 +47,10 @@ void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, 
phys_addr_t ipa)
write_sysreg(0, vttbr_el2);
 }
 
-__alias(__tlb_flush_vmid_ipa)
-void __weak __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
+__alias(__tlb_flush_vmid_ipa) void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm,
+   phys_addr_t ipa);
 
-void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
+static void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
 {
dsb(ishst);
 
@@ -66,10 +66,9 @@ void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
write_sysreg(0, vttbr_el2);
 }
 
-__alias(__tlb_flush_vmid)
-void __weak __kvm_tlb_flush_vmid(struct kvm *kvm);
+__alias(__tlb_flush_vmid) void __kvm_tlb_flush_vmid(struct kvm *kvm);
 
-void __hyp_text __tlb_flush_vm_context(void)
+static void __hyp_text __tlb_flush_vm_context(void)
 {
dsb(ishst);
asm volatile("tlbi alle1is  \n"
@@ -77,5 +76,4 @@ void __hyp_text __tlb_flush_vm_context(void)
dsb(ish);
 }
 
-__alias(__tlb_flush_vm_context)
-void __weak __kvm_flush_vm_context(void);
+__alias(__tlb_flush_vm_context) void __kvm_flush_vm_context(void);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 1b0eedb..82a4f4b 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -216,10 +216,9 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu 
*vcpu)
}
 }
 
-u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
+static u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
 {
return read_gicreg(ICH_VTR_EL2);
 }
 
-__alias(__vgic_v3_read_ich_vtr_el2)
-u64 __weak __vgic_v3_get_ich_vtr_el2(void);
+__alias(__vgic_v3_read_ich_vtr_el2) u64 __vgic_v3_get_ich_vtr_el2(void);
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to 

[PATCH v2 10/21] arm64: KVM: Add patchable function selector

[Marc Zyngier]

KVM so far relies on code patching, and is likely to use it more
in the future. The main issue is that our alternative system works
at the instruction level, while we'd like to have alternatives at
the function level.

In order to cope with this, add the "hyp_alternate_select" macro that
outputs a brief sequence of code that in turn can be patched, allowing
al alternative function to be selected.

Signed-off-by: Marc Zyngier 
---
 arch/arm64/kvm/hyp/hyp.h | 16 
 1 file changed, 16 insertions(+)

diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 7ac8e11..f0427ee 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -27,6 +27,22 @@
 
 #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
 
+/*
+ * Generates patchable code sequences that are used to switch between
+ * two implementations of a function, depending on the availability of
+ * a feature.
+ */
+#define hyp_alternate_select(fname, orig, alt, cond)   \
+typeof(orig) * __hyp_text fname(void)  \
+{  \
+   typeof(alt) *val = orig;\
+   asm volatile(ALTERNATIVE("nop   \n",\
+"mov   %0, %1  \n",\
+cond)  \
+: "+r" (val) : "r" (alt)); \
+   return val; \
+}
+
 void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
 void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] KVM: Create debugfs dir and stat files for each VM

[Tyler Baker]

On 27 November 2015 at 09:08, Tyler Baker  wrote:
> On 27 November 2015 at 00:54, Christian Borntraeger
>  wrote:
>> On 11/26/2015 09:47 PM, Christian Borntraeger wrote:
>>> On 11/26/2015 05:17 PM, Tyler Baker wrote:
 Hi Christian,

 The kernelci.org bot recently has been reporting kvm guest boot
 failures[1] on various arm64 platforms in next-20151126. The bot
 bisected[2] the failures to the commit in -next titled "KVM: Create
 debugfs dir and stat files for each VM". I confirmed by reverting this
 commit on top of next-20151126 it resolves the boot issue.

 In this test case the host and guest are booted with the same kernel.
 The host is booted over nfs, installs qemu (qemu-system arm64 2.4.0),
 and launches a guest. The host is booting fine, but when the guest is
 launched it errors with "Failed to retrieve host CPU features!". I
 checked the host logs, and found an "Unable to handle kernel paging
 request" splat[3] which occurs when the guest is attempting to start.

 I scanned the patch in question but nothing obvious jumped out at me,
 any thoughts?
>>>
>>> Not really.
>>> Do you have processing running that do read the files in 
>>> /sys/kernel/debug/kvm/* ?
>>>
>>> If I read the arm oops message correctly it oopsed inside
>>> __srcu_read_lock. there is actually nothing in there that can oops,
>>> except the access to the preempt count. I am just guessing right now,
>>> but maybe the preempt variable is no longer available (as the process
>>> is gone). As long as a debugfs file is open, we hold a reference to
>>> the kvm, which holds a reference to the mm, so the mm might be killed
>>> after the process. But this is supposed to work, so maybe its something
>>> different. An objdump of __srcu_read_lock might help.
>>
>> Hmm, the preempt thing is done in srcu_read_lock, but the crash is in
>> __srcu_read_lock. This function gets the srcu struct from mmu_notifier.c,
>> which must be present and is initialized during boot.
>>
>>
>> int __srcu_read_lock(struct srcu_struct *sp)
>> {
>> int idx;
>>
>> idx = READ_ONCE(sp->completed) & 0x1;
>> __this_cpu_inc(sp->per_cpu_ref->c[idx]);
>> smp_mb(); /* B */  /* Avoid leaking the critical section. */
>> __this_cpu_inc(sp->per_cpu_ref->seq[idx]);
>> return idx;
>> }
>>
>> Looking at the code I have no clue why the patch does make a difference.
>> Can you try to get an objdump -S for__Srcu_read_lock?

Some other interesting finding below...

On the host, I do _not_ have any nodes under /sys/kernel/debug/kvm/

Running strace on the qemu command I use to launch the guest yields
the following.

[pid  5963] 1448649724.405537 mmap(NULL, 65536, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6652a000
[pid  5963] 1448649724.405586 read(13, "MemTotal:   16414616
kB\nMemF"..., 1024) = 1024
[pid  5963] 1448649724.405699 close(13) = 0
[pid  5963] 1448649724.405755 munmap(0x7f6652a000, 65536) = 0
[pid  5963] 1448649724.405947 brk(0x2552f000) = 0x2552f000
[pid  5963] 1448649724.406148 openat(AT_FDCWD, "/dev/kvm",
O_RDWR|O_CLOEXEC) = 13
[pid  5963] 1448649724.406209 ioctl(13, KVM_CREATE_VM, 0) = -1 ENOMEM
(Cannot allocate memory)
[pid  5963] 1448649724.406382 close(13) = 0
[pid  5963] 1448649724.406435 write(2, "Failed to retrieve host CPU
feat"..., 38Failed to retrieve host CPU features!
) = 38

Tyler
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v1 1/7] drivers/hv: Move HV_SYNIC_STIMER_COUNT into Hyper-V UAPI x86 header

[KY Srinivasan]

> -Original Message-
> From: Andrey Smetanin [mailto:asmeta...@virtuozzo.com]
> Sent: Wednesday, November 25, 2015 7:20 AM
> To: kvm@vger.kernel.org
> Cc: Gleb Natapov ; Paolo Bonzini
> ; KY Srinivasan ; Haiyang
> Zhang ; Vitaly Kuznetsov
> ; Roman Kagan ; Denis V.
> Lunev ; qemu-de...@nongnu.org
> Subject: [PATCH v1 1/7] drivers/hv: Move HV_SYNIC_STIMER_COUNT into
> Hyper-V UAPI x86 header
> 
> This constant is required for Hyper-V SynIC timers MSR's
> support by userspace(QEMU).
> 
> Signed-off-by: Andrey Smetanin 
> Reviewed-by: Roman Kagan 

Acked-by:  K. Y. Srinivasan 
> CC: Gleb Natapov 
> CC: Paolo Bonzini 
> CC: "K. Y. Srinivasan" 
> CC: Haiyang Zhang 
> CC: Vitaly Kuznetsov 
> CC: Roman Kagan 
> CC: Denis V. Lunev 
> CC: qemu-de...@nongnu.org
> ---
>  arch/x86/include/uapi/asm/hyperv.h | 2 ++
>  drivers/hv/hyperv_vmbus.h  | 2 --
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/include/uapi/asm/hyperv.h
> b/arch/x86/include/uapi/asm/hyperv.h
> index 040d408..07981f0 100644
> --- a/arch/x86/include/uapi/asm/hyperv.h
> +++ b/arch/x86/include/uapi/asm/hyperv.h
> @@ -269,4 +269,6 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
>  #define HV_SYNIC_SINT_AUTO_EOI   (1ULL << 17)
>  #define HV_SYNIC_SINT_VECTOR_MASK(0xFF)
> 
> +#define HV_SYNIC_STIMER_COUNT(4)
> +
>  #endif
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 3782636..46e23d1 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -102,8 +102,6 @@ enum hv_message_type {
>   HVMSG_X64_LEGACY_FP_ERROR   = 0x80010005
>  };
> 
> -#define HV_SYNIC_STIMER_COUNT(4)
> -
>  /* Define invalid partition identifier. */
>  #define HV_PARTITION_ID_INVALID  ((u64)0x0)
> 
> --
> 2.4.3

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[KY Srinivasan]

> -Original Message-
> From: Paolo Bonzini [mailto:paolo.bonz...@gmail.com] On Behalf Of Paolo
> Bonzini
> Sent: Friday, November 27, 2015 1:35 AM
> To: Andrey Smetanin ; kvm@vger.kernel.org
> Cc: Gleb Natapov ; KY Srinivasan ;
> Haiyang Zhang ; Vitaly Kuznetsov
> ; Roman Kagan ; Denis V.
> Lunev ; qemu-de...@nongnu.org
> Subject: Re: [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI
> Hyper-V x86 header
> 
> 
> 
> On 25/11/2015 16:20, Andrey Smetanin wrote:
> > This struct is required for Hyper-V SynIC timers implementation inside KVM
> > and for upcoming Hyper-V VMBus support by userspace(QEMU). So place
> it into
> > Hyper-V UAPI header.
> >
> > Signed-off-by: Andrey Smetanin 
> > Reviewed-by: Roman Kagan 
Acked-by: K. Y. Srinivasan 

> > CC: Gleb Natapov 
> > CC: Paolo Bonzini 
> > CC: "K. Y. Srinivasan" 
> > CC: Haiyang Zhang 
> > CC: Vitaly Kuznetsov 
> > CC: Roman Kagan 
> > CC: Denis V. Lunev 
> > CC: qemu-de...@nongnu.org
> > ---
> >  arch/x86/include/uapi/asm/hyperv.h | 91
> ++
> >  drivers/hv/hyperv_vmbus.h  | 91 
> > --
> >  2 files changed, 91 insertions(+), 91 deletions(-)
> >
> > diff --git a/arch/x86/include/uapi/asm/hyperv.h
> b/arch/x86/include/uapi/asm/hyperv.h
> > index 07981f0..e86d77e 100644
> > --- a/arch/x86/include/uapi/asm/hyperv.h
> > +++ b/arch/x86/include/uapi/asm/hyperv.h
> > @@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
> >
> >  #define HV_SYNIC_STIMER_COUNT  (4)
> >
> > +/* Define synthetic interrupt controller message constants. */
> > +#define HV_MESSAGE_SIZE(256)
> > +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT  (240)
> > +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT (30)
> > +
> > +/* Define hypervisor message types. */
> > +enum hv_message_type {
> > +   HVMSG_NONE  = 0x,
> > +
> > +   /* Memory access messages. */
> > +   HVMSG_UNMAPPED_GPA  = 0x8000,
> > +   HVMSG_GPA_INTERCEPT = 0x8001,
> > +
> > +   /* Timer notification messages. */
> > +   HVMSG_TIMER_EXPIRED = 0x8010,
> > +
> > +   /* Error messages. */
> > +   HVMSG_INVALID_VP_REGISTER_VALUE = 0x8020,
> > +   HVMSG_UNRECOVERABLE_EXCEPTION   = 0x8021,
> > +   HVMSG_UNSUPPORTED_FEATURE   = 0x8022,
> > +
> > +   /* Trace buffer complete messages. */
> > +   HVMSG_EVENTLOG_BUFFERCOMPLETE   = 0x8040,
> > +
> > +   /* Platform-specific processor intercept messages. */
> > +   HVMSG_X64_IOPORT_INTERCEPT  = 0x8001,
> > +   HVMSG_X64_MSR_INTERCEPT = 0x80010001,
> > +   HVMSG_X64_CPUID_INTERCEPT   = 0x80010002,
> > +   HVMSG_X64_EXCEPTION_INTERCEPT   = 0x80010003,
> > +   HVMSG_X64_APIC_EOI  = 0x80010004,
> > +   HVMSG_X64_LEGACY_FP_ERROR   = 0x80010005
> > +};
> > +
> > +/* Define synthetic interrupt controller message flags. */
> > +union hv_message_flags {
> > +   __u8 asu8;
> > +   struct {
> > +   __u8 msg_pending:1;
> > +   __u8 reserved:7;
> > +   };
> > +};
> > +
> > +/* Define port identifier type. */
> > +union hv_port_id {
> > +   __u32 asu32;
> > +   struct {
> > +   __u32 id:24;
> > +   __u32 reserved:8;
> > +   } u;
> > +};
> > +
> > +/* Define port type. */
> > +enum hv_port_type {
> > +   HVPORT_MSG  = 1,
> > +   HVPORT_EVENT= 2,
> > +   HVPORT_MONITOR  = 3
> > +};
> > +
> > +/* Define synthetic interrupt controller message header. */
> > +struct hv_message_header {
> > +   enum hv_message_type message_type;
> 
> Do not declare this as an enum, declare it as __u32 to make the size
> portable.  It can be a patch on top.
> 
> KY, can you ack these two patches?
> 
> Paolo
> 
> > +   __u8 payload_size;
> > +   union hv_message_flags message_flags;
> > +   __u8 reserved[2];
> > +   union {
> > +   __u64 sender;
> > +   union hv_port_id port;
> > +   };
> > +};
> > +
> > +/* Define timer message payload structure. */
> > +struct hv_timer_message_payload {
> > +   __u32 timer_index;
> > +   __u32 reserved;
> > +   __u64 expiration_time;  /* When the timer expired */
> > +   __u64 delivery_time;/* When the message was delivered */
> > +};
> > +
> > +/* Define synthetic interrupt controller message format. */
> > +struct hv_message {
> > +   struct hv_message_header header;
> > +   union {
> > +   __u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
> > +   } u;
> > +};
> > +
> > +/* Define the synthetic interrupt message page layout. */
> > +struct hv_message_page {
> > +   

Re: [PATCH v1 0/5] KVM-UNIT-TESTS: Hyper-V SynIC timers test

[Paolo Bonzini]

On 26/11/2015 17:29, Andrey Smetanin wrote:
> The test checks Hyper-V SynIC timers functionality.
> The test runs on every vCPU and performs start/stop
> of periodic/one-shot timers (with period=1ms) and checks
> validity of received expiration messages in appropriate
> ISR's.
> 
> Signed-off-by: Andrey Smetanin 
> Reviewed-by: Roman Kagan 
> CC: Paolo Bonzini 
> CC: Marcelo Tosatti 
> CC: Roman Kagan 
> CC: Denis V. Lunev 
> CC: qemu-de...@nongnu.org
> 
> Andrey Smetanin (5):
>   lib/x86: Added Hyper-V MSR's availability bits into msr.h
>   lib/x86: Added HV_X64_MSR_TIME_REF_COUNT value into msr.h
>   lib/x86: Added Hyper-V SynIC timers MSR's values
>   lib/x86: Make free_page() available to call
>   x86: Hyper-V SynIC timers test

In addition to my comments on 5/5, can you make instead a hyperv.h file
with all that you need in the two testcases?

Paolo

>  config/config-x86-common.mak |   4 +-
>  lib/x86/msr.h|  19 ++
>  lib/x86/vm.h |   1 +
>  x86/hyperv_stimer.c  | 500 
> +++
>  x86/hyperv_synic.c   |   2 +-
>  x86/unittests.cfg|   5 +
>  6 files changed, 529 insertions(+), 2 deletions(-)
>  create mode 100644 x86/hyperv_stimer.c
> 
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 5/5] x86: Hyper-V SynIC timers test

[Paolo Bonzini]

On 27/11/2015 12:30, Andrey Smetanin wrote:
>>>
>>> +
>>> +static void stimer_test_cleanup(void *ctx)
>>> +{
>>> +irq_enable();
>>
>> Why enable again?
> I'll remove it.

I guess you can remove the one in stimer_test_prepare too.  If the
interrupts are disabled you don't get the IPI either, do you?

Paolo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[Andrey Smetanin]

On 11/27/2015 12:34 PM, Paolo Bonzini wrote:



On 25/11/2015 16:20, Andrey Smetanin wrote:

This struct is required for Hyper-V SynIC timers implementation inside KVM
and for upcoming Hyper-V VMBus support by userspace(QEMU). So place it into
Hyper-V UAPI header.

Signed-off-by: Andrey Smetanin 
Reviewed-by: Roman Kagan 
CC: Gleb Natapov 
CC: Paolo Bonzini 
CC: "K. Y. Srinivasan" 
CC: Haiyang Zhang 
CC: Vitaly Kuznetsov 
CC: Roman Kagan 
CC: Denis V. Lunev 
CC: qemu-de...@nongnu.org
---
  arch/x86/include/uapi/asm/hyperv.h | 91 ++
  drivers/hv/hyperv_vmbus.h  | 91 --
  2 files changed, 91 insertions(+), 91 deletions(-)

diff --git a/arch/x86/include/uapi/asm/hyperv.h 
b/arch/x86/include/uapi/asm/hyperv.h
index 07981f0..e86d77e 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {

  #define HV_SYNIC_STIMER_COUNT (4)

+/* Define synthetic interrupt controller message constants. */
+#define HV_MESSAGE_SIZE(256)
+#define HV_MESSAGE_PAYLOAD_BYTE_COUNT  (240)
+#define HV_MESSAGE_PAYLOAD_QWORD_COUNT (30)
+
+/* Define hypervisor message types. */
+enum hv_message_type {
+   HVMSG_NONE  = 0x,
+
+   /* Memory access messages. */
+   HVMSG_UNMAPPED_GPA  = 0x8000,
+   HVMSG_GPA_INTERCEPT = 0x8001,
+
+   /* Timer notification messages. */
+   HVMSG_TIMER_EXPIRED = 0x8010,
+
+   /* Error messages. */
+   HVMSG_INVALID_VP_REGISTER_VALUE = 0x8020,
+   HVMSG_UNRECOVERABLE_EXCEPTION   = 0x8021,
+   HVMSG_UNSUPPORTED_FEATURE   = 0x8022,
+
+   /* Trace buffer complete messages. */
+   HVMSG_EVENTLOG_BUFFERCOMPLETE   = 0x8040,
+
+   /* Platform-specific processor intercept messages. */
+   HVMSG_X64_IOPORT_INTERCEPT  = 0x8001,
+   HVMSG_X64_MSR_INTERCEPT = 0x80010001,
+   HVMSG_X64_CPUID_INTERCEPT   = 0x80010002,
+   HVMSG_X64_EXCEPTION_INTERCEPT   = 0x80010003,
+   HVMSG_X64_APIC_EOI  = 0x80010004,
+   HVMSG_X64_LEGACY_FP_ERROR   = 0x80010005
+};
+
+/* Define synthetic interrupt controller message flags. */
+union hv_message_flags {
+   __u8 asu8;
+   struct {
+   __u8 msg_pending:1;
+   __u8 reserved:7;
+   };
+};
+
+/* Define port identifier type. */
+union hv_port_id {
+   __u32 asu32;
+   struct {
+   __u32 id:24;
+   __u32 reserved:8;
+   } u;
+};
+
+/* Define port type. */
+enum hv_port_type {
+   HVPORT_MSG  = 1,
+   HVPORT_EVENT= 2,
+   HVPORT_MONITOR  = 3
+};
+
+/* Define synthetic interrupt controller message header. */
+struct hv_message_header {
+   enum hv_message_type message_type;


Do not declare this as an enum, declare it as __u32 to make the size
portable.  It can be a patch on top.

Ok, I'll prepare such patch.


KY, can you ack these two patches?

Paolo


+   __u8 payload_size;
+   union hv_message_flags message_flags;
+   __u8 reserved[2];
+   union {
+   __u64 sender;
+   union hv_port_id port;
+   };
+};
+
+/* Define timer message payload structure. */
+struct hv_timer_message_payload {
+   __u32 timer_index;
+   __u32 reserved;
+   __u64 expiration_time;  /* When the timer expired */
+   __u64 delivery_time;/* When the message was delivered */
+};
+
+/* Define synthetic interrupt controller message format. */
+struct hv_message {
+   struct hv_message_header header;
+   union {
+   __u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
+   } u;
+};
+
+/* Define the synthetic interrupt message page layout. */
+struct hv_message_page {
+   struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
+};
+
  #endif
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 46e23d1..d22230c 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -63,10 +63,6 @@ enum hv_cpuid_function {
  /* Define version of the synthetic interrupt controller. */
  #define HV_SYNIC_VERSION  (1)

-/* Define synthetic interrupt controller message constants. */
-#define HV_MESSAGE_SIZE(256)
-#define HV_MESSAGE_PAYLOAD_BYTE_COUNT  (240)
-#define HV_MESSAGE_PAYLOAD_QWORD_COUNT (30)
  #define HV_ANY_VP (0x)

  /* Define synthetic interrupt controller flag constants. */
@@ -74,53 +70,9 @@ enum hv_cpuid_function {
  #define HV_EVENT_FLAGS_BYTE_COUNT (256)
  #define HV_EVENT_FLAGS_DWORD_COUNT(256 / 

Re: [PATCH v1 5/5] x86: Hyper-V SynIC timers test

[Paolo Bonzini]

The test logic is good, but the glue can be improved a bit so that the
output is more useful if it breaks.

On 26/11/2015 17:29, Andrey Smetanin wrote:
> The test checks Hyper-V SynIC timers functionality.
> The test runs on every vCPU and performs start/stop
> of periodic/one-shot timers (with period=1ms) and checks
> validity of received expiration messages in appropriate
> ISR's.
> 
> Signed-off-by: Andrey Smetanin 
> Reviewed-by: Roman Kagan 
> CC: Paolo Bonzini 
> CC: Marcelo Tosatti 
> CC: Roman Kagan 
> CC: Denis V. Lunev 
> CC: qemu-de...@nongnu.org
> 
> ---
>  config/config-x86-common.mak |   4 +-
>  x86/hyperv_stimer.c  | 500 
> +++
>  x86/unittests.cfg|   5 +
>  3 files changed, 508 insertions(+), 1 deletion(-)
>  create mode 100644 x86/hyperv_stimer.c
> 
> diff --git a/config/config-x86-common.mak b/config/config-x86-common.mak
> index f64874d..a75be87 100644
> --- a/config/config-x86-common.mak
> +++ b/config/config-x86-common.mak
> @@ -37,7 +37,7 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat 
> \
> $(TEST_DIR)/s3.flat $(TEST_DIR)/pmu.flat \
> $(TEST_DIR)/tsc_adjust.flat $(TEST_DIR)/asyncpf.flat \
> $(TEST_DIR)/init.flat $(TEST_DIR)/smap.flat \
> -   $(TEST_DIR)/hyperv_synic.flat
> +   $(TEST_DIR)/hyperv_synic.flat $(TEST_DIR)/hyperv_stimer.flat \
>  
>  ifdef API
>  tests-common += api/api-sample
> @@ -115,6 +115,8 @@ $(TEST_DIR)/memory.elf: $(cstart.o) $(TEST_DIR)/memory.o
>  
>  $(TEST_DIR)/hyperv_synic.elf: $(cstart.o) $(TEST_DIR)/hyperv_synic.o
>  
> +$(TEST_DIR)/hyperv_stimer.elf: $(cstart.o) $(TEST_DIR)/hyperv_stimer.o
> +
>  arch_clean:
>   $(RM) $(TEST_DIR)/*.o $(TEST_DIR)/*.flat $(TEST_DIR)/*.elf \
>   $(TEST_DIR)/.*.d lib/x86/.*.d
> diff --git a/x86/hyperv_stimer.c b/x86/hyperv_stimer.c
> new file mode 100644
> index 000..e9186ca
> --- /dev/null
> +++ b/x86/hyperv_stimer.c
> @@ -0,0 +1,500 @@
> +#include "libcflat.h"
> +#include "processor.h"
> +#include "msr.h"
> +#include "isr.h"
> +#include "vm.h"
> +#include "apic.h"
> +#include "desc.h"
> +#include "io.h"
> +#include "smp.h"
> +#include "atomic.h"
> +
> +#define MAX_CPUS 4
> +#define HYPERV_CPUID_FEATURES   0x4003
> +
> +#define HV_SYNIC_CONTROL_ENABLE (1ULL << 0)
> +#define HV_SYNIC_SIMP_ENABLE(1ULL << 0)
> +#define HV_SYNIC_SIEFP_ENABLE   (1ULL << 0)
> +#define HV_SYNIC_SINT_MASKED(1ULL << 16)
> +#define HV_SYNIC_SINT_AUTO_EOI  (1ULL << 17)
> +#define HV_SYNIC_SINT_VECTOR_MASK   (0xFF)
> +#define HV_SYNIC_SINT_COUNT 16
> +
> +#define HV_STIMER_ENABLE(1ULL << 0)
> +#define HV_STIMER_PERIODIC  (1ULL << 1)
> +#define HV_STIMER_LAZY  (1ULL << 2)
> +#define HV_STIMER_AUTOENABLE(1ULL << 3)
> +#define HV_STIMER_SINT(config)  (__u8)(((config) >> 16) & 0x0F)
> +
> +#define HV_SYNIC_STIMER_COUNT   (4)
> +
> +/* Define synthetic interrupt controller message constants. */
> +#define HV_MESSAGE_SIZE (256)
> +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT   (240)
> +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT  (30)
> +
> +/* Define hypervisor message types. */
> +enum hv_message_type {
> +HVMSG_NONE  = 0x,
> +
> +/* Memory access messages. */
> +HVMSG_UNMAPPED_GPA  = 0x8000,
> +HVMSG_GPA_INTERCEPT = 0x8001,
> +
> +/* Timer notification messages. */
> +HVMSG_TIMER_EXPIRED = 0x8010,
> +
> +/* Error messages. */
> +HVMSG_INVALID_VP_REGISTER_VALUE = 0x8020,
> +HVMSG_UNRECOVERABLE_EXCEPTION   = 0x8021,
> +HVMSG_UNSUPPORTED_FEATURE   = 0x8022,
> +
> +/* Trace buffer complete messages. */
> +HVMSG_EVENTLOG_BUFFERCOMPLETE   = 0x8040,
> +
> +/* Platform-specific processor intercept messages. */
> +HVMSG_X64_IOPORT_INTERCEPT  = 0x8001,
> +HVMSG_X64_MSR_INTERCEPT = 0x80010001,
> +HVMSG_X64_CPUID_INTERCEPT   = 0x80010002,
> +HVMSG_X64_EXCEPTION_INTERCEPT   = 0x80010003,
> +HVMSG_X64_APIC_EOI  = 0x80010004,
> +HVMSG_X64_LEGACY_FP_ERROR   = 0x80010005
> +};
> +
> +/* Define synthetic interrupt controller message flags. */
> +union hv_message_flags {
> +uint8_t asu8;
> +struct {
> +uint8_t msg_pending:1;
> +uint8_t reserved:7;
> +};
> +};
> +
> +union hv_port_id {
> +uint32_t asu32;
> +struct {
> +uint32_t id:24;
> +uint32_t 

Re: [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Andrey Smetanin]

On 11/27/2015 01:49 PM, Paolo Bonzini wrote:



On 27/11/2015 09:12, Roman Kagan wrote:

+   n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
+   stimer->exp_time += n * stimer->count;

This is actually just a reminder calculation so I'd rather do it
directly with div64_u64_rem().


It took me a while to understand why it was a remained. :)  Expanding
Andrey's formula you get

exp_time = exp_time + count + (time_now - exp_time) / count * count

the remainder, (time_now - exp_time) % count would be

time_now - exp_time - (time_now - exp_time) / count * count

so -((time_now - exp_time) % count) is

exp_time - time_now + (time_now - exp_time) / count * count

so Andrey's expression is

exp_time = time_now + (count - (time_now - exp_time) % count)

Yeah, that looks nice.

I'll redo this way.


Paolo


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 0/5] KVM-UNIT-TESTS: Hyper-V SynIC timers test

[Andrey Smetanin]

On 11/27/2015 02:17 PM, Paolo Bonzini wrote:



On 26/11/2015 17:29, Andrey Smetanin wrote:

The test checks Hyper-V SynIC timers functionality.
The test runs on every vCPU and performs start/stop
of periodic/one-shot timers (with period=1ms) and checks
validity of received expiration messages in appropriate
ISR's.

Signed-off-by: Andrey Smetanin 
Reviewed-by: Roman Kagan 
CC: Paolo Bonzini 
CC: Marcelo Tosatti 
CC: Roman Kagan 
CC: Denis V. Lunev 
CC: qemu-de...@nongnu.org

Andrey Smetanin (5):
   lib/x86: Added Hyper-V MSR's availability bits into msr.h
   lib/x86: Added HV_X64_MSR_TIME_REF_COUNT value into msr.h
   lib/x86: Added Hyper-V SynIC timers MSR's values
   lib/x86: Make free_page() available to call
   x86: Hyper-V SynIC timers test


In addition to my comments on 5/5, can you make instead a hyperv.h file
with all that you need in the two testcases?

Yes, I'll do it. Thanks!


Paolo


  config/config-x86-common.mak |   4 +-
  lib/x86/msr.h|  19 ++
  lib/x86/vm.h |   1 +
  x86/hyperv_stimer.c  | 500 +++
  x86/hyperv_synic.c   |   2 +-
  x86/unittests.cfg|   5 +
  6 files changed, 529 insertions(+), 2 deletions(-)
  create mode 100644 x86/hyperv_stimer.c


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 5/5] x86: Hyper-V SynIC timers test

[Andrey Smetanin]

On 11/27/2015 02:17 PM, Paolo Bonzini wrote:

The test logic is good, but the glue can be improved a bit so that the
output is more useful if it breaks.

Thanks for comments below. I'll redo this patch.


On 26/11/2015 17:29, Andrey Smetanin wrote:

The test checks Hyper-V SynIC timers functionality.
The test runs on every vCPU and performs start/stop
of periodic/one-shot timers (with period=1ms) and checks
validity of received expiration messages in appropriate
ISR's.

Signed-off-by: Andrey Smetanin 
Reviewed-by: Roman Kagan 
CC: Paolo Bonzini 
CC: Marcelo Tosatti 
CC: Roman Kagan 
CC: Denis V. Lunev 
CC: qemu-de...@nongnu.org

---
  config/config-x86-common.mak |   4 +-
  x86/hyperv_stimer.c  | 500 +++
  x86/unittests.cfg|   5 +
  3 files changed, 508 insertions(+), 1 deletion(-)
  create mode 100644 x86/hyperv_stimer.c

diff --git a/config/config-x86-common.mak b/config/config-x86-common.mak
index f64874d..a75be87 100644
--- a/config/config-x86-common.mak
+++ b/config/config-x86-common.mak
@@ -37,7 +37,7 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \
 $(TEST_DIR)/s3.flat $(TEST_DIR)/pmu.flat \
 $(TEST_DIR)/tsc_adjust.flat $(TEST_DIR)/asyncpf.flat \
 $(TEST_DIR)/init.flat $(TEST_DIR)/smap.flat \
-   $(TEST_DIR)/hyperv_synic.flat
+   $(TEST_DIR)/hyperv_synic.flat $(TEST_DIR)/hyperv_stimer.flat \

  ifdef API
  tests-common += api/api-sample
@@ -115,6 +115,8 @@ $(TEST_DIR)/memory.elf: $(cstart.o) $(TEST_DIR)/memory.o

  $(TEST_DIR)/hyperv_synic.elf: $(cstart.o) $(TEST_DIR)/hyperv_synic.o

+$(TEST_DIR)/hyperv_stimer.elf: $(cstart.o) $(TEST_DIR)/hyperv_stimer.o
+
  arch_clean:
$(RM) $(TEST_DIR)/*.o $(TEST_DIR)/*.flat $(TEST_DIR)/*.elf \
$(TEST_DIR)/.*.d lib/x86/.*.d
diff --git a/x86/hyperv_stimer.c b/x86/hyperv_stimer.c
new file mode 100644
index 000..e9186ca
--- /dev/null
+++ b/x86/hyperv_stimer.c
@@ -0,0 +1,500 @@
+#include "libcflat.h"
+#include "processor.h"
+#include "msr.h"
+#include "isr.h"
+#include "vm.h"
+#include "apic.h"
+#include "desc.h"
+#include "io.h"
+#include "smp.h"
+#include "atomic.h"
+
+#define MAX_CPUS 4
+#define HYPERV_CPUID_FEATURES   0x4003
+
+#define HV_SYNIC_CONTROL_ENABLE (1ULL << 0)
+#define HV_SYNIC_SIMP_ENABLE(1ULL << 0)
+#define HV_SYNIC_SIEFP_ENABLE   (1ULL << 0)
+#define HV_SYNIC_SINT_MASKED(1ULL << 16)
+#define HV_SYNIC_SINT_AUTO_EOI  (1ULL << 17)
+#define HV_SYNIC_SINT_VECTOR_MASK   (0xFF)
+#define HV_SYNIC_SINT_COUNT 16
+
+#define HV_STIMER_ENABLE(1ULL << 0)
+#define HV_STIMER_PERIODIC  (1ULL << 1)
+#define HV_STIMER_LAZY  (1ULL << 2)
+#define HV_STIMER_AUTOENABLE(1ULL << 3)
+#define HV_STIMER_SINT(config)  (__u8)(((config) >> 16) & 0x0F)
+
+#define HV_SYNIC_STIMER_COUNT   (4)
+
+/* Define synthetic interrupt controller message constants. */
+#define HV_MESSAGE_SIZE (256)
+#define HV_MESSAGE_PAYLOAD_BYTE_COUNT   (240)
+#define HV_MESSAGE_PAYLOAD_QWORD_COUNT  (30)
+
+/* Define hypervisor message types. */
+enum hv_message_type {
+HVMSG_NONE  = 0x,
+
+/* Memory access messages. */
+HVMSG_UNMAPPED_GPA  = 0x8000,
+HVMSG_GPA_INTERCEPT = 0x8001,
+
+/* Timer notification messages. */
+HVMSG_TIMER_EXPIRED = 0x8010,
+
+/* Error messages. */
+HVMSG_INVALID_VP_REGISTER_VALUE = 0x8020,
+HVMSG_UNRECOVERABLE_EXCEPTION   = 0x8021,
+HVMSG_UNSUPPORTED_FEATURE   = 0x8022,
+
+/* Trace buffer complete messages. */
+HVMSG_EVENTLOG_BUFFERCOMPLETE   = 0x8040,
+
+/* Platform-specific processor intercept messages. */
+HVMSG_X64_IOPORT_INTERCEPT  = 0x8001,
+HVMSG_X64_MSR_INTERCEPT = 0x80010001,
+HVMSG_X64_CPUID_INTERCEPT   = 0x80010002,
+HVMSG_X64_EXCEPTION_INTERCEPT   = 0x80010003,
+HVMSG_X64_APIC_EOI  = 0x80010004,
+HVMSG_X64_LEGACY_FP_ERROR   = 0x80010005
+};
+
+/* Define synthetic interrupt controller message flags. */
+union hv_message_flags {
+uint8_t asu8;
+struct {
+uint8_t msg_pending:1;
+uint8_t reserved:7;
+};
+};
+
+union hv_port_id {
+uint32_t asu32;
+struct {
+uint32_t id:24;
+uint32_t reserved:8;
+} u;
+};
+
+/* Define port type. */
+enum hv_port_type {
+HVPORT_MSG  = 1,
+HVPORT_EVENT= 2,
+ 

Re: [PATCH 2/2] KVM: Create debugfs dir and stat files for each VM

[Tyler Baker]

On 27 November 2015 at 10:53, Tyler Baker  wrote:
> On 27 November 2015 at 09:08, Tyler Baker  wrote:
>> On 27 November 2015 at 00:54, Christian Borntraeger
>>  wrote:
>>> On 11/26/2015 09:47 PM, Christian Borntraeger wrote:
 On 11/26/2015 05:17 PM, Tyler Baker wrote:
> Hi Christian,
>
> The kernelci.org bot recently has been reporting kvm guest boot
> failures[1] on various arm64 platforms in next-20151126. The bot
> bisected[2] the failures to the commit in -next titled "KVM: Create
> debugfs dir and stat files for each VM". I confirmed by reverting this
> commit on top of next-20151126 it resolves the boot issue.
>
> In this test case the host and guest are booted with the same kernel.
> The host is booted over nfs, installs qemu (qemu-system arm64 2.4.0),
> and launches a guest. The host is booting fine, but when the guest is
> launched it errors with "Failed to retrieve host CPU features!". I
> checked the host logs, and found an "Unable to handle kernel paging
> request" splat[3] which occurs when the guest is attempting to start.
>
> I scanned the patch in question but nothing obvious jumped out at me,
> any thoughts?

 Not really.
 Do you have processing running that do read the files in 
 /sys/kernel/debug/kvm/* ?

 If I read the arm oops message correctly it oopsed inside
 __srcu_read_lock. there is actually nothing in there that can oops,
 except the access to the preempt count. I am just guessing right now,
 but maybe the preempt variable is no longer available (as the process
 is gone). As long as a debugfs file is open, we hold a reference to
 the kvm, which holds a reference to the mm, so the mm might be killed
 after the process. But this is supposed to work, so maybe its something
 different. An objdump of __srcu_read_lock might help.
>>>
>>> Hmm, the preempt thing is done in srcu_read_lock, but the crash is in
>>> __srcu_read_lock. This function gets the srcu struct from mmu_notifier.c,
>>> which must be present and is initialized during boot.
>>>
>>>
>>> int __srcu_read_lock(struct srcu_struct *sp)
>>> {
>>> int idx;
>>>
>>> idx = READ_ONCE(sp->completed) & 0x1;
>>> __this_cpu_inc(sp->per_cpu_ref->c[idx]);
>>> smp_mb(); /* B */  /* Avoid leaking the critical section. */
>>> __this_cpu_inc(sp->per_cpu_ref->seq[idx]);
>>> return idx;
>>> }
>>>
>>> Looking at the code I have no clue why the patch does make a difference.
>>> Can you try to get an objdump -S for__Srcu_read_lock?
>
> Some other interesting finding below...
>
> On the host, I do _not_ have any nodes under /sys/kernel/debug/kvm/
>
> Running strace on the qemu command I use to launch the guest yields
> the following.
>
> [pid  5963] 1448649724.405537 mmap(NULL, 65536, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6652a000
> [pid  5963] 1448649724.405586 read(13, "MemTotal:   16414616
> kB\nMemF"..., 1024) = 1024
> [pid  5963] 1448649724.405699 close(13) = 0
> [pid  5963] 1448649724.405755 munmap(0x7f6652a000, 65536) = 0
> [pid  5963] 1448649724.405947 brk(0x2552f000) = 0x2552f000
> [pid  5963] 1448649724.406148 openat(AT_FDCWD, "/dev/kvm",
> O_RDWR|O_CLOEXEC) = 13
> [pid  5963] 1448649724.406209 ioctl(13, KVM_CREATE_VM, 0) = -1 ENOMEM
> (Cannot allocate memory)

If I comment the call to kvm_create_vm_debugfs(kvm) the guest boots
fine. I put some printk's in the kvm_create_vm_debugfs() function and
it's returning -ENOMEM after it evaluates !kvm->debugfs_dentry. I was
chatting with some folks from the Linaro virtualization team and they
mentioned that ARM is a bit special as the same PID creates two vms in
quick succession, the first one is a scratch vm, and the other is the
'real' vm. With that bit of info, I suspect we may be trying to create
the debugfs directory twice, and the second time it's failing because
it already exists.

Cheers,

Tyler
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[Paolo Bonzini]

On 25/11/2015 16:20, Andrey Smetanin wrote:
> This struct is required for Hyper-V SynIC timers implementation inside KVM
> and for upcoming Hyper-V VMBus support by userspace(QEMU). So place it into
> Hyper-V UAPI header.
> 
> Signed-off-by: Andrey Smetanin 
> Reviewed-by: Roman Kagan 
> CC: Gleb Natapov 
> CC: Paolo Bonzini 
> CC: "K. Y. Srinivasan" 
> CC: Haiyang Zhang 
> CC: Vitaly Kuznetsov 
> CC: Roman Kagan 
> CC: Denis V. Lunev 
> CC: qemu-de...@nongnu.org
> ---
>  arch/x86/include/uapi/asm/hyperv.h | 91 
> ++
>  drivers/hv/hyperv_vmbus.h  | 91 
> --
>  2 files changed, 91 insertions(+), 91 deletions(-)
> 
> diff --git a/arch/x86/include/uapi/asm/hyperv.h 
> b/arch/x86/include/uapi/asm/hyperv.h
> index 07981f0..e86d77e 100644
> --- a/arch/x86/include/uapi/asm/hyperv.h
> +++ b/arch/x86/include/uapi/asm/hyperv.h
> @@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
>  
>  #define HV_SYNIC_STIMER_COUNT(4)
>  
> +/* Define synthetic interrupt controller message constants. */
> +#define HV_MESSAGE_SIZE  (256)
> +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT(240)
> +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT   (30)
> +
> +/* Define hypervisor message types. */
> +enum hv_message_type {
> + HVMSG_NONE  = 0x,
> +
> + /* Memory access messages. */
> + HVMSG_UNMAPPED_GPA  = 0x8000,
> + HVMSG_GPA_INTERCEPT = 0x8001,
> +
> + /* Timer notification messages. */
> + HVMSG_TIMER_EXPIRED = 0x8010,
> +
> + /* Error messages. */
> + HVMSG_INVALID_VP_REGISTER_VALUE = 0x8020,
> + HVMSG_UNRECOVERABLE_EXCEPTION   = 0x8021,
> + HVMSG_UNSUPPORTED_FEATURE   = 0x8022,
> +
> + /* Trace buffer complete messages. */
> + HVMSG_EVENTLOG_BUFFERCOMPLETE   = 0x8040,
> +
> + /* Platform-specific processor intercept messages. */
> + HVMSG_X64_IOPORT_INTERCEPT  = 0x8001,
> + HVMSG_X64_MSR_INTERCEPT = 0x80010001,
> + HVMSG_X64_CPUID_INTERCEPT   = 0x80010002,
> + HVMSG_X64_EXCEPTION_INTERCEPT   = 0x80010003,
> + HVMSG_X64_APIC_EOI  = 0x80010004,
> + HVMSG_X64_LEGACY_FP_ERROR   = 0x80010005
> +};
> +
> +/* Define synthetic interrupt controller message flags. */
> +union hv_message_flags {
> + __u8 asu8;
> + struct {
> + __u8 msg_pending:1;
> + __u8 reserved:7;
> + };
> +};
> +
> +/* Define port identifier type. */
> +union hv_port_id {
> + __u32 asu32;
> + struct {
> + __u32 id:24;
> + __u32 reserved:8;
> + } u;
> +};
> +
> +/* Define port type. */
> +enum hv_port_type {
> + HVPORT_MSG  = 1,
> + HVPORT_EVENT= 2,
> + HVPORT_MONITOR  = 3
> +};
> +
> +/* Define synthetic interrupt controller message header. */
> +struct hv_message_header {
> + enum hv_message_type message_type;

Do not declare this as an enum, declare it as __u32 to make the size
portable.  It can be a patch on top.

KY, can you ack these two patches?

Paolo

> + __u8 payload_size;
> + union hv_message_flags message_flags;
> + __u8 reserved[2];
> + union {
> + __u64 sender;
> + union hv_port_id port;
> + };
> +};
> +
> +/* Define timer message payload structure. */
> +struct hv_timer_message_payload {
> + __u32 timer_index;
> + __u32 reserved;
> + __u64 expiration_time;  /* When the timer expired */
> + __u64 delivery_time;/* When the message was delivered */
> +};
> +
> +/* Define synthetic interrupt controller message format. */
> +struct hv_message {
> + struct hv_message_header header;
> + union {
> + __u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
> + } u;
> +};
> +
> +/* Define the synthetic interrupt message page layout. */
> +struct hv_message_page {
> + struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
> +};
> +
>  #endif
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 46e23d1..d22230c 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -63,10 +63,6 @@ enum hv_cpuid_function {
>  /* Define version of the synthetic interrupt controller. */
>  #define HV_SYNIC_VERSION (1)
>  
> -/* Define synthetic interrupt controller message constants. */
> -#define HV_MESSAGE_SIZE  (256)
> -#define HV_MESSAGE_PAYLOAD_BYTE_COUNT(240)
> -#define HV_MESSAGE_PAYLOAD_QWORD_COUNT   (30)
>  #define HV_ANY_VP(0x)
>  
>  /* Define synthetic interrupt controller flag constants. */
> @@ -74,53 +70,9 @@ enum 

Re: [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Roman Kagan]

On Wed, Nov 25, 2015 at 06:20:21PM +0300, Andrey Smetanin wrote:
> Per Hyper-V specification (and as required by Hyper-V-aware guests),
> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
> of MSRs, and signals expiration by delivering a special format message
> to the configured SynIC message slot and triggering the corresponding
> synthetic interrupt.
> 
> Note: as implemented by this patch, all periodic timers are "lazy"
> (i.e. if the vCPU wasn't scheduled for more than the timer period the
> timer events are lost), regardless of the corresponding configuration
> MSR.  If deemed necessary, the "catch up" mode (the timer period is
> shortened until the timer catches up) will be implemented later.
> 
> Signed-off-by: Andrey Smetanin 
> CC: Gleb Natapov 
> CC: Paolo Bonzini 
> CC: "K. Y. Srinivasan" 
> CC: Haiyang Zhang 
> CC: Vitaly Kuznetsov 
> CC: Roman Kagan 
> CC: Denis V. Lunev 
> CC: qemu-de...@nongnu.org
> ---
>  arch/x86/include/asm/kvm_host.h|  13 ++
>  arch/x86/include/uapi/asm/hyperv.h |   6 +
>  arch/x86/kvm/hyperv.c  | 325 
> -
>  arch/x86/kvm/hyperv.h  |  24 +++
>  arch/x86/kvm/x86.c |   9 +
>  include/linux/kvm_host.h   |   1 +
>  6 files changed, 375 insertions(+), 3 deletions(-)

A couple of nitpicks:

> +static void stimer_restart(struct kvm_vcpu_hv_stimer *stimer)
> +{
> + u64 time_now;
> + ktime_t ktime_now;
> + u64 n;
> +
> + time_now = get_time_ref_counter(stimer_to_vcpu(stimer)->kvm);
> + ktime_now = ktime_get();
> +
> + /*
> +  * Calculate positive integer n for which condtion -
> +  * (stimer->exp_time + n * stimer->count) > time_now
> +  * is true. We will use (stimer->exp_time + n * stimer->count)
> +  * as new stimer->exp_time.
> +  */
> +
> + n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
> + stimer->exp_time += n * stimer->count;

This is actually just a reminder calculation so I'd rather do it
directly with div64_u64_rem().

> +void kvm_hv_process_stimers(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_vcpu_hv *hv_vcpu = vcpu_to_hv_vcpu(vcpu);
> + struct kvm_vcpu_hv_stimer *stimer;
> + u64 time_now;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(hv_vcpu->stimer); i++)
> + if (test_and_clear_bit(i, hv_vcpu->stimer_pending_bitmap)) {
> + stimer = _vcpu->stimer[i];
> + stimer_stop(stimer);

I think there's no need in this explicit stop: I see no way to arrive
here with a running timer, and even if there were, it would be safe as
your timer callback only manipulates the bitmaps atomically.

Neither comment is critical so

Reviewed-by: Roman Kagan 

Roman.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Roman Kagan]

On Wed, Nov 25, 2015 at 06:20:20PM +0300, Andrey Smetanin wrote:
> The SynIC message protocol mandates that the message slot is claimed
> by atomically setting message type to something other than HVMSG_NONE.
> If another message is to be delivered while the slot is still busy,
> message pending flag is asserted to indicate to the guest that the
> hypervisor wants to be notified when the slot is released.
> 
> To make sure the protocol works regardless of where the message
> sources are (kernel or userspace), clear the pending flag on SINT ACK
> notification, and let the message sources compete for the slot again.
> 
> Signed-off-by: Andrey Smetanin 
> CC: Gleb Natapov 
> CC: Paolo Bonzini 
> CC: "K. Y. Srinivasan" 
> CC: Haiyang Zhang 
> CC: Vitaly Kuznetsov 
> CC: Roman Kagan 
> CC: Denis V. Lunev 
> CC: qemu-de...@nongnu.org

Reviewed-by: Roman Kagan 

Roman.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] KVM: Create debugfs dir and stat files for each VM

[Christian Borntraeger]

On 11/26/2015 09:47 PM, Christian Borntraeger wrote:
> On 11/26/2015 05:17 PM, Tyler Baker wrote:
>> Hi Christian,
>>
>> The kernelci.org bot recently has been reporting kvm guest boot
>> failures[1] on various arm64 platforms in next-20151126. The bot
>> bisected[2] the failures to the commit in -next titled "KVM: Create
>> debugfs dir and stat files for each VM". I confirmed by reverting this
>> commit on top of next-20151126 it resolves the boot issue.
>>
>> In this test case the host and guest are booted with the same kernel.
>> The host is booted over nfs, installs qemu (qemu-system arm64 2.4.0),
>> and launches a guest. The host is booting fine, but when the guest is
>> launched it errors with "Failed to retrieve host CPU features!". I
>> checked the host logs, and found an "Unable to handle kernel paging
>> request" splat[3] which occurs when the guest is attempting to start.
>>
>> I scanned the patch in question but nothing obvious jumped out at me,
>> any thoughts?
> 
> Not really.
> Do you have processing running that do read the files in 
> /sys/kernel/debug/kvm/* ?
> 
> If I read the arm oops message correctly it oopsed inside 
> __srcu_read_lock. there is actually nothing in there that can oops, 
> except the access to the preempt count. I am just guessing right now,
> but maybe the preempt variable is no longer available (as the process
> is gone). As long as a debugfs file is open, we hold a reference to 
> the kvm, which holds a reference to the mm, so the mm might be killed
> after the process. But this is supposed to work, so maybe its something
> different. An objdump of __srcu_read_lock might help.

Hmm, the preempt thing is done in srcu_read_lock, but the crash is in
__srcu_read_lock. This function gets the srcu struct from mmu_notifier.c,
which must be present and is initialized during boot.


int __srcu_read_lock(struct srcu_struct *sp)
{
int idx;

idx = READ_ONCE(sp->completed) & 0x1;
__this_cpu_inc(sp->per_cpu_ref->c[idx]);
smp_mb(); /* B */  /* Avoid leaking the critical section. */
__this_cpu_inc(sp->per_cpu_ref->seq[idx]);
return idx;
}

Looking at the code I have no clue why the patch does make a difference.
Can you try to get an objdump -S for__Srcu_read_lock?






> 
> I will drop it from my tree until we understand the problem
> 
> Christian
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Paolo Bonzini]

On 27/11/2015 09:12, Roman Kagan wrote:
>> > +  n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
>> > +  stimer->exp_time += n * stimer->count;
> This is actually just a reminder calculation so I'd rather do it
> directly with div64_u64_rem().

It took me a while to understand why it was a remained. :)  Expanding
Andrey's formula you get

exp_time = exp_time + count + (time_now - exp_time) / count * count

the remainder, (time_now - exp_time) % count would be

time_now - exp_time - (time_now - exp_time) / count * count

so -((time_now - exp_time) % count) is

exp_time - time_now + (time_now - exp_time) / count * count

so Andrey's expression is

exp_time = time_now + (count - (time_now - exp_time) % count)

Yeah, that looks nice.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html