Re: [RFC PATCH v2 37/38] KVM: arm64: Respect the virtual HCR_EL2.NV1 bit setting

On Tue, Jul 18, 2017 at 12:59 PM, Jintack Lim wrote: > Forward ELR_EL1, SPSR_EL1 and VBAR_EL1 traps to the virtual EL2 if the > virtual HCR_EL2.NV bit is set. > > This is for recursive nested virtualization. > > Signed-off-by: Jintack Lim This

Re: [RFC PATCH v2 00/38] Nested Virtualization on KVM/ARM

On Tue, Jul 18, 2017 at 12:58 PM, Jintack Lim wrote: > Nested virtualization is the ability to run a virtual machine inside another > virtual machine. In other words, it’s about running a hypervisor (the guest > hypervisor) on top of another hypervisor (the host

[RFC PATCH v2 37/38] KVM: arm64: Respect the virtual HCR_EL2.NV1 bit setting

Forward ELR_EL1, SPSR_EL1 and VBAR_EL1 traps to the virtual EL2 if the virtual HCR_EL2.NV bit is set. This is for recursive nested virtualization. Signed-off-by: Jintack Lim --- arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/kvm/sys_regs.c| 18

[RFC PATCH v2 23/38] KVM: arm64: Inject HVC exceptions to the virtual EL2

Now that the psci call is done by the smc instruction when nested virtualization is enabled, it is clear that all hvc instruction from the VM (including from the virtual EL2) are supposed to handled in the virtual EL2. Signed-off-by: Jintack Lim ---

[RFC PATCH v2 38/38] KVM: arm64: Respect the virtual CPTR_EL2.TCPAC setting

Forward CPACR_EL1 traps to the virtual EL2 if virtual CPTR_EL2 is configured to trap CPACR_EL1 accesses from EL1. This is for recursive nested virtualization. Signed-off-by: Jintack Lim --- arch/arm64/kvm/sys_regs.c | 5 + 1 file changed, 5 insertions(+) diff --git

[RFC PATCH v2 34/38] KVM: arm64: Respect the virtual HCR_EL2.NV bit setting

Forward traps due to HCR_EL2.NV bit to the virtual EL2 if they are not coming from the virtual EL2 and the virtual HCR_EL2.NV bit is set. This is for recursive nested virtualization. Signed-off-by: Jintack Lim --- arch/arm64/include/asm/kvm_arm.h| 1 +

[RFC PATCH v2 35/38] KVM: arm64: Respect the virtual HCR_EL2.NV bit setting for EL12 register traps

In addition to EL2 register accesses, setting NV bit will also make EL12 register accesses trap to EL2. To emulate this for the virtual EL2, forword traps due to EL12 register accessses to the virtual EL2 if the virtual HCR_EL2.NV bit is set. This is for recursive nested virtualization.

[RFC PATCH v2 33/38] KVM: arm64: Emulate appropriate VM control system registers

Now that the virtual EL2 can access EL2 register states via EL1 registers, we need to consider it when selecting the register to emulate. Signed-off-by: Jintack Lim --- arch/arm64/kvm/sys_regs.c | 46 -- 1 file changed, 44

[RFC PATCH v2 30/38] KVM: arm64: Allow the virtual EL2 to access EL2 states without trap

When the virtual E2H bit is set, we can support EL2 register accesses via EL1 registers from the virtual EL2 by doing trap-and-emulate. A better alternative, however, is to allow the virtual EL2 to access EL2 register states without trap. This can be easily achieved by not traping EL1 registers

[RFC PATCH v2 24/38] KVM: arm64: Respect virtual HCR_EL2.TWX setting

Forward exceptions due to WFI or WFE instructions to the virtual EL2 if they are not coming from the virtual EL2 and virtual HCR_EL2.TWX is set. Signed-off-by: Jintack Lim --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/handle_exit.c | 13 -

[RFC PATCH v2 31/38] KVM: arm64: Manage the shadow states when virtual E2H bit enabled

When creating the shadow context for the virtual EL2 execution, we can directly copy the EL2 register states to the shadow EL1 register states if the virtual HCR_EL2.E2H bit is set. This is because EL1 and EL2 system register formats compatible with E2H=1. Now that we allow the virtual EL2 modify

[RFC PATCH v2 32/38] KVM: arm64: Trap and emulate CPTR_EL2 accesses via CPACR_EL1 from the virtual EL2 with VHE

While the EL1 virtual memory control registers can be accessed in the virtual EL2 with VHE without trap to manuplate the virtual EL2 states, we can't do that for CPTR_EL2 for an unfortunate reason. This is because the top bit of CPTR_EL2, which is TCPAC, will be ignored if it is accessed via

[RFC PATCH v2 29/38] KVM: arm64: Support a VM with VHE considering EL0 of the VHE host

On VHE systems, EL0 of the host kernel is considered as a part of 'VHE host'; The execution of EL0 is affected by system registers set by the VHE kernel including the hypervisor. To emulate this for a VM, we use the same set of system registers (i.e. shadow registers) for the virtual EL2 and EL0

[RFC PATCH v2 27/38] KVM: arm64: Add EL2 registers defined in ARMv8.1 to vcpu context

ARMv8.1 added more EL2 registers: TTBR1_EL2, CONTEXTIDR_EL2, and three EL2 virtual timer registers. Add the first two registers to vcpu context and set their handlers. The timer registers and their handlers will be added in a separate patch. Signed-off-by: Jintack Lim ---

[RFC PATCH v2 22/38] KVM: arm64: Handle PSCI call via smc from the guest

VMs used to execute hvc #0 for the psci call if EL3 is not implemented. However, when we come to provide the virtual EL2 mode to the VM, the host OS inside the VM calls kvm_call_hyp() which is also hvc #0. So, it's hard to differentiate between them from the host hypervisor's point of view. So,

[RFC PATCH v2 26/38] KVM: arm64: Add macros to support the virtual EL2 with VHE

These macros will be used to support the virtual EL2 with VHE. Signed-off-by: Jintack Lim --- arch/arm64/include/asm/kvm_emulate.h | 24 1 file changed, 24 insertions(+) diff --git a/arch/arm64/include/asm/kvm_emulate.h

[RFC PATCH v2 25/38] KVM: arm64: Respect virtual CPTR_EL2.TFP setting

Forward traps due to FP/ASIMD register accesses to the virtual EL2 if virtual CPTR_EL2.TFP is set. Note that if TFP bit is set, then even accesses to FP/ASIMD register from EL2 as well as NS EL0/1 will trap to EL2. So, we don't check the VM's exception level. Signed-off-by: Jintack Lim

[RFC PATCH v2 28/38] KVM: arm64: Emulate EL12 register accesses from the virtual EL2

With HCR_EL2.NV bit set, accesses to EL12 registers in the virtual EL2 trap to EL2. Handle those traps just like we do for EL1 registers. One exception is CNTKCTL_EL12. We don't trap on CNTKCTL_EL1 for non-VHE virtual EL2 because we don't have to. However, accessing CNTKCTL_EL12 will trap since

[RFC PATCH v2 20/38] KVM: arm64: Handle eret instruction traps

When HCR.NV bit is set, eret instructions trap to EL2 with EC code 0x1A. Emulate eret instructions by setting pc and pstate. Note that the current exception level is always the virtual EL2, since we set HCR_EL2.NV bit only when entering the virtual EL2. So, we take spsr and elr states from the

[RFC PATCH v2 17/38] KVM: arm64: Trap EL1 VM register accesses in virtual EL2

From: Christoffer Dall When running in virtual EL2 mode, we actually run the hardware in EL1 and therefore have to use the EL1 registers to ensure correct operation. By setting the HCR.TVM and HCR.TVRM we ensure that the virtual EL2 mode doesn't shoot itself in the

[RFC PATCH v2 12/38] arm64: Add missing TCR hw defines

From: Christoffer Dall Some bits of the TCR weren't defined and since we're about to use these in KVM, add these defines. Signed-off-by: Christoffer Dall --- arch/arm64/include/asm/pgtable-hwdef.h | 6 ++ 1 file changed, 6

[RFC PATCH v2 21/38] KVM: arm64: Set a handler for the system instruction traps

When HCR.NV bit is set, execution of the EL2 translation regime address aranslation instructions and TLB maintenance instructions are trapped to EL2. In addition, execution of the EL1 translation regime address aranslation instructions and TLB maintenance instructions that are only accessible from

[RFC PATCH v2 18/38] KVM: arm64: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2

For the same reason we trap virtual memory register accesses at virtual EL2, we need to trap SPSR_EL1, ELR_EL1 and VBAR_EL1 accesses. ARM v8.3 introduces the HCR_EL2.NV1 bit to be able to trap on those register accesses in EL1. Do not set this bit until the whole nesting support is completed.

[RFC PATCH v2 16/38] KVM: arm64: Support to inject exceptions to the virtual EL2

Support inject synchronous exceptions to the virtual EL2 as described in ARM ARM AArch64.TakeException(). This can be easily extended to support to inject asynchronous exceptions to the virtual EL2, but it will be added in a later patch when appropriate. Signed-off-by: Jintack Lim

[RFC PATCH v2 11/38] KVM: arm64: Set vcpu context depending on the guest exception level

If the guest exception level is EL2, then set up the shadow context of the virtual EL2 to hardware. Otherwise, set the regular EL0/EL1 context. Note that the shadow context content will be prepared in subsequent patches. Signed-off-by: Jintack Lim ---

[RFC PATCH v2 13/38] KVM: arm64: Create shadow EL1 registers

From: Christoffer Dall When entering virtual EL2, we need to reflect virtual EL2 register states to corresponding shadow EL1 registers. We can simply copy them if their formats are identical. Otherwise, we need to convert EL2 register state to EL1 register state.

[RFC PATCH v2 19/38] KVM: arm64: Trap CPACR_EL1 access in virtual EL2

For the same reason we trap virtual memory register accesses in virtual EL2, we trap CPACR_EL1 access too; We allow the virtual EL2 mode to access EL1 system register state instead of the virtual EL2 one. Signed-off-by: Jintack Lim --- arch/arm64/kvm/hyp/switch.c | 10

[RFC PATCH v2 15/38] KVM: arm64: Move exception macros and enums to a common file

These macros and enums can be reused to inject exceptions for nested virtualization. Signed-off-by: Jintack Lim --- arch/arm64/include/asm/kvm_emulate.h | 12 arch/arm64/kvm/inject_fault.c| 12 2 files changed, 12 insertions(+), 12

[RFC PATCH v2 10/38] KVM: arm/arm64: Add a framework to prepare virtual EL2 execution

From: Christoffer Dall Add functions setting up and restoring the guest's context on each entry and exit. These functions will come in handy when we want to use different context for normal EL0/EL1 and virtual EL2 execution. No functional change yet. Signed-off-by:

[RFC PATCH v2 14/38] KVM: arm64: Synchronize EL1 system registers on virtual EL2 entry and exit

When running in virtual EL2 we use the shadow EL1 systerm register array for the save/restore process, so that hardware and especially the memory subsystem behaves as code written for EL2 expects while really running in EL1. This works great for EL1 system register accesses that we trap, because

[RFC PATCH v2 09/38] KVM: arm64: Add the shadow context for virtual EL2 execution

With the nested virtualization support, a hypervisor running inside a VM (i.e. a guest hypervisor) is now deprivilaged and runs in EL1 instead of EL2. So, the host hypervisor manages the shadow context for the virtual EL2 execution. Signed-off-by: Jintack Lim ---

[RFC PATCH v2 08/38] KVM: arm64: Add EL2 special registers to vcpu context

To support the virtual EL2 execution, we need to maintain the EL2 special registers such as SPSR_EL2, ELR_EL2 and SP_EL2 in vcpu context. Note that SP_EL2 is not accessible in EL2, so we don't need a trap handler for this register. Signed-off-by: Jintack Lim ---

[RFC PATCH v2 05/38] KVM: arm64: Allow userspace to set PSR_MODE_EL2x

From: Christoffer Dall We were not allowing userspace to set a more privileged mode for the VCPU than EL1, but now that we support nesting with a virtual EL2 mode, do allow this! Signed-off-by: Christoffer Dall ---

[RFC PATCH v2 07/38] KVM: arm64: Add EL2 system registers to vcpu context

ARM v8.3 introduces a new bit in the HCR_EL2, which is the NV bit. When this bit is set, accessing EL2 registers in EL1 traps to EL2. In addition, executing the following instructions in EL1 will trap to EL2: tlbi, at, eret, and msr/mrs instructions to access SP_EL1. Most of the instructions that

[RFC PATCH v2 06/38] KVM: arm64: Add vcpu_mode_el2 primitive to support nesting

From: Christoffer Dall When running a nested hypervisor we occasionally have to figure out if the mode we are switching into is the virtual EL2 mode or a regular EL0/1 mode. Signed-off-by: Christoffer Dall ---

[RFC PATCH v2 03/38] KVM: arm64: Add KVM nesting feature

From: Christoffer Dall Set the initial exception level of the guest to EL2 if nested virtualization feature is enabled. Signed-off-by: Christoffer Dall Signed-off-by: Jintack Lim ---

[RFC PATCH v2 04/38] KVM: arm/arm64: Check if nested virtualization is in use

Nested virtualizaion is in use only if all three conditions are met: - The architecture supports nested virtualization. - The kernel parameter is set. - The userspace uses nested virtualiztion feature. Signed-off-by: Jintack Lim --- arch/arm/include/asm/kvm_host.h | 11

[RFC PATCH v2 00/38] Nested Virtualization on KVM/ARM

Nested virtualization is the ability to run a virtual machine inside another virtual machine. In other words, it’s about running a hypervisor (the guest hypervisor) on top of another hypervisor (the host hypervisor). Supporting nested virtualization on ARM means that the hypervisor provides not

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 4:29 PM, Andrew Jones wrote: > On Tue, Jul 18, 2017 at 04:15:23PM +0200, Christoffer Dall wrote: >> On Tue, Jul 18, 2017 at 3:50 PM, Andrew Jones wrote: >> > On Tue, Jul 18, 2017 at 03:31:03PM +0200, Christoffer Dall wrote: >> >> On

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 04:15:23PM +0200, Christoffer Dall wrote: > On Tue, Jul 18, 2017 at 3:50 PM, Andrew Jones wrote: > > On Tue, Jul 18, 2017 at 03:31:03PM +0200, Christoffer Dall wrote: > >> On Tue, Jul 18, 2017 at 03:23:24PM +0200, Andrew Jones wrote: > >> > On Tue, Jul

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 3:50 PM, Andrew Jones wrote: > On Tue, Jul 18, 2017 at 03:31:03PM +0200, Christoffer Dall wrote: >> On Tue, Jul 18, 2017 at 03:23:24PM +0200, Andrew Jones wrote: >> > On Tue, Jul 18, 2017 at 03:01:53PM +0200, Christoffer Dall wrote: >> > > On Tue, Jul

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 03:31:03PM +0200, Christoffer Dall wrote: > On Tue, Jul 18, 2017 at 03:23:24PM +0200, Andrew Jones wrote: > > On Tue, Jul 18, 2017 at 03:01:53PM +0200, Christoffer Dall wrote: > > > On Tue, Jul 18, 2017 at 02:09:57PM +0200, Andrew Jones wrote: > > > > On Thu, Jul 13, 2017

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 03:23:24PM +0200, Andrew Jones wrote: > On Tue, Jul 18, 2017 at 03:01:53PM +0200, Christoffer Dall wrote: > > On Tue, Jul 18, 2017 at 02:09:57PM +0200, Andrew Jones wrote: > > > On Thu, Jul 13, 2017 at 09:20:09PM +0200, Christoffer Dall wrote: > > > > Rearrange the code to

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 03:01:53PM +0200, Christoffer Dall wrote: > On Tue, Jul 18, 2017 at 02:09:57PM +0200, Andrew Jones wrote: > > On Thu, Jul 13, 2017 at 09:20:09PM +0200, Christoffer Dall wrote: > > > Rearrange the code to be able to reuse as much as posible and add > > > support for testing

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Tue, Jul 18, 2017 at 02:09:57PM +0200, Andrew Jones wrote: > On Thu, Jul 13, 2017 at 09:20:09PM +0200, Christoffer Dall wrote: > > Rearrange the code to be able to reuse as much as posible and add > > support for testing the physical timer as well. > > > > Also change the default unittests

[PATCH] KVM: arm/arm64: Fix guest external abort matching

The ARM-ARM has two bits in the ESR/HSR relevant to external aborts. A range of {I,D}FSC values (of which bit 5 is always set) and bit 9 'EA' which provides: > an IMPLEMENTATION DEFINED classification of External Aborts. This bit is in addition to the {I,D}FSC range, and has an implementation

Re: [PATCH kvm-unit-tests 0/3] Add physical timer test

On Tue, Jul 18, 2017 at 12:42:26PM +0200, Christoffer Dall wrote: > On Tue, Jul 18, 2017 at 12:17:07PM +0200, Andrew Jones wrote: > > On Thu, Jul 13, 2017 at 09:20:06PM +0200, Christoffer Dall wrote: > > > Add a test for the vtimer. I've tested on > > > accel=tcg > > > accel=kvm

Re: [PATCH kvm-unit-tests 2/3] arm64: timer: Fix test on APM X-Gene

On Tue, Jul 18, 2017 at 12:35:12PM +0200, Christoffer Dall wrote: > On Tue, Jul 18, 2017 at 12:05:03PM +0200, Andrew Jones wrote: > > On Fri, Jul 14, 2017 at 08:45:12AM -0700, Christoffer Dall wrote: > > > On Fri, Jul 14, 2017 at 09:04:10AM +0100, Marc Zyngier wrote: > > > > On 13/07/17 20:20,

Re: [PATCH kvm-unit-tests 3/3] arm64: timer: Add support for phys timer testing

On Thu, Jul 13, 2017 at 09:20:09PM +0200, Christoffer Dall wrote: > Rearrange the code to be able to reuse as much as posible and add > support for testing the physical timer as well. > > Also change the default unittests configuration to run a separate vtimer > and ptimer test so that older

Re: [report] boot a vm that with PCI only hierarchy devices and with GICv3 , it's failed.

On 2017/7/18 19:22, Robin Murphy wrote: > On 18/07/17 12:07, wanghaibin wrote: >> On 2017/7/18 18:02, Robin Murphy wrote: >> >>> On 18/07/17 10:15, Marc Zyngier wrote: On 18/07/17 05:07, wanghaibin wrote: > Hi, all: > > I met a problem, I just try to test PCI only hierarchy

Re: [report] boot a vm that with PCI only hierarchy devices and with GICv3 , it's failed.

On 18/07/17 12:07, wanghaibin wrote: > On 2017/7/18 18:02, Robin Murphy wrote: > >> On 18/07/17 10:15, Marc Zyngier wrote: >>> On 18/07/17 05:07, wanghaibin wrote: Hi, all: I met a problem, I just try to test PCI only hierarchy devices model (qemu/docs/pcie.txt sections 2.3)

Re: [report] boot a vm that with PCI only hierarchy devices and with GICv3 , it's failed.

On 2017/7/18 18:02, Robin Murphy wrote: > On 18/07/17 10:15, Marc Zyngier wrote: >> On 18/07/17 05:07, wanghaibin wrote: >>> Hi, all: >>> >>> I met a problem, I just try to test PCI only hierarchy devices model >>> (qemu/docs/pcie.txt sections 2.3) >>> >>> Here is part of qemu cmd: >>> -device

Re: [PATCH kvm-unit-tests 0/3] Add physical timer test

On Tue, Jul 18, 2017 at 12:17:07PM +0200, Andrew Jones wrote: > On Thu, Jul 13, 2017 at 09:20:06PM +0200, Christoffer Dall wrote: > > Add a test for the vtimer. I've tested on > > accel=tcg > > accel=kvm : on seattle, and mustang > > accel=kvm,kernel-irqchip=off : on

Re: [PATCH kvm-unit-tests 2/3] arm64: timer: Fix test on APM X-Gene

On Tue, Jul 18, 2017 at 12:05:03PM +0200, Andrew Jones wrote: > On Fri, Jul 14, 2017 at 08:45:12AM -0700, Christoffer Dall wrote: > > On Fri, Jul 14, 2017 at 09:04:10AM +0100, Marc Zyngier wrote: > > > On 13/07/17 20:20, Christoffer Dall wrote: > > > > When running the vtimer test on an APM

Re: [PATCH kvm-unit-tests 0/3] Add physical timer test

On Thu, Jul 13, 2017 at 09:20:06PM +0200, Christoffer Dall wrote: > Add a test for the vtimer. I've tested on > accel=tcg > accel=kvm : on seattle, and mustang > accel=kvm,kernel-irqchip=off : on mustang > > I first fix two issues I had running the basic timer test on APM

Re: [PATCH kvm-unit-tests 2/3] arm64: timer: Fix test on APM X-Gene

On Fri, Jul 14, 2017 at 08:45:12AM -0700, Christoffer Dall wrote: > On Fri, Jul 14, 2017 at 09:04:10AM +0100, Marc Zyngier wrote: > > On 13/07/17 20:20, Christoffer Dall wrote: > > > When running the vtimer test on an APM X-Gene, setting the timer value > > > to (2^64 - 1) apparently results in

Re: [report] boot a vm that with PCI only hierarchy devices and with GICv3 , it's failed.

On 18/07/17 10:15, Marc Zyngier wrote: > On 18/07/17 05:07, wanghaibin wrote: >> Hi, all: >> >> I met a problem, I just try to test PCI only hierarchy devices model >> (qemu/docs/pcie.txt sections 2.3) >> >> Here is part of qemu cmd: >> -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1