Re: [RFC v2 00/10] Provide the EL1 physical timer to the VM

[Jintack Lim]

Hi Marc,

On Sun, Jan 29, 2017 at 10:55 AM, Marc Zyngier  wrote:
> Hi Jintack,
>
> On Fri, Jan 27 2017 at 01:04:50 AM, Jintack Lim  
> wrote:
>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> and it is reasonable for an OS to expect to be able to access both.
>> However, the current KVM implementation does not provide the EL1 physical
>> timer to VMs but terminates VMs on access to the timer.
>>
>> This patch series enables VMs to use the EL1 physical timer through
>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> access and sets up the background timer accordingly.  When the background
>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> physical timer without trapping.  However, this requires somehow using the
>> EL2 physical timer for the Linux host while running the VM instead of the
>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> this was straightforward to do, and I leave it to future work to determine
>> if transferring the EL1 physical timer state to the EL2 timer provides any
>> performance benefit.
>>
>> This feature will be useful for any OS that wishes to access the EL1
>> physical timer. Nested virtualization is one of those use cases. A nested
>> hypervisor running inside a VM would think it has full access to the
>> hardware and naturally tries to use the EL1 physical timer as Linux would
>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> would do, but supporting the EL2 physical timer to the VM is out of scope
>> of this patch series. This patch series will make it easy to add the EL2
>> timer support in the future, though.
>>
>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> and will continue to use only the virtual timer and this patch series will
>> therefore not introduce any performance degredation as a result of
>> trap-and-emulate.
>
> Thanks for respining this series. Overall, this looks quite good, and
> the couple of comments I have should be easy to address.

Thanks for the review!

>
> My main concern is that we do expose a timer that doesn't hide
> CNTVOFF. I appreciate that that was already the case, since CNTPCT was
> always available (and this avoided trapping the counter), but maybe we
> should have a way for userspace to ask for a mode where CNTPCT=CNTVCT,
> byt trapping the physical counter and taking CNTVOFF in all physical
> timer calculations.

As discussed in the other thread, I think we can expose CNTVOFF to the
guest OS. I have a patch that lets the guest hypervisor observe CNTVCT
= CNTPCT - offset (virtual CNTVOFF_EL2) and I will include it in the
next nesting patch series.

Thanks,
Jintack

>
> I'm pretty sure you've addressed this one way or another in your nested
> virt series, so maybe extracting the relevant patches and adding them on
> top of this series could be an option?
>
> Thanks,
>
> M.
> --
> Jazz is not dead. It just smells funny.
>

___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: [RFC v2 00/10] Provide the EL1 physical timer to the VM

[Marc Zyngier]

Hi Jintack,

On Fri, Jan 27 2017 at 01:04:50 AM, Jintack Lim  wrote:
> The ARM architecture defines the EL1 physical timer and the virtual timer,
> and it is reasonable for an OS to expect to be able to access both.
> However, the current KVM implementation does not provide the EL1 physical
> timer to VMs but terminates VMs on access to the timer.
>
> This patch series enables VMs to use the EL1 physical timer through
> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> access and sets up the background timer accordingly.  When the background
> timer expires, the KVM host injects EL1 physical timer interrupts to the
> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> physical timer without trapping.  However, this requires somehow using the
> EL2 physical timer for the Linux host while running the VM instead of the
> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> this was straightforward to do, and I leave it to future work to determine
> if transferring the EL1 physical timer state to the EL2 timer provides any
> performance benefit.
>
> This feature will be useful for any OS that wishes to access the EL1
> physical timer. Nested virtualization is one of those use cases. A nested
> hypervisor running inside a VM would think it has full access to the
> hardware and naturally tries to use the EL1 physical timer as Linux would
> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> would do, but supporting the EL2 physical timer to the VM is out of scope
> of this patch series. This patch series will make it easy to add the EL2
> timer support in the future, though.
>
> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> and will continue to use only the virtual timer and this patch series will
> therefore not introduce any performance degredation as a result of
> trap-and-emulate.

Thanks for respining this series. Overall, this looks quite good, and
the couple of comments I have should be easy to address.

My main concern is that we do expose a timer that doesn't hide
CNTVOFF. I appreciate that that was already the case, since CNTPCT was
always available (and this avoided trapping the counter), but maybe we
should have a way for userspace to ask for a mode where CNTPCT=CNTVCT,
byt trapping the physical counter and taking CNTVOFF in all physical
timer calculations.

I'm pretty sure you've addressed this one way or another in your nested
virt series, so maybe extracting the relevant patches and adding them on
top of this series could be an option?

Thanks,

M.
-- 
Jazz is not dead. It just smells funny.
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

[RFC v2 00/10] Provide the EL1 physical timer to the VM

[Jintack Lim]

The ARM architecture defines the EL1 physical timer and the virtual timer,
and it is reasonable for an OS to expect to be able to access both.
However, the current KVM implementation does not provide the EL1 physical
timer to VMs but terminates VMs on access to the timer.

This patch series enables VMs to use the EL1 physical timer through
trap-and-emulate.  The KVM host emulates each EL1 physical timer register
access and sets up the background timer accordingly.  When the background
timer expires, the KVM host injects EL1 physical timer interrupts to the
VM.  Alternatively, it's also possible to allow VMs to access the EL1
physical timer without trapping.  However, this requires somehow using the
EL2 physical timer for the Linux host while running the VM instead of the
EL1 physical timer.  Right now I just implemented trap-and-emulate because
this was straightforward to do, and I leave it to future work to determine
if transferring the EL1 physical timer state to the EL2 timer provides any
performance benefit.

This feature will be useful for any OS that wishes to access the EL1
physical timer. Nested virtualization is one of those use cases. A nested
hypervisor running inside a VM would think it has full access to the
hardware and naturally tries to use the EL1 physical timer as Linux would
do. Other nested hypervisors may try to use the EL2 physical timer as Xen
would do, but supporting the EL2 physical timer to the VM is out of scope
of this patch series. This patch series will make it easy to add the EL2
timer support in the future, though.

Note that Linux VMs booting in EL1 will be unaffected by this patch series
and will continue to use only the virtual timer and this patch series will
therefore not introduce any performance degredation as a result of
trap-and-emulate.

v1 => v2:
 - Rebase on kvm-arm-for-4.10-rc4
 - To make it simple, schedule the background timer for the EL1 physical timer
   emulation on every entry to the VM and cancel it on exit.
 - Change timer_context structure to have cntvoff and restore enable field back
   to arch_timer_cpu structure

Jintack Lim (10):
  KVM: arm/arm64: Abstract virtual timer context into separate structure
  KVM: arm/arm64: Move cntvoff to each timer context
  KVM: arm/arm64: Decouple kvm timer functions from virtual timer
  KVM: arm/arm64: Add the EL1 physical timer context
  KVM: arm/arm64: Initialize the emulated EL1 physical timer
  KVM: arm/arm64: Update the physical timer interrupt level
  KVM: arm/arm64: Set a background timer to the earliest timer
expiration
  KVM: arm/arm64: Set up a background timer for the physical timer
emulation
  KVM: arm64: Add the EL1 physical timer access handler
  KVM: arm/arm64: Emulate the EL1 phys timer register access

 arch/arm/include/asm/kvm_host.h   |   6 +-
 arch/arm/kvm/arm.c|   3 +-
 arch/arm/kvm/reset.c  |   9 +-
 arch/arm64/include/asm/kvm_host.h |   4 +-
 arch/arm64/kvm/reset.c|   9 +-
 arch/arm64/kvm/sys_regs.c |  60 +++
 include/kvm/arm_arch_timer.h  |  39 
 virt/kvm/arm/arch_timer.c | 204 --
 virt/kvm/arm/hyp/timer-sr.c   |  13 +--
 9 files changed, 261 insertions(+), 86 deletions(-)

-- 
1.9.1


___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm