Re: [PATCH v8 00/20] KVM: ARM64: Add guest PMU support

[Marc Zyngier]

On 07/01/16 14:12, Will Deacon wrote:
> On Thu, Jan 07, 2016 at 02:10:38PM +, Marc Zyngier wrote:
>> On 22/12/15 08:07, Shannon Zhao wrote:
>>> From: Shannon Zhao 
>>>
>>> This patchset adds guest PMU support for KVM on ARM64. It takes
>>> trap-and-emulate approach. When guest wants to monitor one event, it
>>> will be trapped by KVM and KVM will call perf_event API to create a perf
>>> event and call relevant perf_event APIs to get the count value of event.
>>>
>>> Use perf to test this patchset in guest. When using "perf list", it
>>> shows the list of the hardware events and hardware cache events perf
>>> supports. Then use "perf stat -e EVENT" to monitor some event. For
>>> example, use "perf stat -e cycles" to count cpu cycles and
>>> "perf stat -e cache-misses" to count cache misses.
>>
>> I finally feel like we're pretty close to something we could merge. My
>> current concerns are:
> 
> The merge window opens on Monday and I'm not prepared to take further
> PMU changes at this point (unless they're bug fixes, of course).
> 
> This needs to wait until 4.6.

Fair enough. I guess I'll continue the review process and queue that
early in the next cycle.

Thanks,

M.
-- 
Jazz is not dead. It just smells funny...
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v8 00/20] KVM: ARM64: Add guest PMU support

[Shannon Zhao]

From: Shannon Zhao 

This patchset adds guest PMU support for KVM on ARM64. It takes
trap-and-emulate approach. When guest wants to monitor one event, it
will be trapped by KVM and KVM will call perf_event API to create a perf
event and call relevant perf_event APIs to get the count value of event.

Use perf to test this patchset in guest. When using "perf list", it
shows the list of the hardware events and hardware cache events perf
supports. Then use "perf stat -e EVENT" to monitor some event. For
example, use "perf stat -e cycles" to count cpu cycles and
"perf stat -e cache-misses" to count cache misses.

Below are the outputs of "perf stat -r 5 sleep 5" when running in host
and guest.

Host:
 Performance counter stats for 'sleep 5' (5 runs):

  0.549456  task-clock (msec) #0.000 CPUs utilized  
  ( +-  5.68% )
 1  context-switches  #0.002 M/sec
 0  cpu-migrations#0.000 K/sec
48  page-faults   #0.088 M/sec  
  ( +-  1.40% )
   1146243  cycles#2.086 GHz
  ( +-  5.71% )
 stalled-cycles-frontend
 stalled-cycles-backend
627195  instructions  #0.55  insns per cycle
  ( +- 15.65% )
 branches
  9826  branch-misses #   17.883 M/sec  
  ( +-  1.10% )

   5.000875516 seconds time elapsed 
 ( +-  0.00% )


Guest:
 Performance counter stats for 'sleep 5' (5 runs):

  0.640712  task-clock (msec) #0.000 CPUs utilized  
  ( +-  0.41% )
 1  context-switches  #0.002 M/sec
 0  cpu-migrations#0.000 K/sec
50  page-faults   #0.077 M/sec  
  ( +-  1.37% )
   1320428  cycles#2.061 GHz
  ( +-  0.29% )
 stalled-cycles-frontend
 stalled-cycles-backend
642373  instructions  #0.49  insns per cycle
  ( +-  0.46% )
 branches
 10399  branch-misses #   16.230 M/sec  
  ( +-  1.57% )

   5.001181020 seconds time elapsed 
 ( +-  0.00% )


Have a cycle counter read test like below in guest and host:

static void test(void)
{
unsigned long count, count1, count2;
count1 = read_cycles();
count++;
count2 = read_cycles();
}

Host:
count1: 3049567104
count2: 3049567247
delta: 143

Guest:
count1: 5281420890
count2: 5281421068
delta: 178

The gap between guest and host is very small. One reason for this I
think is that it doesn't count the cycles in EL2 and host since we add
exclude_hv = 1. So the cycles spent to store/restore registers which
happens at EL2 are not included.

This patchset can be fetched from [1] and the relevant QEMU version for
test can be fetched from [2].

The results of 'perf test' can be found from [3][4].
The results of perf_event_tests test suite can be found from [5][6].

Also, I have tested "perf top" in two VMs and host at the same time. It
works well.

Thanks,
Shannon

[1] https://git.linaro.org/people/shannon.zhao/linux-mainline.git  
KVM_ARM64_PMU_v8
[2] https://git.linaro.org/people/shannon.zhao/qemu.git  virtual_PMU
[3] http://people.linaro.org/~shannon.zhao/PMU/perf-test-host.txt
[4] http://people.linaro.org/~shannon.zhao/PMU/perf-test-guest.txt
[5] http://people.linaro.org/~shannon.zhao/PMU/perf_event_tests-host.txt
[6] http://people.linaro.org/~shannon.zhao/PMU/perf_event_tests-guest.txt

Changes since v7:
* Rebase on kvm-arm next
* Fix the handler of PMUSERENR and add a helper to forward trap to guest
  EL1
* Fix some small bugs found by Marc

Changes since v6:
* Rebase on v4.4-rc5
* Drop access_pmu_cp15_regs() so that it could use same handler for both
  arch64 and arch32. And it could drop the definitions of CP15 register
  offsets, also avoid same codes added twice
* Use vcpu_sys_reg() when accessing PMU registers to avoid endian things
* Add handler for PMUSERENR and some checkers for other registers
* Add kvm_arm_pmu_get_attr()

Changes since v5:
* Rebase on new linux kernel mainline
* Remove state duplications and drop PMOVSCLR, PMCNTENCLR, PMINTENCLR,
  PMXEVCNTR, PMXEVTYPER
* Add a helper to check if vPMU is already initialized
* remove kvm_vcpu from kvm_pmc

Changes since v4:
* Rebase on new linux kernel mainline 
* Drop the reset handler of CP15 registers
* Fix a compile failure on arch ARM due to lack of asm/pmu.h
* Refactor the interrupt injecting flow according to Marc's suggestion
* Check the value of PMSELR register
* Calculate the attr.disabled according to PMCR.E and PMCNTENSET/CLR
* Fix some coding style
* Document the vPMU irq