Hi Everyone,
I' currently testing KVM arm realtime performance on a hikey960 board. My test
benchmark is cyclictest to measure thread wake up latency both on Host linux OS
and KVM Guest linux OS.
Host OS:
hikey960:/mnt/debian/usr/src/linux# cyclictest -p 99 -t 4 -m -n -a 0-3 -l
100000
# /dev/cpu_dma_latency set to 0us
WARN: Running on unknown kernel version...YMMV
policy: fifo: loadavg: 0.00 0.00 0.00 1/165 3270
T: 0 ( 3266) P:99 I:1000 C: 100000 Min: 4 Act: 15 Avg: 15 Max: 139
T: 1 ( 3267) P:99 I:1500 C: 66736 Min: 4 Act: 15 Avg: 15 Max: 239
T: 2 ( 3268) P:99 I:2000 C: 50051 Min: 4 Act: 19 Avg: 15 Max: 43
T: 3 ( 3269) P:99 I:2500 C: 40039 Min: 5 Act: 15 Avg: 16 Max: 74
Guest OS:
root@genericarmv8:~# cyclictest -p 99 -t 4 -m -n -a 0-3 -l 100000
# /dev/cpu_dma_latency set to 0us
WARN: Running on unknown kernel version...YMMV
policy: fifo: loadavg: 0.13 0.05 0.01 1/70 293
T: 0 ( 290) P:99 I:1000 C: 100000 Min: 7 Act: 44 Avg: 85 Max: 16111
T: 1 ( 291) P:99 I:1500 C: 66665 Min: 7 Act: 81 Avg: 90 Max: 15306
T: 2 ( 292) P:99 I:2000 C: 49995 Min: 7 Act: 88 Avg: 87 Max: 16703
T: 3 ( 293) P:99 I:2500 C: 39992 Min: 8 Act: 72 Avg: 97 Max: 14976
RT performance on KVM guest OS is poor compared to that on host OS. The average
wake up latency is about 6 - 7 times on Guest OS vs on Host OS.
I've tried some configurations to improve RT in KVM, like:
1 Can be combined with CPU isolation
2 Host OS and Guest OS use RT preempt kernel
3 Host CPU avoid frequency change
4 Configure NO_HZ_FULL for Guest OS
There could be a little improvement after apply above configuration, but the RT
performance is still very poor.
5 Guest OS use idle poll instead of WFI to avoid trap and switch out
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 2dc0f84..53aef78 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -83,7 +83,7 @@ void arch_cpu_idle(void)
* tricks
*/
trace_cpu_idle_rcuidle(1, smp_processor_id());
- cpu_do_idle();
+ cpu_relax();
local_irq_enable();
trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, smp_processor_id());
}
root@genericarmv8:~# cyclictest -p 99 -t 4 -m -n -l 100000
# /dev/cpu_dma_latency set to 0us
WARN: Running on unknown kernel version...YMMV
policy: fifo: loadavg: 0.07 0.03 0.00 1/99 328
T: 0 ( 325) P:99 I:1000 C: 100000 Min: 3 Act: 6 Avg: 13 Max: 4999
T: 1 ( 326) P:99 I:1500 C: 66659 Min: 5 Act: 7 Avg: 14 Max: 3449
T: 2 ( 327) P:99 I:2000 C: 49989 Min: 4 Act: 7 Avg: 9 Max: 11471
T: 3 ( 328) P:99 I:2500 C: 39986 Min: 4 Act: 14 Avg: 14 Max: 11253
The method 5 can improve Guest OS RT performance a lot, the average thread wake
up latency on Guest OS is almost same as its on Host OS, but the Max wake up
latency is still very poor.
Anyone has any idea to improve RT performance on KVM Guest OS? Although method
5 can improve RT performance on Guest OS a lot, I think it is not good idea.
Thanks,
Steven
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
