[Qemu-devel] [Bug 950692] Re: High CPU usage in Host (revisited)
Ok, thanks for your confirmation! ** Changed in: qemu Status: Incomplete => Fix Released -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/950692 Title: High CPU usage in Host (revisited) Status in QEMU: Fix Released Bug description: Hi, last time QEMU(KVM) was working for us flawlessly was 2.6.35 kernel. Actually it still works flawlessly on that one single machine, that still has this kernel. Qemu version is meanwhile 1.0-r3, so the problem seems to be dependent on kernel version and not qemu version. We have several other machines, where the "high CPU usage in host" problem is present in various degrees of annoyingness. Both host and guest are Gentoo linux, at least that's what we test with. Several tested systems with other linux distributions and FreeBSD show similar - if not worse - behaviour. I will talk about 3 hosts, machine A, machine B and machine C A: 2.6.35-gentoo-r9 #2 SMP Sat Nov 6 22:32:28 CET 2010 x86_64 Intel(R) Xeon(R) CPU L5410 @ 2.33GHz 32GB, runs about 15 KVM guests (all Gentoo, some 32bit, some 64bit, all SMP) no problems whatsoever, host CPU usage corresponds to Guest CPU usage + 1-2%, that's how we like it qemu 1.0-r3 B: 3.0.6-gentoo #1 SMP Sun Oct 16 18:57:31 CEST 2011 x86_64 Intel(R) Xeon(R) CPU L5630 @ 2.13GHz 144GB, runs 1(!) KVM guest (Debian 6.x) /usr/bin/qemu-system-x86_64 --enable-kvm -daemonize -cpu host -k de -net tap -tdf -hda /data/vm/disk.raw -m 768 -smp 1 -vnc :5 -net nic,model=e1000,macaddr=... 100% host CPU load always, therefore it got only "smp 1", if we gave it smp 2, it would have 200%, smp 4 400% and so on. qemu 1.0-r3 C: 3.1.6-gentoo #5 SMP Tue Mar 6 20:34:44 CET 2012 x86_64 Intel(R) Xeon(R) CPU 5148 @ 2.33GHz 16GB, runs 1-4 KVM guests (mostly Gentoo machines from A, plus some SuSE, RedHat etc.) X00% CPU usage, where x corresponds to the smp X parameter, at startup as well as if someone "touches" the VM, like logging in, doing a "ls". If the machine is ABSOLUTELY IDLE, the process also exhibits 1-2% CPU load in host, but as soon as you do a simple ls, usage goes to - say - 400%, where it remains for some seconds, then slowly falls 280%, 120%, 60%, ... back to 1-2% qemu 1.0-r3 B is no go, C tries to well-behave but ultimatively fails, A is golden. There seems to be REAL high CPU usage and not only an error in displaying it. Other processes get less CPU power and exhibit definitely a slower runtime. On B, definitely one CPU core is hogged all the time Some years ago we experienced something similar with ~2.6.26 and after a long and woeful period, we found out that compiling the host kernel as a tickless system caused the problem. Enabling high resolution timers made the problem go away and that is the situation on machine A until today. Since then no one dared to touch this production server. Unfortunately, this recipe didn't help with the other machines. I have scanned the net for similar problems and there are people complaining about high CPU usage. Unfortunately very often the devs or maintainers cannot reproduce it and the issue is dropped. Well - we cannot reproduce a "good behaviour"(tm) on any but one machine with any recent (read: post-2.6.35) linux kernel. Summary what we tried so far: * different linux kernels @ host, and @ guest -> no difference, especially there are guests @ A, that run newer kernels and there are Guests at B and C that run older kernels than is the host kernel * smp and non-smp, 32bit and 64bit guests -> 32/64bit in the guest makes no difference whatsoever. The smp just limits how much of the host CPU the guest hogs on non-well behaving systems (smp X -> X * 100%) * various linux guest OS, as well as FreeBSD -> no difference whatsoever * various options parameters in the host kernel (other schedulers, HRT, tickless,...) -> no difference whatsoever * various versions of qemu/kvm since 0.13 -> no difference whatsoever * various qemu/kvm options, virtio and non-virtio configurations (most of the VMs @ A run blk-virtio but emulate an e1000) -> no difference whatsoever You could say, we've reached wits' end. We could try 2.6.35 @ machine C with the same configuration from A (they are identical except CPU and RAM size, but same RAID, mainboard, etc. plus A once had also the 5148 Xeons and an upgrade luckily made no difference in good behaviour, so I would exclude the CPU factor) but honestly that is not the way I'd like to go. The goal is to update A to something recent and not to loose it's VM-hosting well behaviour. Ideally to propagate this well beaviour to the other machines. Arjan Minski PetaMem IT To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/950692/+subscriptions
[Qemu-devel] [Bug 950692] Re: High CPU usage in Host (revisited)
>From our point of view, this ticket can be closed. KVM is running without issues on all our servers for more than 5 years now. The problem described above, was due to a weird combination of "timer" kernel parameters in the early 3.x kernels. IIRC, enabling a high- frequency timer and/or "tickless system" solved the issues we had. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/950692 Title: High CPU usage in Host (revisited) Status in QEMU: Incomplete Bug description: Hi, last time QEMU(KVM) was working for us flawlessly was 2.6.35 kernel. Actually it still works flawlessly on that one single machine, that still has this kernel. Qemu version is meanwhile 1.0-r3, so the problem seems to be dependent on kernel version and not qemu version. We have several other machines, where the "high CPU usage in host" problem is present in various degrees of annoyingness. Both host and guest are Gentoo linux, at least that's what we test with. Several tested systems with other linux distributions and FreeBSD show similar - if not worse - behaviour. I will talk about 3 hosts, machine A, machine B and machine C A: 2.6.35-gentoo-r9 #2 SMP Sat Nov 6 22:32:28 CET 2010 x86_64 Intel(R) Xeon(R) CPU L5410 @ 2.33GHz 32GB, runs about 15 KVM guests (all Gentoo, some 32bit, some 64bit, all SMP) no problems whatsoever, host CPU usage corresponds to Guest CPU usage + 1-2%, that's how we like it qemu 1.0-r3 B: 3.0.6-gentoo #1 SMP Sun Oct 16 18:57:31 CEST 2011 x86_64 Intel(R) Xeon(R) CPU L5630 @ 2.13GHz 144GB, runs 1(!) KVM guest (Debian 6.x) /usr/bin/qemu-system-x86_64 --enable-kvm -daemonize -cpu host -k de -net tap -tdf -hda /data/vm/disk.raw -m 768 -smp 1 -vnc :5 -net nic,model=e1000,macaddr=... 100% host CPU load always, therefore it got only "smp 1", if we gave it smp 2, it would have 200%, smp 4 400% and so on. qemu 1.0-r3 C: 3.1.6-gentoo #5 SMP Tue Mar 6 20:34:44 CET 2012 x86_64 Intel(R) Xeon(R) CPU 5148 @ 2.33GHz 16GB, runs 1-4 KVM guests (mostly Gentoo machines from A, plus some SuSE, RedHat etc.) X00% CPU usage, where x corresponds to the smp X parameter, at startup as well as if someone "touches" the VM, like logging in, doing a "ls". If the machine is ABSOLUTELY IDLE, the process also exhibits 1-2% CPU load in host, but as soon as you do a simple ls, usage goes to - say - 400%, where it remains for some seconds, then slowly falls 280%, 120%, 60%, ... back to 1-2% qemu 1.0-r3 B is no go, C tries to well-behave but ultimatively fails, A is golden. There seems to be REAL high CPU usage and not only an error in displaying it. Other processes get less CPU power and exhibit definitely a slower runtime. On B, definitely one CPU core is hogged all the time Some years ago we experienced something similar with ~2.6.26 and after a long and woeful period, we found out that compiling the host kernel as a tickless system caused the problem. Enabling high resolution timers made the problem go away and that is the situation on machine A until today. Since then no one dared to touch this production server. Unfortunately, this recipe didn't help with the other machines. I have scanned the net for similar problems and there are people complaining about high CPU usage. Unfortunately very often the devs or maintainers cannot reproduce it and the issue is dropped. Well - we cannot reproduce a "good behaviour"(tm) on any but one machine with any recent (read: post-2.6.35) linux kernel. Summary what we tried so far: * different linux kernels @ host, and @ guest -> no difference, especially there are guests @ A, that run newer kernels and there are Guests at B and C that run older kernels than is the host kernel * smp and non-smp, 32bit and 64bit guests -> 32/64bit in the guest makes no difference whatsoever. The smp just limits how much of the host CPU the guest hogs on non-well behaving systems (smp X -> X * 100%) * various linux guest OS, as well as FreeBSD -> no difference whatsoever * various options parameters in the host kernel (other schedulers, HRT, tickless,...) -> no difference whatsoever * various versions of qemu/kvm since 0.13 -> no difference whatsoever * various qemu/kvm options, virtio and non-virtio configurations (most of the VMs @ A run blk-virtio but emulate an e1000) -> no difference whatsoever You could say, we've reached wits' end. We could try 2.6.35 @ machine C with the same configuration from A (they are identical except CPU and RAM size, but same RAID, mainboard, etc. plus A once had also the 5148 Xeons and an upgrade luckily made no difference in good behaviour, so I would exclude the CPU factor) but honestly that is not the way I'd like to go. The goal is to update A to something recent and not to loose it's VM-hosting well behaviour. Ideally
[Qemu-devel] [Bug 950692] Re: High CPU usage in Host (revisited)
Triaging old bug tickets ... can you somehow still reproduce this problem with the latest version of QEMU (currently v2.9), or could we close this ticket nowadays? ** Changed in: qemu Status: New => Incomplete -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/950692 Title: High CPU usage in Host (revisited) Status in QEMU: Incomplete Bug description: Hi, last time QEMU(KVM) was working for us flawlessly was 2.6.35 kernel. Actually it still works flawlessly on that one single machine, that still has this kernel. Qemu version is meanwhile 1.0-r3, so the problem seems to be dependent on kernel version and not qemu version. We have several other machines, where the "high CPU usage in host" problem is present in various degrees of annoyingness. Both host and guest are Gentoo linux, at least that's what we test with. Several tested systems with other linux distributions and FreeBSD show similar - if not worse - behaviour. I will talk about 3 hosts, machine A, machine B and machine C A: 2.6.35-gentoo-r9 #2 SMP Sat Nov 6 22:32:28 CET 2010 x86_64 Intel(R) Xeon(R) CPU L5410 @ 2.33GHz 32GB, runs about 15 KVM guests (all Gentoo, some 32bit, some 64bit, all SMP) no problems whatsoever, host CPU usage corresponds to Guest CPU usage + 1-2%, that's how we like it qemu 1.0-r3 B: 3.0.6-gentoo #1 SMP Sun Oct 16 18:57:31 CEST 2011 x86_64 Intel(R) Xeon(R) CPU L5630 @ 2.13GHz 144GB, runs 1(!) KVM guest (Debian 6.x) /usr/bin/qemu-system-x86_64 --enable-kvm -daemonize -cpu host -k de -net tap -tdf -hda /data/vm/disk.raw -m 768 -smp 1 -vnc :5 -net nic,model=e1000,macaddr=... 100% host CPU load always, therefore it got only "smp 1", if we gave it smp 2, it would have 200%, smp 4 400% and so on. qemu 1.0-r3 C: 3.1.6-gentoo #5 SMP Tue Mar 6 20:34:44 CET 2012 x86_64 Intel(R) Xeon(R) CPU 5148 @ 2.33GHz 16GB, runs 1-4 KVM guests (mostly Gentoo machines from A, plus some SuSE, RedHat etc.) X00% CPU usage, where x corresponds to the smp X parameter, at startup as well as if someone "touches" the VM, like logging in, doing a "ls". If the machine is ABSOLUTELY IDLE, the process also exhibits 1-2% CPU load in host, but as soon as you do a simple ls, usage goes to - say - 400%, where it remains for some seconds, then slowly falls 280%, 120%, 60%, ... back to 1-2% qemu 1.0-r3 B is no go, C tries to well-behave but ultimatively fails, A is golden. There seems to be REAL high CPU usage and not only an error in displaying it. Other processes get less CPU power and exhibit definitely a slower runtime. On B, definitely one CPU core is hogged all the time Some years ago we experienced something similar with ~2.6.26 and after a long and woeful period, we found out that compiling the host kernel as a tickless system caused the problem. Enabling high resolution timers made the problem go away and that is the situation on machine A until today. Since then no one dared to touch this production server. Unfortunately, this recipe didn't help with the other machines. I have scanned the net for similar problems and there are people complaining about high CPU usage. Unfortunately very often the devs or maintainers cannot reproduce it and the issue is dropped. Well - we cannot reproduce a "good behaviour"(tm) on any but one machine with any recent (read: post-2.6.35) linux kernel. Summary what we tried so far: * different linux kernels @ host, and @ guest -> no difference, especially there are guests @ A, that run newer kernels and there are Guests at B and C that run older kernels than is the host kernel * smp and non-smp, 32bit and 64bit guests -> 32/64bit in the guest makes no difference whatsoever. The smp just limits how much of the host CPU the guest hogs on non-well behaving systems (smp X -> X * 100%) * various linux guest OS, as well as FreeBSD -> no difference whatsoever * various options parameters in the host kernel (other schedulers, HRT, tickless,...) -> no difference whatsoever * various versions of qemu/kvm since 0.13 -> no difference whatsoever * various qemu/kvm options, virtio and non-virtio configurations (most of the VMs @ A run blk-virtio but emulate an e1000) -> no difference whatsoever You could say, we've reached wits' end. We could try 2.6.35 @ machine C with the same configuration from A (they are identical except CPU and RAM size, but same RAID, mainboard, etc. plus A once had also the 5148 Xeons and an upgrade luckily made no difference in good behaviour, so I would exclude the CPU factor) but honestly that is not the way I'd like to go. The goal is to update A to something recent and not to loose it's VM-hosting well behaviour. Ideally to propagate this well beaviour to the other machines. Arjan Minski PetaMem IT To manage notifications ab
[Qemu-devel] [Bug 950692] Re: High CPU usage in Host (revisited)
I see similar problem when few I/Os are pumped and the VM goes non-responsive. The host sees nearly 100% CPU utilization. top - 08:58:57 up 18:42, 2 users, load average: 0.99, 0.98, 0.95 Tasks: 355 total, 1 running, 354 sleeping, 0 stopped, 0 zombie %Cpu(s): 1.5 us, 2.7 sy, 0.0 ni, 95.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 65937388 total, 11895920 used, 54041468 free, 8163244 buffers KiB Swap: 67073532 total,0 used, 67073532 free. 545132 cached Mem PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 2317 libvirt+ 20 0 18.612g 2.556g 8972 S 98.8 4.1 1120:00 qemu-system-x86 276 root 25 5 0 0 0 S 0.7 0.0 8:21.94 ksmd 312 root 20 0 0 0 0 S 0.3 0.0 0:02.63 kworker/5:1 315 root 20 0 0 0 0 S 0.3 0.0 0:00.21 kworker/20:1 Please let me know if this is fixed. I am currently using QEMU 2.0 -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/950692 Title: High CPU usage in Host (revisited) Status in QEMU: New Bug description: Hi, last time QEMU(KVM) was working for us flawlessly was 2.6.35 kernel. Actually it still works flawlessly on that one single machine, that still has this kernel. Qemu version is meanwhile 1.0-r3, so the problem seems to be dependent on kernel version and not qemu version. We have several other machines, where the "high CPU usage in host" problem is present in various degrees of annoyingness. Both host and guest are Gentoo linux, at least that's what we test with. Several tested systems with other linux distributions and FreeBSD show similar - if not worse - behaviour. I will talk about 3 hosts, machine A, machine B and machine C A: 2.6.35-gentoo-r9 #2 SMP Sat Nov 6 22:32:28 CET 2010 x86_64 Intel(R) Xeon(R) CPU L5410 @ 2.33GHz 32GB, runs about 15 KVM guests (all Gentoo, some 32bit, some 64bit, all SMP) no problems whatsoever, host CPU usage corresponds to Guest CPU usage + 1-2%, that's how we like it qemu 1.0-r3 B: 3.0.6-gentoo #1 SMP Sun Oct 16 18:57:31 CEST 2011 x86_64 Intel(R) Xeon(R) CPU L5630 @ 2.13GHz 144GB, runs 1(!) KVM guest (Debian 6.x) /usr/bin/qemu-system-x86_64 --enable-kvm -daemonize -cpu host -k de -net tap -tdf -hda /data/vm/disk.raw -m 768 -smp 1 -vnc :5 -net nic,model=e1000,macaddr=... 100% host CPU load always, therefore it got only "smp 1", if we gave it smp 2, it would have 200%, smp 4 400% and so on. qemu 1.0-r3 C: 3.1.6-gentoo #5 SMP Tue Mar 6 20:34:44 CET 2012 x86_64 Intel(R) Xeon(R) CPU 5148 @ 2.33GHz 16GB, runs 1-4 KVM guests (mostly Gentoo machines from A, plus some SuSE, RedHat etc.) X00% CPU usage, where x corresponds to the smp X parameter, at startup as well as if someone "touches" the VM, like logging in, doing a "ls". If the machine is ABSOLUTELY IDLE, the process also exhibits 1-2% CPU load in host, but as soon as you do a simple ls, usage goes to - say - 400%, where it remains for some seconds, then slowly falls 280%, 120%, 60%, ... back to 1-2% qemu 1.0-r3 B is no go, C tries to well-behave but ultimatively fails, A is golden. There seems to be REAL high CPU usage and not only an error in displaying it. Other processes get less CPU power and exhibit definitely a slower runtime. On B, definitely one CPU core is hogged all the time Some years ago we experienced something similar with ~2.6.26 and after a long and woeful period, we found out that compiling the host kernel as a tickless system caused the problem. Enabling high resolution timers made the problem go away and that is the situation on machine A until today. Since then no one dared to touch this production server. Unfortunately, this recipe didn't help with the other machines. I have scanned the net for similar problems and there are people complaining about high CPU usage. Unfortunately very often the devs or maintainers cannot reproduce it and the issue is dropped. Well - we cannot reproduce a "good behaviour"(tm) on any but one machine with any recent (read: post-2.6.35) linux kernel. Summary what we tried so far: * different linux kernels @ host, and @ guest -> no difference, especially there are guests @ A, that run newer kernels and there are Guests at B and C that run older kernels than is the host kernel * smp and non-smp, 32bit and 64bit guests -> 32/64bit in the guest makes no difference whatsoever. The smp just limits how much of the host CPU the guest hogs on non-well behaving systems (smp X -> X * 100%) * various linux guest OS, as well as FreeBSD -> no difference whatsoever * various options parameters in the host kernel (other schedulers, HRT, tickless,...) -> no difference whatsoever * various versions of qemu/kvm since 0.13 -> no difference whatsoever * various qemu/kvm opti
Re: [Qemu-devel] [Bug 950692] Re: High CPU usage in Host (revisited)
On Sun, Mar 11, 2012 at 05:30, PetaMem wrote: > *Newsflash* > > We do have a "well-behaving" KVM Host with 3.2.9 kernel on machine C Note: I am not Qemu developer :) OK, I read your bug report many times. I think you need deeper profiling here. Perhaps "perf top" is the best bet. Just make sure the kernel has debug symbols included so "perf" has as little difficulties as possible to interpret addresses into symbol names. Once you found the culprit, it could be easier to fix it. NB: -smp triggered it? hm bad locking somewhere perhaps? Anyway, i am not sure Qemu/KVM could really flawlessly implement SMP. So maybe it points to hidden bug in the vCPU parallel execution code somewhere. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
[Qemu-devel] [Bug 950692] Re: High CPU usage in Host (revisited)
*Newsflash* We do have a "well-behaving" KVM Host with 3.2.9 kernel on machine C After again numerous attempts to find the culprit, I decided to copy the kernel 2.6.35 and modules from machine A to machine C, where it exhibited also the desired "well-behaving". I then simply copied its config to a 3.2.9 kernel and did "make oldconfig", kept all defaults offered and restarted the machine with that newly created 3.2.9 and it seems it got soem right genes from 2.6.35 config. I will now poke the config and see if something breaks. Currently the only significant difference to our unsuccessfull 3.2.9 kernel is the fact, that the bad kernel was configured with kvm and kvm_intel not as module but compiled in. Should that be the culprit... oh man... I will test that and report. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/950692 Title: High CPU usage in Host (revisited) Status in QEMU: New Bug description: Hi, last time QEMU(KVM) was working for us flawlessly was 2.6.35 kernel. Actually it still works flawlessly on that one single machine, that still has this kernel. Qemu version is meanwhile 1.0-r3, so the problem seems to be dependent on kernel version and not qemu version. We have several other machines, where the "high CPU usage in host" problem is present in various degrees of annoyingness. Both host and guest are Gentoo linux, at least that's what we test with. Several tested systems with other linux distributions and FreeBSD show similar - if not worse - behaviour. I will talk about 3 hosts, machine A, machine B and machine C A: 2.6.35-gentoo-r9 #2 SMP Sat Nov 6 22:32:28 CET 2010 x86_64 Intel(R) Xeon(R) CPU L5410 @ 2.33GHz 32GB, runs about 15 KVM guests (all Gentoo, some 32bit, some 64bit, all SMP) no problems whatsoever, host CPU usage corresponds to Guest CPU usage + 1-2%, that's how we like it qemu 1.0-r3 B: 3.0.6-gentoo #1 SMP Sun Oct 16 18:57:31 CEST 2011 x86_64 Intel(R) Xeon(R) CPU L5630 @ 2.13GHz 144GB, runs 1(!) KVM guest (Debian 6.x) /usr/bin/qemu-system-x86_64 --enable-kvm -daemonize -cpu host -k de -net tap -tdf -hda /data/vm/disk.raw -m 768 -smp 1 -vnc :5 -net nic,model=e1000,macaddr=... 100% host CPU load always, therefore it got only "smp 1", if we gave it smp 2, it would have 200%, smp 4 400% and so on. qemu 1.0-r3 C: 3.1.6-gentoo #5 SMP Tue Mar 6 20:34:44 CET 2012 x86_64 Intel(R) Xeon(R) CPU 5148 @ 2.33GHz 16GB, runs 1-4 KVM guests (mostly Gentoo machines from A, plus some SuSE, RedHat etc.) X00% CPU usage, where x corresponds to the smp X parameter, at startup as well as if someone "touches" the VM, like logging in, doing a "ls". If the machine is ABSOLUTELY IDLE, the process also exhibits 1-2% CPU load in host, but as soon as you do a simple ls, usage goes to - say - 400%, where it remains for some seconds, then slowly falls 280%, 120%, 60%, ... back to 1-2% qemu 1.0-r3 B is no go, C tries to well-behave but ultimatively fails, A is golden. There seems to be REAL high CPU usage and not only an error in displaying it. Other processes get less CPU power and exhibit definitely a slower runtime. On B, definitely one CPU core is hogged all the time Some years ago we experienced something similar with ~2.6.26 and after a long and woeful period, we found out that compiling the host kernel as a tickless system caused the problem. Enabling high resolution timers made the problem go away and that is the situation on machine A until today. Since then no one dared to touch this production server. Unfortunately, this recipe didn't help with the other machines. I have scanned the net for similar problems and there are people complaining about high CPU usage. Unfortunately very often the devs or maintainers cannot reproduce it and the issue is dropped. Well - we cannot reproduce a "good behaviour"(tm) on any but one machine with any recent (read: post-2.6.35) linux kernel. Summary what we tried so far: * different linux kernels @ host, and @ guest -> no difference, especially there are guests @ A, that run newer kernels and there are Guests at B and C that run older kernels than is the host kernel * smp and non-smp, 32bit and 64bit guests -> 32/64bit in the guest makes no difference whatsoever. The smp just limits how much of the host CPU the guest hogs on non-well behaving systems (smp X -> X * 100%) * various linux guest OS, as well as FreeBSD -> no difference whatsoever * various options parameters in the host kernel (other schedulers, HRT, tickless,...) -> no difference whatsoever * various versions of qemu/kvm since 0.13 -> no difference whatsoever * various qemu/kvm options, virtio and non-virtio configurations (most of the VMs @ A run blk-virtio but emulate an e1000) -> no difference whatsoever You could say, we've reache
