Re: [Qemu-devel] Disabling KVM on the fly
Il 17/10/2012 20:37, Jan Kiszka ha scritto: On 2012-10-17 18:44, Paolo Bonzini wrote: Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using migrate exec:catfoo.save with a KVM machine and -incoming 'exec:cat foo.save' with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), Unnecessary. Both models (KVM in-kernel and QEMU userspace) are compatible - in the absence of bugs. He wants to really switch it on the fly---not just migrate out and in---and for that you need to disable the KVM-specific devices. But loading a KVM image into TCG lets non-trival guests lock up. Likely due to differences in the CPU virtualization/emulation (MSRs...). Perhaps that can be mitigated by using an older machine model. Start with something simple like a pentium2 and work up from there... Paolo
Re: [Qemu-devel] Disabling KVM on the fly
On 2012-10-18 08:29, Paolo Bonzini wrote: Il 17/10/2012 20:37, Jan Kiszka ha scritto: On 2012-10-17 18:44, Paolo Bonzini wrote: Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using migrate exec:catfoo.save with a KVM machine and -incoming 'exec:cat foo.save' with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), Unnecessary. Both models (KVM in-kernel and QEMU userspace) are compatible - in the absence of bugs. He wants to really switch it on the fly---not just migrate out and in---and for that you need to disable the KVM-specific devices. Well, that's even more unrealistic than via migration. But loading a KVM image into TCG lets non-trival guests lock up. Likely due to differences in the CPU virtualization/emulation (MSRs...). Perhaps that can be mitigated by using an older machine model. Start with something simple like a pentium2 and work up from there... Even if, there are still too many untranslated, maybe even untranslatable states of the KVM CPU model, at least. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux
Re: [Qemu-devel] Disabling KVM on the fly
On Thu, Oct 18, 2012 at 10:24 AM, Jan Kiszka jan.kis...@siemens.com wrote: On 2012-10-18 08:29, Paolo Bonzini wrote: Il 17/10/2012 20:37, Jan Kiszka ha scritto: On 2012-10-17 18:44, Paolo Bonzini wrote: Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using migrate exec:catfoo.save with a KVM machine and -incoming 'exec:cat foo.save' with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), Unnecessary. Both models (KVM in-kernel and QEMU userspace) are compatible - in the absence of bugs. He wants to really switch it on the fly---not just migrate out and in---and for that you need to disable the KVM-specific devices. Well, that's even more unrealistic than via migration. I am just looking for a way to do it, regardless whether it causes a big performance penalty due to migration, etc. Even if this means preparing a special environment (emulating specific hardware, etc), this is fine with me. Thanks a lot for all the feedback, now I know what I will have to look into! -Clemens
[Qemu-devel] Disabling KVM on the fly
Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. I know nothing of that sort is implemented at this point. Leaving aside the issue of certain CPUID instructions suddenly returning different results, are there obvious problems even trying to go down that road? Would love to hear some thoughts on this - don't hesitate to tell me that's stupid and impossible because XYZ (assuming you also fill in the blanks ;) ) Thanks! Clemens -- Clemens Kolbitsch Security Researcher kolbit...@lastline.com +1 (206) 356-7745 Lastline, Inc. 6950 Hollister Avenue, Suite 101 Goleta, CA 93117 www.lastline.com
Re: [Qemu-devel] Disabling KVM on the fly
Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using migrate exec:catfoo.save with a KVM machine and -incoming 'exec:cat foo.save' with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), the differences between KVM and TCG are just that KVM doesn't initialize some TCG-only data structure, and that KVM uses many CPU threads; TCG uses one that goes through CPUs round-robin. The CPU threads of course execute different code. So no, in theory there is nothing that prevents this from working in principle, except for kvmclock. Paolo I know nothing of that sort is implemented at this point. Leaving aside the issue of certain CPUID instructions suddenly returning different results, are there obvious problems even trying to go down that road? Would love to hear some thoughts on this - don't hesitate to tell me that's stupid and impossible because XYZ (assuming you also fill in the blanks ;) ) Thanks! Clemens
Re: [Qemu-devel] Disabling KVM on the fly
On 2012-10-17 18:44, Paolo Bonzini wrote: Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using migrate exec:catfoo.save with a KVM machine and -incoming 'exec:cat foo.save' with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), Unnecessary. Both models (KVM in-kernel and QEMU userspace) are compatible - in the absence of bugs. the differences between KVM and TCG are just that KVM doesn't initialize some TCG-only data structure, and that KVM uses many CPU threads; TCG uses one that goes through CPUs round-robin. The CPU threads of course execute different code. So no, in theory there is nothing that prevents this from working in principle, except for kvmclock. -cpu qemu64,-kvmclock should solve that. You also need -global pc-sysfw.rom_only=1 as KVM does not support write protected memory areas and creates an old-style BIOS region. But loading a KVM image into TCG lets non-trival guests lock up. Likely due to differences in the CPU virtualization/emulation (MSRs...). Also, certain KVM specific CPU states cannot be easily translated into TCG (and are definitely just ignored in TCG so far). Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux
