Re: [Qemu-devel] Disabling KVM "on the fly"
On Thu, Oct 18, 2012 at 10:24 AM, Jan Kiszka wrote: > On 2012-10-18 08:29, Paolo Bonzini wrote: >> Il 17/10/2012 20:37, Jan Kiszka ha scritto: >>> On 2012-10-17 18:44, Paolo Bonzini wrote: Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: > Guys, > > I know this is question might seem a bit odd, but I'm curious: > > Has anyone ever tried to write code to disable KVM on the fly / is it > at all possible? I have a situation where I need to use TCG for > certain parts of the code, but would love to have acceleration for > everything else. My idea was to pause the VM, then use the > snapshotting mechanism to dump the state, and then to resume the > snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using "migrate exec:cat>foo.save" with a KVM machine and "-incoming 'exec:cat foo.save'" with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), >>> >>> Unnecessary. Both models (KVM in-kernel and QEMU userspace) are >>> compatible - in the absence of bugs. >> >> He wants to really switch it on the fly---not just migrate out and >> in---and for that you need to disable the KVM-specific devices. > > Well, that's even more unrealistic than via migration. I am just looking for a way to do it, regardless whether it causes a big performance penalty due to migration, etc. Even if this means preparing a special environment (emulating specific hardware, etc), this is fine with me. Thanks a lot for all the feedback, now I know what I will have to look into! -Clemens
Re: [Qemu-devel] Disabling KVM "on the fly"
On 2012-10-18 08:29, Paolo Bonzini wrote: > Il 17/10/2012 20:37, Jan Kiszka ha scritto: >> On 2012-10-17 18:44, Paolo Bonzini wrote: >>> Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. >>> >>> As a start, you can try using "migrate exec:cat>foo.save" with a KVM >>> machine and "-incoming 'exec:cat foo.save'" with a TCG machine. The >>> main problem should be that TCG doesn't implement kvmclock. >>> >>> If you disable the KVM interrupt controller and timer (which is just an >>> implementation detail, not a hardware difference), >> >> Unnecessary. Both models (KVM in-kernel and QEMU userspace) are >> compatible - in the absence of bugs. > > He wants to really switch it on the fly---not just migrate out and > in---and for that you need to disable the KVM-specific devices. Well, that's even more unrealistic than via migration. > >> But loading a KVM image into TCG lets non-trival guests lock up. Likely >> due to differences in the CPU virtualization/emulation (MSRs...). > > Perhaps that can be mitigated by using an older machine model. Start > with something simple like a pentium2 and work up from there... Even if, there are still too many untranslated, maybe even untranslatable states of the KVM CPU model, at least. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux
Re: [Qemu-devel] Disabling KVM "on the fly"
Il 17/10/2012 20:37, Jan Kiszka ha scritto: > On 2012-10-17 18:44, Paolo Bonzini wrote: >> Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: >>> Guys, >>> >>> I know this is question might seem a bit odd, but I'm curious: >>> >>> Has anyone ever tried to write code to disable KVM on the fly / is it >>> at all possible? I have a situation where I need to use TCG for >>> certain parts of the code, but would love to have acceleration for >>> everything else. My idea was to pause the VM, then use the >>> snapshotting mechanism to dump the state, and then to resume the >>> snapshot, but writing the KVM state into the non-KVM structures. >> >> As a start, you can try using "migrate exec:cat>foo.save" with a KVM >> machine and "-incoming 'exec:cat foo.save'" with a TCG machine. The >> main problem should be that TCG doesn't implement kvmclock. >> >> If you disable the KVM interrupt controller and timer (which is just an >> implementation detail, not a hardware difference), > > Unnecessary. Both models (KVM in-kernel and QEMU userspace) are > compatible - in the absence of bugs. He wants to really switch it on the fly---not just migrate out and in---and for that you need to disable the KVM-specific devices. > But loading a KVM image into TCG lets non-trival guests lock up. Likely > due to differences in the CPU virtualization/emulation (MSRs...). Perhaps that can be mitigated by using an older machine model. Start with something simple like a pentium2 and work up from there... Paolo
Re: [Qemu-devel] Disabling KVM "on the fly"
On 2012-10-17 18:44, Paolo Bonzini wrote: > Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: >> Guys, >> >> I know this is question might seem a bit odd, but I'm curious: >> >> Has anyone ever tried to write code to disable KVM on the fly / is it >> at all possible? I have a situation where I need to use TCG for >> certain parts of the code, but would love to have acceleration for >> everything else. My idea was to pause the VM, then use the >> snapshotting mechanism to dump the state, and then to resume the >> snapshot, but writing the KVM state into the non-KVM structures. > > As a start, you can try using "migrate exec:cat>foo.save" with a KVM > machine and "-incoming 'exec:cat foo.save'" with a TCG machine. The > main problem should be that TCG doesn't implement kvmclock. > > If you disable the KVM interrupt controller and timer (which is just an > implementation detail, not a hardware difference), Unnecessary. Both models (KVM in-kernel and QEMU userspace) are compatible - in the absence of bugs. > the differences > between KVM and TCG are just that KVM doesn't initialize some TCG-only > data structure, and that KVM uses many CPU threads; TCG uses one that > goes through CPUs round-robin. The CPU threads of course execute > different code. > > So no, in theory there is nothing that prevents this from working in > principle, except for kvmclock. -cpu qemu64,-kvmclock should solve that. You also need -global pc-sysfw.rom_only=1 as KVM does not support write protected memory areas and creates an "old-style" BIOS region. But loading a KVM image into TCG lets non-trival guests lock up. Likely due to differences in the CPU virtualization/emulation (MSRs...). Also, certain KVM specific CPU states cannot be easily translated into TCG (and are definitely just ignored in TCG so far). Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux
Re: [Qemu-devel] Disabling KVM "on the fly"
Il 17/10/2012 18:37, Clemens Kolbitsch ha scritto: > Guys, > > I know this is question might seem a bit odd, but I'm curious: > > Has anyone ever tried to write code to disable KVM on the fly / is it > at all possible? I have a situation where I need to use TCG for > certain parts of the code, but would love to have acceleration for > everything else. My idea was to pause the VM, then use the > snapshotting mechanism to dump the state, and then to resume the > snapshot, but writing the KVM state into the non-KVM structures. As a start, you can try using "migrate exec:cat>foo.save" with a KVM machine and "-incoming 'exec:cat foo.save'" with a TCG machine. The main problem should be that TCG doesn't implement kvmclock. If you disable the KVM interrupt controller and timer (which is just an implementation detail, not a hardware difference), the differences between KVM and TCG are just that KVM doesn't initialize some TCG-only data structure, and that KVM uses many CPU threads; TCG uses one that goes through CPUs round-robin. The CPU threads of course execute different code. So no, in theory there is nothing that prevents this from working in principle, except for kvmclock. Paolo > I know nothing of that sort is implemented at this point. Leaving > aside the issue of certain CPUID instructions suddenly returning > different results, are there obvious problems even trying to go down > that road? > > Would love to hear some thoughts on this - don't hesitate to tell me > "that's stupid and impossible because XYZ" (assuming you also fill in > the blanks ;) ) > > Thanks! > Clemens >
[Qemu-devel] Disabling KVM "on the fly"
Guys, I know this is question might seem a bit odd, but I'm curious: Has anyone ever tried to write code to disable KVM on the fly / is it at all possible? I have a situation where I need to use TCG for certain parts of the code, but would love to have acceleration for everything else. My idea was to pause the VM, then use the snapshotting mechanism to dump the state, and then to resume the snapshot, but writing the KVM state into the non-KVM structures. I know nothing of that sort is implemented at this point. Leaving aside the issue of certain CPUID instructions suddenly returning different results, are there obvious problems even trying to go down that road? Would love to hear some thoughts on this - don't hesitate to tell me "that's stupid and impossible because XYZ" (assuming you also fill in the blanks ;) ) Thanks! Clemens -- Clemens Kolbitsch Security Researcher kolbit...@lastline.com +1 (206) 356-7745 Lastline, Inc. 6950 Hollister Avenue, Suite 101 Goleta, CA 93117 www.lastline.com