Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
On Tue, Sep 25, 2007 at 03:28:24AM +0200, andrzej zaborowski wrote: Hi, On 24/09/2007, Dan Kenigsberg [EMAIL PROTECTED] wrote: As with previous Takes of this patch, its purpose is to expose host +{ +asm(cpuid +: =a (*ax), + =b (*bx), + =c (*cx), + =d (*dx) +: a (function)); +} I haven't really read through the rest of your code but this piece appears to be outside any #ifdef/#endif so it will only build on x86. I might be missing something here, but isn't not being on the TARGET_PATH of Makefile.target enough? I don't see #ifdef TARGET_I386 elsewhere under target-i386. I don't mind adding extra protection, I just be happy to better understand the whats and whys. Dan.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Dan Kenigsberg wrote: On Tue, Sep 25, 2007 at 03:28:24AM +0200, andrzej zaborowski wrote: Hi, On 24/09/2007, Dan Kenigsberg [EMAIL PROTECTED] wrote: As with previous Takes of this patch, its purpose is to expose host +{ +asm(cpuid +: =a (*ax), + =b (*bx), + =c (*cx), + =d (*dx) +: a (function)); +} I haven't really read through the rest of your code but this piece appears to be outside any #ifdef/#endif so it will only build on x86. I might be missing something here, but isn't not being on the TARGET_PATH of Makefile.target enough? I don't see #ifdef TARGET_I386 elsewhere under target-i386. I don't mind adding extra protection, I just be happy to better understand the whats and whys. target-i386 means the guest will run i386 instructions, but the host can be something else (say, powerpc). Nothing else uses host instructions in that directory, so no protection was necessary before. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
On Tue, 2007-09-25 at 11:01 +0200, Avi Kivity wrote: Dan Kenigsberg wrote: On Tue, Sep 25, 2007 at 03:28:24AM +0200, andrzej zaborowski wrote: Hi, On 24/09/2007, Dan Kenigsberg [EMAIL PROTECTED] wrote: As with previous Takes of this patch, its purpose is to expose host +{ +asm(cpuid +: =a (*ax), + =b (*bx), + =c (*cx), + =d (*dx) +: a (function)); +} I haven't really read through the rest of your code but this piece appears to be outside any #ifdef/#endif so it will only build on x86. I might be missing something here, but isn't not being on the TARGET_PATH of Makefile.target enough? I don't see #ifdef TARGET_I386 elsewhere under target-i386. I don't mind adding extra protection, I just be happy to better understand the whats and whys. target-i386 means the guest will run i386 instructions, but the host can be something else (say, powerpc). Nothing else uses host instructions in that directory, so no protection was necessary before. I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Regards. -- J. Mayer [EMAIL PROTECTED] Never organized
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
J. Mayer wrote: On Tue, 2007-09-25 at 11:01 +0200, Avi Kivity wrote: Dan Kenigsberg wrote: On Tue, Sep 25, 2007 at 03:28:24AM +0200, andrzej zaborowski wrote: Hi, On 24/09/2007, Dan Kenigsberg [EMAIL PROTECTED] wrote: As with previous Takes of this patch, its purpose is to expose host +{ +asm(cpuid +: =a (*ax), + =b (*bx), + =c (*cx), + =d (*dx) +: a (function)); +} I haven't really read through the rest of your code but this piece appears to be outside any #ifdef/#endif so it will only build on x86. I might be missing something here, but isn't not being on the TARGET_PATH of Makefile.target enough? I don't see #ifdef TARGET_I386 elsewhere under target-i386. I don't mind adding extra protection, I just be happy to better understand the whats and whys. target-i386 means the guest will run i386 instructions, but the host can be something else (say, powerpc). Nothing else uses host instructions in that directory, so no protection was necessary before. I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Re: [Qemu-devel] expose host CPU features to guests: Take 3
andrzej zaborowski wrote: Hi, On 24/09/2007, Dan Kenigsberg [EMAIL PROTECTED] wrote: As with previous Takes of this patch, its purpose is to expose host CPU features to guests. It proved rather helpful to KVM in various benchmarks, and it should similarly speed kqemu up. Note that it does not extend the set of emulated opcodes, only exposes what's supported by the host CPU. Another purpose for Take 2 is to add the -cpu option to the x86 architecture, similarly to that of other architectures. -cpu 486, pentium, pentium2 and pentium3 are supported in addition to finer-grained features such as -cpu pentium,-mmx . As in Take 1, -cpu host exposes host features to guest. This patch exposes the requested CPU also right after RESET command, and not only in CPUID. Please let me know if you have more suggestions, Dan. [...] I haven't really read through the rest of your code but this piece appears to be outside any #ifdef/#endif so it will only build on x86. Regards A few remarks: 1) I cannot accept GPL code in target-i386, so the code from the Linux kernel must be removed or rewritten. 2) cpuid is already defined in kqemu.c, so it would be better to use it (I consider this is not a blocking point though). 3) For the future, I suggest that the function cpu_xxx_register() is removed and that the parameter xxx_def_t is added to cpu_xxx_init(). Rationale: I see no point in initializing a CPU without specifying its exact model and I am not sure that cpu_xxx_register() can be called once some code is executed. Regards, Fabrice.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Avi Kivity wrote: I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). Actually (as Izik Eidus reminds me), this isn't very useful for kqemu as it can't trap cpuid in all circumstances. So this is mainly useful for kvm. I hope it will be applied regardless of that, as there is agreement that kvm support should be merged. I'd much rather pull the feature from qemu rather than carry it as an additional patch. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
On Tue, 2007-09-25 at 12:40 +0200, Avi Kivity wrote: Avi Kivity wrote: I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). Actually (as Izik Eidus reminds me), this isn't very useful for kqemu as it can't trap cpuid in all circumstances. So this is mainly useful for kvm. I hope it will be applied regardless of that, as there is agreement that kvm support should be merged. I'd much rather pull the feature from qemu rather than carry it as an additional patch. Still I don't understand why it's usefull to put it inside the emulator and why: # qemu -cpu `guess_host_cpu` would not do the work properly. Adding a specific case for '-cpu host' seems useless to me. And this way of doing potentially work for any family of emulated targets, without any modification in Qemu. If the string returned by 'guess_host_cpu' is not recognized for the specific target you used it with, Qemu just stops telling it cannot find this CPU model, no more, no less. The only case it could be interresting, imho, is if you do not allow the -cpu option in KVM case and force the cpu model instead using this function. This behavior does not seem to be great idea to me. Or maybe I missed something ? -- J. Mayer [EMAIL PROTECTED] Never organized
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
J. Mayer wrote: On Tue, 2007-09-25 at 12:40 +0200, Avi Kivity wrote: Avi Kivity wrote: I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). Actually (as Izik Eidus reminds me), this isn't very useful for kqemu as it can't trap cpuid in all circumstances. So this is mainly useful for kvm. I hope it will be applied regardless of that, as there is agreement that kvm support should be merged. I'd much rather pull the feature from qemu rather than carry it as an additional patch. Still I don't understand why it's usefull to put it inside the emulator and why: # qemu -cpu `guess_host_cpu` would not do the work properly. Adding a specific case for '-cpu host' seems useless to me. And this way of doing potentially work for any family of emulated targets, without any modification in Qemu. If the string returned by 'guess_host_cpu' is not recognized for the specific target you used it with, Qemu just stops telling it cannot find this CPU model, no more, no less. It's a usability issue. I agree your suggestion would work, but I'd like the default for kvm to be using the host cpu features, whereas managed environments would specify some subset to enable live migration. The only case it could be interresting, imho, is if you do not allow the -cpu option in KVM case and force the cpu model instead using this function. This behavior does not seem to be great idea to me. I think we can move the host cpu checks to kvm-specific code, since it is not useful for kqemu. So, running qemu without any parameters would use host capabilities if kvm is available and the default qemu cpu if not. The -cpu option can be used to override this if necessary. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Avi Kivity wrote: J. Mayer wrote: On Tue, 2007-09-25 at 12:40 +0200, Avi Kivity wrote: Avi Kivity wrote: I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). Actually (as Izik Eidus reminds me), this isn't very useful for kqemu as it can't trap cpuid in all circumstances. So this is mainly useful for kvm. I hope it will be applied regardless of that, as there is agreement that kvm support should be merged. I'd much rather pull the feature from qemu rather than carry it as an additional patch. Still I don't understand why it's usefull to put it inside the emulator and why: # qemu -cpu `guess_host_cpu` would not do the work properly. Adding a specific case for '-cpu host' seems useless to me. And this way of doing potentially work for any family of emulated targets, without any modification in Qemu. If the string returned by 'guess_host_cpu' is not recognized for the specific target you used it with, Qemu just stops telling it cannot find this CPU model, no more, no less. It's a usability issue. I agree your suggestion would work, but I'd like the default for kvm to be using the host cpu features, whereas managed environments would specify some subset to enable live migration. The only case it could be interresting, imho, is if you do not allow the -cpu option in KVM case and force the cpu model instead using this function. This behavior does not seem to be great idea to me. I think we can move the host cpu checks to kvm-specific code, since it is not useful for kqemu. So, running qemu without any parameters would use host capabilities if kvm is available and the default qemu cpu if not. The -cpu option can be used to override this if necessary. Rectification: this is useful for kqemu too. I strongly suggest to look at kqemu.c:kqemu_update_cpuid() ! Moreover I believe that using the same CPU as host can be useful for pure emulation too, for example if code to do cache profiling is added. Regards, Fabrice.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
On Tuesday 25 September 2007, Avi Kivity wrote: J. Mayer wrote: On Tue, 2007-09-25 at 11:01 +0200, Avi Kivity wrote: Dan Kenigsberg wrote: On Tue, Sep 25, 2007 at 03:28:24AM +0200, andrzej zaborowski wrote: Hi, On 24/09/2007, Dan Kenigsberg [EMAIL PROTECTED] wrote: As with previous Takes of this patch, its purpose is to expose host +{ +asm(cpuid +: =a (*ax), + =b (*bx), + =c (*cx), + =d (*dx) +: a (function)); +} Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). And, as discussed before, it should be asking the hypervisor what features it supports instead of trying to guess from the cpuid output. Paul
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
On Tue, 2007-09-25 at 13:36 +0200, Avi Kivity wrote: J. Mayer wrote: On Tue, 2007-09-25 at 12:40 +0200, Avi Kivity wrote: Avi Kivity wrote: I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). Actually (as Izik Eidus reminds me), this isn't very useful for kqemu as it can't trap cpuid in all circumstances. So this is mainly useful for kvm. I hope it will be applied regardless of that, as there is agreement that kvm support should be merged. I'd much rather pull the feature from qemu rather than carry it as an additional patch. Still I don't understand why it's usefull to put it inside the emulator and why: # qemu -cpu `guess_host_cpu` would not do the work properly. Adding a specific case for '-cpu host' seems useless to me. And this way of doing potentially work for any family of emulated targets, without any modification in Qemu. If the string returned by 'guess_host_cpu' is not recognized for the specific target you used it with, Qemu just stops telling it cannot find this CPU model, no more, no less. It's a usability issue. I agree your suggestion would work, but I'd like the default for kvm to be using the host cpu features, whereas managed environments would specify some subset to enable live migration. The only case it could be interresting, imho, is if you do not allow the -cpu option in KVM case and force the cpu model instead using this function. This behavior does not seem to be great idea to me. I think we can move the host cpu checks to kvm-specific code, since it is not useful for kqemu. So, running qemu without any parameters would use host capabilities if kvm is available and the default qemu cpu if not. The -cpu option can be used to override this if necessary. Well, it may be needed to integrate the -cpu host option. But I think it's a really bad idea that running qemu without the -cpu option would not act the same on different host. When I want to test qemu with the same command line, I want it to behave exactly the same, whatever the host I'm running on, like any other tool. Think about gcc, for example, if you launch it without option, it compiles for a generic CPU. If you want to tune the generated code, even for the host you're running on, you have to use -mcpu/-march/-mtune. Using one command line always have gives the same result. Then, my opinion is that running qemu without any -cpu option should always use a default target.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Jocelyn Mayer wrote: On Tue, 2007-09-25 at 13:36 +0200, Avi Kivity wrote: J. Mayer wrote: On Tue, 2007-09-25 at 12:40 +0200, Avi Kivity wrote: Avi Kivity wrote: I've got a remark about this: why this has to be added to the Qemu code ? Imho, all is needed is an implementation of the -cpu option for x86/x86_64 target. Then, an external tool (even a shell script) can be used to determine what is the host CPU if you want to select the exact same CPU to be emulated in Qemu. It seems to me that trying to do so is out of the scope of Qemu code and just add unneeded complexity. Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). Actually (as Izik Eidus reminds me), this isn't very useful for kqemu as it can't trap cpuid in all circumstances. So this is mainly useful for kvm. I hope it will be applied regardless of that, as there is agreement that kvm support should be merged. I'd much rather pull the feature from qemu rather than carry it as an additional patch. Still I don't understand why it's usefull to put it inside the emulator and why: # qemu -cpu `guess_host_cpu` would not do the work properly. Adding a specific case for '-cpu host' seems useless to me. And this way of doing potentially work for any family of emulated targets, without any modification in Qemu. If the string returned by 'guess_host_cpu' is not recognized for the specific target you used it with, Qemu just stops telling it cannot find this CPU model, no more, no less. It's a usability issue. I agree your suggestion would work, but I'd like the default for kvm to be using the host cpu features, whereas managed environments would specify some subset to enable live migration. The only case it could be interresting, imho, is if you do not allow the -cpu option in KVM case and force the cpu model instead using this function. This behavior does not seem to be great idea to me. I think we can move the host cpu checks to kvm-specific code, since it is not useful for kqemu. So, running qemu without any parameters would use host capabilities if kvm is available and the default qemu cpu if not. The -cpu option can be used to override this if necessary. Well, it may be needed to integrate the -cpu host option. But I think it's a really bad idea that running qemu without the -cpu option would not act the same on different host. When I want to test qemu with the same command line, I want it to behave exactly the same, whatever the host I'm running on, like any other tool. Think about gcc, for example, if you launch it without option, it compiles for a generic CPU. If you want to tune the generated code, even for the host you're running on, you have to use -mcpu/-march/-mtune. Using one command line always have gives the same result. Then, my opinion is that running qemu without any -cpu option should always use a default target. That's a valid usage model. For many kvm users, however, the primary reason to run qemu is to get the performance that kvm provides, so they'd like to see host cpuid as the default. Maybe a .qemurc can help here. -- Any sufficiently difficult bug is indistinguishable from a feature.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Paul Brook wrote: Indeed for regular qemu this is useless. But it is useful for kqemu (for which there is support in mainline qemu), and for kvm (which we hope to merge one day). And, as discussed before, it should be asking the hypervisor what features it supports instead of trying to guess from the cpuid output. Agreed. -- Any sufficiently difficult bug is indistinguishable from a feature.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
On Tue, Sep 25, 2007 at 03:07:44PM +0200, Jocelyn Mayer wrote: So, running qemu without any parameters would use host capabilities if kvm is available and the default qemu cpu if not. The -cpu option can be used to override this if necessary. Well, it may be needed to integrate the -cpu host option. But I think it's a really bad idea that running qemu without the -cpu option would not act the same on different host. When I want to test qemu with the same command line, I want it to behave exactly the same, whatever the host I'm running on, like any other tool. Think about gcc, for example, if you launch it without option, it compiles for a generic CPU. If you want to tune the generated code, even for the host you're running on, you have to use -mcpu/-march/-mtune. Using one command line always have gives the same result. Then, my opinion is that running qemu without any -cpu option should always use a default target. Note that in take 3 the default is indeed to expose the basic default cpu features if no -cpu is specified. One must add -cpu host in order to read and expose the host cpuid.
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Jocelyn Mayer wrote: Well, it may be needed to integrate the -cpu host option. But I think it's a really bad idea that running qemu without the -cpu option would not act the same on different host. When I want to test qemu with the same command line, I want it to behave exactly the same, whatever the host I'm running on, like any other tool. Think about gcc, for example, if you launch it without option, it compiles for a generic CPU. If you want to tune the generated code, even for the host you're running on, you have to use -mcpu/-march/-mtune. Using one command line always have gives the same result. Then, my opinion is that running qemu without any -cpu option should always use a default target. A major feature of qemu is reproducible behaviour. This is especially important when resuming snapshots or booting pre-installed images for testing things. A major feature of kvm/kqemu is performance. But not always performance at the expense of reproducibility. Sometimes you want to use kvm/kqemu to make qemu as fast as possible while still behaving the same with some image. Where someone wants performance to have precedence, it's easy enough to advise use -cpu host for that. Where someone wants performance, but reproducibility to have precedence, it would be more awkward to have to advise use -cpu list,of,common,features. I think reproducibility is something which needs to be working by default. This is because people make images before they decide to move them to other hosts, and often before they would consider at all issues about host differences, and making images is sometimes a lot of work. Images should be portable among hosts by default. But, if we say that the default is a baseline CPU, and there's a -cpu host option to maximise performance, then we have to decide what features the baseline CPU has. Obviously a 386 isn't very useful for emulation any more - many OSes won't run on it. Then, if we change our mind about the default baseline CPU, then we've lost reproducibility for old images anyway. That's an argument for storing the emulated CPU features in images. If that's done, then for image portability among hosts, it is not so important that the default CPU have only the features which are native to every host. Instead, the critical thing is simply that it has only features which are either native or can be emulated on every host. E.g. TSC is in this category. -- Jamie
Re: [kvm-devel] [Qemu-devel] expose host CPU features to guests: Take 3
Jamie Lokier wrote: Jocelyn Mayer wrote: Well, it may be needed to integrate the -cpu host option. But I think it's a really bad idea that running qemu without the -cpu option would not act the same on different host. When I want to test qemu with the same command line, I want it to behave exactly the same, whatever the host I'm running on, like any other tool. Think about gcc, for example, if you launch it without option, it compiles for a generic CPU. If you want to tune the generated code, even for the host you're running on, you have to use -mcpu/-march/-mtune. Using one command line always have gives the same result. Then, my opinion is that running qemu without any -cpu option should always use a default target. A major feature of qemu is reproducible behaviour. This is especially important when resuming snapshots or booting pre-installed images for testing things. A major feature of kvm/kqemu is performance. But not always performance at the expense of reproducibility. Sometimes you want to use kvm/kqemu to make qemu as fast as possible while still behaving the same with some image. Where someone wants performance to have precedence, it's easy enough to advise use -cpu host for that. Where someone wants performance, but reproducibility to have precedence, it would be more awkward to have to advise use -cpu list,of,common,features. Or maybe -cpu baseline or some other neutral word. I think reproducibility is something which needs to be working by default. This is because people make images before they decide to move them to other hosts, and often before they would consider at all issues about host differences, and making images is sometimes a lot of work. Images should be portable among hosts by default. But, if we say that the default is a baseline CPU, and there's a -cpu host option to maximise performance, then we have to decide what features the baseline CPU has. Obviously a 386 isn't very useful for emulation any more - many OSes won't run on it. The baseline cpu should be whatever qemu supports today. Then, if we change our mind about the default baseline CPU, then we've lost reproducibility for old images anyway. That's an argument for storing the emulated CPU features in images. That's another thread -- I want memory size and network configuration in there too :) If that's done, then for image portability among hosts, it is not so important that the default CPU have only the features which are native to every host. Instead, the critical thing is simply that it has only features which are either native or can be emulated on every host. E.g. TSC is in this category. Qemu is used to support three different user classes which have different requirements. Developers want reproducibility and control, so -cpu baseline should be the default for them. Home users want speed to run Windows or test distros, so -cpu host and kvm should be the default for them. Managed deployments want something in between, so they'd specify -cpu feature,feature and won't care about the default. We can make it a ./configure option. I suspect distros will want to set -cpu host as default as their users primarily care about speed. -- Any sufficiently difficult bug is indistinguishable from a feature.