Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 11:23:16PM +0200, Michael S. Tsirkin wrote:
 
 So any single CPU flag now needs to be added in
 - kvm
 - qemu
 - libvirt

This is in fact already the case, and it will also possibly need
to be added to openstack too.

 Next thing libvirt will decide it's a policy thing and so
 needs to be pushed up to openstack.

In fact openstack would really like it if we did exactly that, but
even just having CPUs versioned separately from machine types would
be a big step forward as far as openstack is concerned.

The openstack schedular does not have full visibility into the way
the guest is going to be configured by libvirt/QEMU, in particular
it does not know anything about machine type that will be used by
the guest.

The compute hosts report what CPU features they can support, and
the user / admin will be able to express what CPU model and/or
features they desire their guest to run with, and the schedular
has to match that up to decide on hosts to use. If the CPU QEMU
machine type used can alter what the CPU model means in terms
of features, then the schedling decisions OpenStack is making
are going to be wrong some portion of the time.

So from the POV of the OpenStack schedular, we'd much rather
have CPU models versioned explicitly so their semantics do not
change behind our back based on machine types.

OpenStack is also looking at how to present a consistent
description of CPU models to users, which is independant of
the cloud. Since libvirt/QEMU doesn't allow 100% control of
the CPU model, OpenStack is likely going to have to define
some kind of manual mapping of its own.

 We should just figure out what you want to do and support it in QEMU.

Main thing is versioned CPU models with fixed semantics that
don't magically change based on other aspects of VM configuration,
such as the machine type. This could be accomplished by QEMU
alone.

Following on from that though, there's two other aspects which
we'd like to address. First, be able to present a consistent
set of CPU models across all hypervisors libvirt manages,
regardless of type or version. This is a key reason why we have
always maintained our own CPU model database, even though it
duplicates QEMU's.

More interesting is the question of host passthrough. We have
two modes for that - either 'host-model' or 'host-passthrough'.
The 'host-passthrough' option is something that explicitly
maps to QEMU's  '-cpu host'. This is good because it exposes
100% of the host CPU to the guest. This is bad because it then
prevents use of migration in general, unless both machines
are 100% identical - libvirt just blocks it rather than trying
todo such a fine grained host CPU check.

For that reason we have 'host-model', which is supposed to be
essentially the same thing instead of '-cpu host' we explicitly
list all the features alongside a named model. Since we control
exactly what the guest is being given, we can permit guests
with 'host-model' to be migrated, even if the destination host
is a superset of the source host, we know the guest won't
suddenly see a model change after migration. Currently we are
limited though, as we can only express the CPU features - we
cannot expose the other aspects like level, xlevel, etc. So
our 'host-model' is not quite as perfect a  match as '-cpu host'
is. The '-cpu custom' would help us getting a better match
for 'host-model' by allowing these extra things to be configured.

 Are there many examples where a single flag used to work and then
 stopped? I would say such a change is problematic anyway:
 not everyone uses libvirt, you are breaking things for people
 that run -M pc.
 
 IMHO -M pc is not supposed to mean can break at any time.

Well 'pc' is an unversioned machine type, so it explicitly is said to
break at any time - users/apps are supposed to translate that into a
versioned type if they want a guarantee of no breakage.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Jiri Denemark
On Tue, Jun 23, 2015 at 14:32:00 +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script is provided,
  that will generate a config file that can be loaded using -readconfig, 
  based on
  the -cpu and -machine options provided in the command-line.
  
  Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
  configuration data to libvirt, but now I think it actually makes sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
 That sounds like FUD to me. Any concrete data points where QEMU does not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
 for.

QEMU provides stable ABI for x86 CPUs only if you use -cpu ...,enforce.
Without enforce the CPU may change everytime a domain is started or
migrated. A small example: let's say a CPU model called Model includes
feature xyz; when QEMU is started with -cpu Model (no enforce) on a
host which supports xyz, the guest OS will see a CPU with xyz, but when
you migrate it to a host which does not support xyz, QEMU will just
silently drop xyz. In other words, we need to use enforce to make sure
CPU ABI does not change.

But the problem is we can't use enforce because we don't know how a
specific CPU model looks like for a given machine type. Remember, while
libvirt allows users to explicitly ask for a specific CPU model and
features, it also has a mode when libvirt itself computes the right CPU
model and features. And this is impossible with enforce without us
knowing all details about CPU models.

So there are two possible ways to address this:
1. enhance QEMU to give us all we need
- either by providing commands that would do all the computations
  (CPU model comparison, intersections or denominator, something
  like -cpu best)
- or provide a way to probe for all (currently 700+) combinations of
  a CPU model and a machine type without actually having to start
  QEMU with each CPU and a machine type separately

2. manage CPU models in libvirt (aka -cpu custom)

During the past several years Eduardo tried to do (1) without getting
anywhere close to something that QEMU would be willing to accept. On the
other hand (2) is a pretty minimal change to QEMU and is more flexible
than (1) because it allows CPU model versions to be decoupled from
machine types (but this was already discussed a lot in the other emails
in this thread).

Jirka



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 11:20:50AM +0200, Jiri Denemark wrote:
 On Tue, Jun 23, 2015 at 14:32:00 +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
   To help libvirt in the transition, a x86-cpu-model-dump script is 
   provided,
   that will generate a config file that can be loaded using -readconfig, 
   based on
   the -cpu and -machine options provided in the command-line.
   
   Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
   configuration data to libvirt, but now I think it actually makes sense.
   We already have a partial copy of CPU model definitions in libvirt
   anyway, but as QEMU changes some CPU models in some machine types (and
   libvirt does not do that) we have no real control over the guest CPU
   configuration. While what we really want is full control to enforce
   stable guest ABI.
  
  That sounds like FUD to me. Any concrete data points where QEMU does not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
  for.
 
 QEMU provides stable ABI for x86 CPUs only if you use -cpu ...,enforce.
 Without enforce the CPU may change everytime a domain is started or
 migrated. A small example: let's say a CPU model called Model includes
 feature xyz; when QEMU is started with -cpu Model (no enforce) on a
 host which supports xyz, the guest OS will see a CPU with xyz, but when
 you migrate it to a host which does not support xyz, QEMU will just
 silently drop xyz. In other words, we need to use enforce to make sure
 CPU ABI does not change.

Are there really many examples like this?  Could someone supply some
examples? Eduardo gave examples of CPU changes across machine types
but I haven't seen examples where we would break runnability.

 But the problem is we can't use enforce because we don't know how a
 specific CPU model looks like for a given machine type. Remember, while
 libvirt allows users to explicitly ask for a specific CPU model and
 features, it also has a mode when libvirt itself computes the right CPU
 model and features. And this is impossible with enforce without us
 knowing all details about CPU models.
 
 So there are two possible ways to address this:
 1. enhance QEMU to give us all we need
 - either by providing commands that would do all the computations
   (CPU model comparison, intersections or denominator, something
   like -cpu best)
 - or provide a way to probe for all (currently 700+) combinations of
   a CPU model and a machine type without actually having to start
   QEMU with each CPU and a machine type separately
 
 2. manage CPU models in libvirt (aka -cpu custom)
 
 During the past several years Eduardo tried to do (1) without getting
 anywhere close to something that QEMU would be willing to accept.

And the reason, presumably, is because it's a hard problem to solve.
Why is it easier to solve at the libvirt level?

 On the
 other hand (2) is a pretty minimal change to QEMU and is more flexible
 than (1) because it allows CPU model versions to be decoupled from
 machine types (but this was already discussed a lot in the other emails
 in this thread).
 
 Jirka

I'm fine with the change itself, it's useful e.g. for testing.

But how is it a solution for libvirt's problems?
What is libvirt going to do in the above cases?

-- 
MST



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 09:52:09AM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 11:23:16PM +0200, Michael S. Tsirkin wrote:
  
  So any single CPU flag now needs to be added in
  - kvm
  - qemu
  - libvirt
 
 This is in fact already the case, and it will also possibly need
 to be added to openstack too.
 
  Next thing libvirt will decide it's a policy thing and so
  needs to be pushed up to openstack.
 
 In fact openstack would really like it if we did exactly that, but
 even just having CPUs versioned separately from machine types would
 be a big step forward as far as openstack is concerned.
 
 The openstack schedular does not have full visibility into the way
 the guest is going to be configured by libvirt/QEMU, in particular
 it does not know anything about machine type that will be used by
 the guest.
 
 The compute hosts report what CPU features they can support, and
 the user / admin will be able to express what CPU model and/or
 features they desire their guest to run with, and the schedular
 has to match that up to decide on hosts to use. If the CPU QEMU
 machine type used can alter what the CPU model means in terms
 of features, then the schedling decisions OpenStack is making
 are going to be wrong some portion of the time.

Is this all just theoretical, or are there real examples
where things stop running? I keep hearing feature xyz
and it's impossible to argue reasonably about that IMHO.


 So from the POV of the OpenStack schedular, we'd much rather
 have CPU models versioned explicitly so their semantics do not
 change behind our back based on machine types.
 
 OpenStack is also looking at how to present a consistent
 description of CPU models to users, which is independant of
 the cloud. Since libvirt/QEMU doesn't allow 100% control of
 the CPU model, OpenStack is likely going to have to define
 some kind of manual mapping of its own.

  We should just figure out what you want to do and support it in QEMU.
 
 Main thing is versioned CPU models with fixed semantics that
 don't magically change based on other aspects of VM configuration,
 such as the machine type. This could be accomplished by QEMU
 alone.
 
 Following on from that though, there's two other aspects which
 we'd like to address. First, be able to present a consistent
 set of CPU models across all hypervisors libvirt manages,
 regardless of type or version. This is a key reason why we have
 always maintained our own CPU model database, even though it
 duplicates QEMU's.


Do you also want to migrate that? If yes, the problem
definitely becomes more than just CPU specific.


 More interesting is the question of host passthrough. We have
 two modes for that - either 'host-model' or 'host-passthrough'.
 The 'host-passthrough' option is something that explicitly
 maps to QEMU's  '-cpu host'. This is good because it exposes
 100% of the host CPU to the guest. This is bad because it then
 prevents use of migration in general, unless both machines
 are 100% identical - libvirt just blocks it rather than trying
 todo such a fine grained host CPU check.
 
 For that reason we have 'host-model', which is supposed to be
 essentially the same thing instead of '-cpu host' we explicitly
 list all the features alongside a named model. Since we control
 exactly what the guest is being given, we can permit guests
 with 'host-model' to be migrated, even if the destination host
 is a superset of the source host, we know the guest won't
 suddenly see a model change after migration. Currently we are
 limited though, as we can only express the CPU features - we
 cannot expose the other aspects like level, xlevel, etc. So
 our 'host-model' is not quite as perfect a  match as '-cpu host'
 is. The '-cpu custom' would help us getting a better match
 for 'host-model' by allowing these extra things to be configured.

This duplicates code from QEMU though.
It looks like we need a tool to get the legal CPUs
that can run on the given host?
Would be easy to add, reusing QEMU codebase.
Maybe make it a new QEMU flag.


  Are there many examples where a single flag used to work and then
  stopped? I would say such a change is problematic anyway:
  not everyone uses libvirt, you are breaking things for people
  that run -M pc.
  
  IMHO -M pc is not supposed to mean can break at any time.
 
 Well 'pc' is an unversioned machine type, so it explicitly is said to
 break at any time - users/apps are supposed to translate that into a
 versioned type if they want a guarantee of no breakage.
 
 Regards,
 Daniel

That's because you are looking at it from libvirt perspective.  From
QEMU command line, since pc is the default, this makes no sense IMHO:
look up usage advice on the internet, and you will see no one
specifies a machine type.


 -- 
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Daniel P. Berrange
On Wed, Jun 24, 2015 at 12:21:57PM +0200, Michael S. Tsirkin wrote:
 On Wed, Jun 24, 2015 at 11:20:50AM +0200, Jiri Denemark wrote:
  On Tue, Jun 23, 2015 at 14:32:00 +0200, Andreas Färber wrote:
   Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
To help libvirt in the transition, a x86-cpu-model-dump script is 
provided,
that will generate a config file that can be loaded using -readconfig, 
based on
the -cpu and -machine options provided in the command-line.

Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
configuration data to libvirt, but now I think it actually makes sense.
We already have a partial copy of CPU model definitions in libvirt
anyway, but as QEMU changes some CPU models in some machine types (and
libvirt does not do that) we have no real control over the guest CPU
configuration. While what we really want is full control to enforce
stable guest ABI.
   
   That sounds like FUD to me. Any concrete data points where QEMU does not
   have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
   for.
  
  QEMU provides stable ABI for x86 CPUs only if you use -cpu ...,enforce.
  Without enforce the CPU may change everytime a domain is started or
  migrated. A small example: let's say a CPU model called Model includes
  feature xyz; when QEMU is started with -cpu Model (no enforce) on a
  host which supports xyz, the guest OS will see a CPU with xyz, but when
  you migrate it to a host which does not support xyz, QEMU will just
  silently drop xyz. In other words, we need to use enforce to make sure
  CPU ABI does not change.
 
 Are there really many examples like this?  Could someone supply some
 examples? Eduardo gave examples of CPU changes across machine types
 but I haven't seen examples where we would break runnability.
 
  But the problem is we can't use enforce because we don't know how a
  specific CPU model looks like for a given machine type. Remember, while
  libvirt allows users to explicitly ask for a specific CPU model and
  features, it also has a mode when libvirt itself computes the right CPU
  model and features. And this is impossible with enforce without us
  knowing all details about CPU models.
  
  So there are two possible ways to address this:
  1. enhance QEMU to give us all we need
  - either by providing commands that would do all the computations
(CPU model comparison, intersections or denominator, something
like -cpu best)
  - or provide a way to probe for all (currently 700+) combinations of
a CPU model and a machine type without actually having to start
QEMU with each CPU and a machine type separately
  
  2. manage CPU models in libvirt (aka -cpu custom)
  
  During the past several years Eduardo tried to do (1) without getting
  anywhere close to something that QEMU would be willing to accept.
 
 And the reason, presumably, is because it's a hard problem to solve.
 Why is it easier to solve at the libvirt level?

One of the main reasons it is hard is because QEMU machine types are
not statically introspectable - you have to actually instantiate the
machine type to determine what config it produces. This is ultimately
a limitation of QOM, and while it could be fixed it would be a pretty
significant design change for QEMU at this point. So the reason it
would be simpler in libvirt is that we would not have any need to
attempt such introspection - the data we need would immediately
available to libvirt in the format it needs to use it in.

The OpenStack scheduling example I mentioned elsewhere is another
reason where the current scheme causes pain - the point at which
OpenStack wants to make decisions about host/guest CPU compatibility,
we don't even have a guest configuration available yet, so we don't
know what machine type we'd want to use, and QEMU isn't even installed
on the hosts doing this decision making. Currently OpenStack just has
to pretend that CPU models don't change based on machine type. Most of
the time we'll be lucky and that won't hurt us, but obviously it is
not a desirable thing to have todo.

  On the
  other hand (2) is a pretty minimal change to QEMU and is more flexible
  than (1) because it allows CPU model versions to be decoupled from
  machine types (but this was already discussed a lot in the other emails
  in this thread).
  
  Jirka
 
 I'm fine with the change itself, it's useful e.g. for testing.
 
 But how is it a solution for libvirt's problems?
 What is libvirt going to do in the above cases?

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Paolo Bonzini


On 24/06/2015 12:21, Michael S. Tsirkin wrote:
  QEMU provides stable ABI for x86 CPUs only if you use -cpu ...,enforce.
  Without enforce the CPU may change everytime a domain is started or
  migrated. A small example: let's say a CPU model called Model includes
  feature xyz; when QEMU is started with -cpu Model (no enforce) on a
  host which supports xyz, the guest OS will see a CPU with xyz, but when
  you migrate it to a host which does not support xyz, QEMU will just
  silently drop xyz. In other words, we need to use enforce to make sure
  CPU ABI does not change.
 
 Are there really many examples like this?  Could someone supply some
 examples? Eduardo gave examples of CPU changes across machine types
 but I haven't seen examples where we would break runnability.

Same here, and I would be quite surprised of seeing any.  Except perhaps
when we introduced x2apic: that would have broken kernels 2.6.32, but I
hope we can ignore those.

At least as far as I've maintained KVM, we've not added new models to
QEMU after KVM's kernel support was added.

The only case I can imagine is that the kernel is ancient so it doesn't
have anything newer than say IvyBridge, while QEMU is new.  If the user
starts a Haswell VM on a SandyBridge host without enforce, you will
have a problem when you reboot and get a newer kernel, but this is
independent of machine types.

We have added new features in exactly three cases:

1) F16 and RDRAND to {Haswell,Broadwell} in 2.3;

2) MOVBE to n270 in 1.5;

3) PCLMULQDQ to Westmere also in 1.5.

In all three cases, libvirt is still buggy and doesn't let you use the
features if you have the appropriate host.

We were close to breaking libvirt's expectations when we wanted to
remove TSX from Haswell/Broadwell, but in the end we did it right by
adding separate CPU models and no final release broke libvirt.

So, what is the broken case?  And BTW, what is _exactly_ preventing
libvirt from using enforce for models other than qemu32/qemu64?

Paolo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 11:31:37AM +0100, Daniel P. Berrange wrote:
 On Wed, Jun 24, 2015 at 12:21:57PM +0200, Michael S. Tsirkin wrote:
  On Wed, Jun 24, 2015 at 11:20:50AM +0200, Jiri Denemark wrote:
   On Tue, Jun 23, 2015 at 14:32:00 +0200, Andreas Färber wrote:
Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script is 
 provided,
 that will generate a config file that can be loaded using 
 -readconfig, based on
 the -cpu and -machine options provided in the command-line.
 
 Thanks Eduardo, I never was a big fan of moving (or copying) all the 
 CPU
 configuration data to libvirt, but now I think it actually makes 
 sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

That sounds like FUD to me. Any concrete data points where QEMU does not
have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
for.
   
   QEMU provides stable ABI for x86 CPUs only if you use -cpu ...,enforce.
   Without enforce the CPU may change everytime a domain is started or
   migrated. A small example: let's say a CPU model called Model includes
   feature xyz; when QEMU is started with -cpu Model (no enforce) on a
   host which supports xyz, the guest OS will see a CPU with xyz, but when
   you migrate it to a host which does not support xyz, QEMU will just
   silently drop xyz. In other words, we need to use enforce to make sure
   CPU ABI does not change.
  
  Are there really many examples like this?  Could someone supply some
  examples? Eduardo gave examples of CPU changes across machine types
  but I haven't seen examples where we would break runnability.

^^^ ???


   But the problem is we can't use enforce because we don't know how a
   specific CPU model looks like for a given machine type. Remember, while
   libvirt allows users to explicitly ask for a specific CPU model and
   features, it also has a mode when libvirt itself computes the right CPU
   model and features. And this is impossible with enforce without us
   knowing all details about CPU models.
   
   So there are two possible ways to address this:
   1. enhance QEMU to give us all we need
   - either by providing commands that would do all the computations
 (CPU model comparison, intersections or denominator, something
 like -cpu best)
   - or provide a way to probe for all (currently 700+) combinations of
 a CPU model and a machine type without actually having to start
 QEMU with each CPU and a machine type separately
   
   2. manage CPU models in libvirt (aka -cpu custom)
   
   During the past several years Eduardo tried to do (1) without getting
   anywhere close to something that QEMU would be willing to accept.
  
  And the reason, presumably, is because it's a hard problem to solve.
  Why is it easier to solve at the libvirt level?
 
 One of the main reasons it is hard is because QEMU machine types are
 not statically introspectable - you have to actually instantiate the
 machine type to determine what config it produces.
 This is ultimately
 a limitation of QOM, and while it could be fixed it would be a pretty
 significant design change for QEMU at this point. So the reason it
 would be simpler in libvirt is that we would not have any need to
 attempt such introspection - the data we need would immediately
 available to libvirt in the format it needs to use it in.

Why do you want to poke at QEMU machine types?
Looks like a small utility printing list of CPU types
compatible with the host should be enough.

 
 The OpenStack scheduling example I mentioned elsewhere is another
 reason where the current scheme causes pain - the point at which
 OpenStack wants to make decisions about host/guest CPU compatibility,
 we don't even have a guest configuration available yet, so we don't
 know what machine type we'd want to use, and QEMU isn't even installed
 on the hosts doing this decision making.

And libvirt is installed? Make that host query utility separate from
qemu then, make libvirt depend on it.

 Currently OpenStack just has
 to pretend that CPU models don't change based on machine type. Most of
 the time we'll be lucky and that won't hurt us, but obviously it is
 not a desirable thing to have todo.

Still wonder whether if it's a theoretical problem.
If yes, why write a ton of code to support it?

   On the
   other hand (2) is a pretty minimal change to QEMU and is more flexible
   than (1) because it allows CPU model versions to be decoupled from
   machine types (but this was already discussed a lot in the other emails
   in this thread).
   
   Jirka
  
  I'm fine with the change itself, it's useful e.g. 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 11:23:16PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:42:04PM +0100, Daniel P. Berrange wrote:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
   On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script 
  is provided,
  that will generate a config file that can be loaded using 
  -readconfig, based on
  the -cpu and -machine options provided in the command-line.
 
  Thanks Eduardo, I never was a big fan of moving (or copying) 
  all the CPU
  configuration data to libvirt, but now I think it actually 
  makes sense.
  We already have a partial copy of CPU model definitions in 
  libvirt
  anyway, but as QEMU changes some CPU models in some machine 
  types (and
  libvirt does not do that) we have no real control over the 
  guest CPU
  configuration. While what we really want is full control to 
  enforce
  stable guest ABI.
 
  That sounds like FUD to me. Any concrete data points where QEMU 
  does not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
  machines
  for.
 
  What Jiri is saying that the CPUs change depending on -mmachine, 
  not
  that the ABI is broken by a given machine.
 
  The problem here is that libvirt needs to provide CPU models whose
  runnability does not depend on the machine-type. If users have a 
  VM that
  is running in a host and the VM machine-type changes,
 
  How does it change, and why?
  
  Sometimes we add features to a CPU model because they were not 
  emulated by KVM
  and now they are. Sometimes we remove or add features or change 
  other fields
  because we are fixing previous mistakes. Recently we we were going 
  to remove
  features from models because of an Intel CPU errata, but then 
  decided to create
  a new CPU model name instead.
  
  See some examples at the end of this message.
  
 
  the VM should be
  still runnable in that host. QEMU doesn't provide that, our CPU 
  models
  may change when we introduce new machine-types, so we are giving 
  them a
  mechanism that allows libvirt to implement the policy they need.
 
  I don't mind wrt CPU specifically, but we absolutely do change 
  guest ABI
  in many ways when we change machine types.
  
  All the other ABI changes we introduce in QEMU don't affect 
  runnability of the
  VM in a given host, that's the problem we are trying to address 
  here. ABI
  changes are expected when changing to a new machine, runnability 
  changes
  aren't.
  
  
  Examples of commits changing CPU models:
 [snip]
 
 I've always advocated remaining backwards-compatible and only making 
 CPU
 model changes for new machines. You among others felt that was not
 always necessary, and now you're using the lack thereof as an argument
 to stop using QEMU's CPU models at all? That sounds convoluted...

Whether QEMU changed the CPU for existing machines, or only for new
machines is actually not the core problem. Even if we only changed
the CPU in new machines that would still be an unsatisfactory situation
because we want to be able to be able to access different versions of
the CPU without the machine type changing, and access different versions
of the machine type, without the CPU changing. IOW it is the fact that 
the
changes in CPU are tied to changes in machine type that is the core
problem.
   
   But that's because we are fixing bugs.  If CPU X used to work on
   hardware Y in machine type A and stopped in machine type B, this is
   because we have determined that it's the right thing to do for the
   guests and the users. We don't break stuff just for fun.
   Why do you want to bring back the bugs we fixed?
  
  Huh, I never said we wanted to bring back bugs. This is about allowing
  libvirt to fix the CPU bugs in a way that is independant of the machine
  types and portable across hypervisors we deal with. We're absolutely
  still going to fix CPU model bugs and ensure stable guest ABI.
  
  Regards,
  Daniel
 
 So any single CPU flag now needs to be added in
 - kvm
 - qemu
 - libvirt
 
 Next thing libvirt will decide it's a policy thing and so
 needs to be pushed up to openstack.

I don't think that will happen, but if they really decide 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Paolo Bonzini


On 24/06/2015 16:54, Peter Maydell wrote:
  It's certainly okay for libvirt and OpenStack to use the host CPU
  features in order to check whether a node will run a given VM.  However,
  libvirt should trust that QEMU developers will not prevent a VM from
  running on a previously viable host, just because you change the machine
  type.
 
 Note that this might be true for x86 but isn't (in full generality)
 necessarily so for other architectures like ARM. For instance if
 you have a host machine with a GICv3 with no v2-back-compatibility
 support, then you can run a VM whose machine type specifies a GICv3,
 but not machine types that require a GICv2...

Hmm, that was too terse.  Just because you change the machine type *to a
newer version* with the same chipset.

Paolo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Peter Maydell
On 24 June 2015 at 15:38, Paolo Bonzini pbonz...@redhat.com wrote:
 It's certainly okay for libvirt and OpenStack to use the host CPU
 features in order to check whether a node will run a given VM.  However,
 libvirt should trust that QEMU developers will not prevent a VM from
 running on a previously viable host, just because you change the machine
 type.

Note that this might be true for x86 but isn't (in full generality)
necessarily so for other architectures like ARM. For instance if
you have a host machine with a GICv3 with no v2-back-compatibility
support, then you can run a VM whose machine type specifies a GICv3,
but not machine types that require a GICv2...

thanks
-- PMM



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Paolo Bonzini


On 24/06/2015 16:16, Eduardo Habkost wrote:
  So any single CPU flag now needs to be added in
  - kvm
  - qemu
  - libvirt
  
  Next thing libvirt will decide it's a policy thing and so
  needs to be pushed up to openstack.
 
 I don't think that will happen, but if they really decide do do it, why
 should we try to stop them? libvirt and OpenStack know what their users
 do/need better than us, and if they believe moving data to OpenStack
 will provide what users need, they are free to do it. I trust libvirt
 developers to do the right thing, here.

After talking to Daniel for more than 1 hour I actually think that
OpenStack's scheduler is totally broken with respect to QEMU upgrades.
For some reason they focused on CPU model changes, but actually there's
no past case where runnability actually changed with a QEMU upgrade, and
there will be no such future case; even without enforce, QEMU
introduces new models long after all features have been available in
KVM.  At this point CPU is no different from any other device.

At the same time, OpenStack's scheduler is not trying to use a machine
type that is available on all nodes of a compute node pool.  I'm not
sure how one can be sure that migration would succeed in these
circumstances.

It's certainly okay for libvirt and OpenStack to use the host CPU
features in order to check whether a node will run a given VM.  However,
libvirt should trust that QEMU developers will not prevent a VM from
running on a previously viable host, just because you change the machine
type.

And OpenStack should _really_ use the machine type as the abstract
representation of what is compatible with what inside a node pool.  This
is what RHEV does, and I see no reason to do it differently.  In
particular there will not be excessive fragmentation of the nodes into
too many pools, because a deployment with too many active machine types
is just asking for trouble.

The possible exception are weird cases involving nested virt and
outdated QEMU on the L0 host, but even those cases can be fixed just by
having an up-to-date cpu_map.xml.

Other random notes from my chat:

1) libvirt should _not_ change the flags that the user passes via XML
just because it thinks that QEMU has those flags.  This makes it
possible for libvirt to keep cpu_map.xml up-to-date without worrying
about versioning.

2) libvirt should not add/remove flags when the user specifies
host-model (i.e. -cpu SandyBridge, not -cpu
SandyBridge,+f16c,+rdrand,+erms,+whatever).  host-model has had so many
bugs reported for it that I hope this could be done unconditionally even
if it is not backwards-compatible.  Or perhaps introduce a new name and
deprecate host-model.  I don't know.

3) regarding enforce, there are indeed some cases where it would break:

- Haswell/Broadwell CPU model after TSX removal

- qemu64 with KVM

- pretty much everything including qemu64 with TCG

So libvirt here could allow _now_ the user to specify enforce, either
via XML or via qemu.conf (or via XML + a default specified via qemu.conf).


So, I _hate_ to block a feature that is anyway useful for debugging.
And this feels too much like the typical kernel developers don't like
systemd rant.  But it looks like this feature is not only too easy to
misuse: there are _already_ plans for misusing it and put the whole
machine compatibility issue under the rug in OpenStack.  So I agree with
Andreas, and would prefer to have a serious discussion with the
OpenStack folks before accepting it.

Paolo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 11:16:51AM -0300, Eduardo Habkost wrote:
  IMHO -M pc is not supposed to mean can break at any time.
 
 It means it may have new host-side requirements and may become runnable
 in your host (or require additional command-line flags to run) at any time.

That would be pretty bad. I don't think we ever had such cases in practice.

 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 11:28:06PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 02:42:37PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 07:29:15PM +0200, Andreas Färber wrote:
   In summary you seem to be saying that all the years we have spent
   fiddling around with those mind-boggling compat_props in QEMU were in
   vain because libvirt now wants to start their own versioning system to
   give users more degrees of freedom even when you can't articulate a
   single concrete reason why users may want to do so.
  
  I had a similar reaction when I learned about this libvirt
  expectation/requirement I was never aware of. But we spent lots of
  effort trying to do things differently doesn't seem like a valid
  justification for design decision.
 
 Users will be hurt because they'll run untested configurations
 seems like a valid reason.

I trust libvirt developers to test their CPU definitions as carefully as
we test ours.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 11:34:45PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 02:11:22PM -0300, Eduardo Habkost wrote:
  Even if it is a bug fix. If it is a change that can make the VM
  unrunnable, it needs to be controlled by a separate flag, not by the
  machine-type.
 
 I agree - command line compatibility is important.  But we are supposed
 to provide that.  I am surprised that libvirt suddenly wants to avoid
 some command line flags because they are not stable. IMHO we did something
 wrong here if so. Maybe there was a valid reason for it. But then won't
 it apply to libvirt as well?

Maybe we are having the same misunderstanding here: the problem is not
compatibility/stability of existing machines, but the kind of
(intentional) changes introduced in _new_ machines (when the -machine
argument is changed). There are two kinds of changes introduced in new
machines:

1) Guest-side-only ABI changes: those are OK, libvirt normally ignore
   them, they can't make a VM not-runnable.
2) Changes in the host-side dependencies: those need to be more carefully
   controlled by libvirt. That's where CPU features are special: all CPU
   features depend KVM-side features, and enabling them by default on
   new machines makes it impossible for libvirt to know/report in
   advance what's necessary to make a VM runnable and to implement their
   existing runnability APIs[1].

Unless we guarantee that QEMU would never introduce type-(2) changes in
new machines (which I don't think will ever happen because that means
never changing existing CPU models in QEMU), libvirt needs to control
CPU features individually (that's why they need -cpu custom).

 
 Now, if people want to update CPU models outside the QEMU binary,
 that might be doable simply by moving them to a separate package,
 with a text file that QEMU reads at startup.

You seem to be describing exactly what is made possible by
-cpu custom -readconfig cpu-config-file


[1] http://libvirt.org/html/libvirt-libvirt-host.html#virConnectCompareCPU
(See the cover letter of this series)

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 11:18:18AM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 11:28:06PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 02:42:37PM -0300, Eduardo Habkost wrote:
   On Tue, Jun 23, 2015 at 07:29:15PM +0200, Andreas Färber wrote:
In summary you seem to be saying that all the years we have spent
fiddling around with those mind-boggling compat_props in QEMU were in
vain because libvirt now wants to start their own versioning system to
give users more degrees of freedom even when you can't articulate a
single concrete reason why users may want to do so.
   
   I had a similar reaction when I learned about this libvirt
   expectation/requirement I was never aware of. But we spent lots of
   effort trying to do things differently doesn't seem like a valid
   justification for design decision.
  
  Users will be hurt because they'll run untested configurations
  seems like a valid reason.
 
 I trust libvirt developers to test their CPU definitions as carefully as
 we test ours.

Maybe someone can do a write-up explaining how do requirements
and needs differ?

So far, it looks like gratituos code duplication based on a
mis-understanding, or highly unlikely theoretical what-if scenarious.

Basing our stable interfaces on such grounds might not be a good idea.


 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Andreas Färber
Am 24.06.2015 um 16:57 schrieb Michael S. Tsirkin:
 On Wed, Jun 24, 2015 at 04:35:13PM +0200, Andreas Färber wrote:
 Am 24.06.2015 um 16:19 schrieb Michael S. Tsirkin:
 On Wed, Jun 24, 2015 at 11:16:51AM -0300, Eduardo Habkost wrote:
 IMHO -M pc is not supposed to mean can break at any time.

 It means it may have new host-side requirements and may become runnable
 in your host (or require additional command-line flags to run) at any 
 time.

 That would be pretty bad. I don't think we ever had such cases in practice.

 Why is that bad or unexpected? If you install new software, it may have
 new dependencies. QEMU would be no different from other software there.
 
 That's just basic compatibility.  If I run using same flags, I expect
 compatible behaviour.
 
 How would you like it if each time you update bash, all your scripts had
 to stop working, unless you specify a specific compability flag in
 scripts, in which case you miss (some) bugfixes?
 
 Or if you found code snippets and they can't both work.  Why? Because
 snippet 1 says request version 2.4 compatibility and another says use version 
 2.5
 compatibility, so you can't use both.
 
 Each time you break command line compatibility, you
 invalidate an unknown chunk of useful documentation somewhere.

Not sure if we're talking about the same thing.

For starters, bash or qemu-system-foo may have a dependency on a new
library. sudo recently grew dependencies on some auditing syscall.
Certain machine or CPU level KVM features may require new ioctls.
Therefore new features may have new dependencies, and by default when
not in backwards-compatibility mode we want to enable such new features
and not just some ancient frozen subset - which would be comparable to
Solaris' /bin/sh being a really ancient version with new ones under new
names.

Don't understand what you mean with those two snippets.

Either way it seems a fundamental non-issue at the moment.

qemu64 apart from x2apic did not change much. I certainly didn't propose
any functional feature changes myself and only took patches once okayed
from KVM and x86 reviewers, raising the compat_props card.

However, looking beyond the artificial qemu64, a causal problem beneath
this discussion seems to be our white-listing of CPU features rather
than transparently black-listing all but implemented features.
Apart from omission bugs, there's two ways CPU features can get added,
implementation in TCG and kernel support in KVM. Only the KVM support
depends on the host, whereas the TCG support solely depends on the QEMU
version. Bug fixes would not fall into the implementation-dependent
category. And for the feature-moratorium Paolo proposed I wonder whether
we have any mechanism to assure and test that?

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Eduardo Habkost
On Wed, Jun 24, 2015 at 04:38:51PM +0200, Paolo Bonzini wrote:
[...]
 1) libvirt should _not_ change the flags that the user passes via XML
 just because it thinks that QEMU has those flags.  This makes it
 possible for libvirt to keep cpu_map.xml up-to-date without worrying
 about versioning.

That's true, and it would solve cases like people asking libvirt for
Haswell without rdand and getting QEMU running -cpu Haswell (which
has rdrand enabled) because libvirt thinks Haswell doesn't have rdrand.

 
 2) libvirt should not add/remove flags when the user specifies
 host-model (i.e. -cpu SandyBridge, not -cpu
 SandyBridge,+f16c,+rdrand,+erms,+whatever).  host-model has had so many
 bugs reported for it that I hope this could be done unconditionally even
 if it is not backwards-compatible.  Or perhaps introduce a new name and
 deprecate host-model.  I don't know.

In other words, libvirt should not assume anything about the QEMU-side
CPU models. If the user asks for Broadwell, it should use -cpu
Broadwell, if the user asks for Broadwell + fpu, it should use -cpu
Broadwell,+fpu even if it believes every CPU model since 486 has FPU
enabled.

The reason for that is that we may still introduce CPU model changes, as
long as they are not going to mean new host-side dependencies (see
commit 78a611f1936b3eac8ed78a2be2146a742a85212c for an example, where we
added f16c and rdand to Haswell and Broadwell).

 
 3) regarding enforce, there are indeed some cases where it would break:
 
 - Haswell/Broadwell CPU model after TSX removal
 
 - qemu64 with KVM
 
 - pretty much everything including qemu64 with TCG

- Everything involving KVM CPUID features, that change depending on the
  kernel version.

- Intel CPU models that had VMX enabled in older machine-types,
  and may or may not have VMX enabled depending on kernel VMX nesting
  configuration.

 
 So libvirt here could allow _now_ the user to specify enforce, either
 via XML or via qemu.conf (or via XML + a default specified via qemu.conf).

More precisely, libvirt would emulate enforce mode by checking the
filtered-features, because enforce error messages are not
machine-friendly.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 11:24:46AM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 11:34:45PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 02:11:22PM -0300, Eduardo Habkost wrote:
   Even if it is a bug fix. If it is a change that can make the VM
   unrunnable, it needs to be controlled by a separate flag, not by the
   machine-type.
  
  I agree - command line compatibility is important.  But we are supposed
  to provide that.  I am surprised that libvirt suddenly wants to avoid
  some command line flags because they are not stable. IMHO we did something
  wrong here if so. Maybe there was a valid reason for it. But then won't
  it apply to libvirt as well?
 
 Maybe we are having the same misunderstanding here: the problem is not
 compatibility/stability of existing machines, but the kind of
 (intentional) changes introduced in _new_ machines (when the -machine
 argument is changed). There are two kinds of changes introduced in new
 machines:
 
 1) Guest-side-only ABI changes: those are OK, libvirt normally ignore
them, they can't make a VM not-runnable.
 2) Changes in the host-side dependencies: those need to be more carefully
controlled by libvirt. That's where CPU features are special: all CPU
features depend KVM-side features, and enabling them by default on
new machines makes it impossible for libvirt to know/report in
advance what's necessary to make a VM runnable and to implement their
existing runnability APIs[1].

Not all that special.  e.g. virtio device offloads depend on host
networking's ability to offload them too. And hey, QEMU is part of host
too, so if you want to be pedantic, any device configuration might be
broken on an old host and fixed on a new one, so vm seems to run but
will lose data or crashes.

All this is unusual enough that no one bothers.

 Unless we guarantee that QEMU would never introduce type-(2) changes in
 new machines (which I don't think will ever happen because that means
 never changing existing CPU models in QEMU), libvirt needs to control
 CPU features individually (that's why they need -cpu custom).

Did we ever do it? I don't think so, so yes, we very likely never will.

It's as if someone wrote a wrapper around all kernel system calls
on the assumption that kernel can not guarantee kernel/userspace ABI
will never change. It can and it does.

So let's promise not to break things, and avoid a ton of copy and paste bugs.


  
  Now, if people want to update CPU models outside the QEMU binary,
  that might be doable simply by moving them to a separate package,
  with a text file that QEMU reads at startup.
 
 You seem to be describing exactly what is made possible by
 -cpu custom -readconfig cpu-config-file
 
 
 [1] http://libvirt.org/html/libvirt-libvirt-host.html#virConnectCompareCPU
 (See the cover letter of this series)

Only if you also add the config files as part of qemu, reworking
existing models to load config file from a pre-defined directory.

My whole point is to avoid useless code duplication.

 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Michael S. Tsirkin
On Wed, Jun 24, 2015 at 04:35:13PM +0200, Andreas Färber wrote:
 Am 24.06.2015 um 16:19 schrieb Michael S. Tsirkin:
  On Wed, Jun 24, 2015 at 11:16:51AM -0300, Eduardo Habkost wrote:
  IMHO -M pc is not supposed to mean can break at any time.
 
  It means it may have new host-side requirements and may become runnable
  in your host (or require additional command-line flags to run) at any 
  time.
  
  That would be pretty bad. I don't think we ever had such cases in practice.
 
 Why is that bad or unexpected? If you install new software, it may have
 new dependencies. QEMU would be no different from other software there.

That's just basic compatibility.  If I run using same flags, I expect
compatible behaviour.

How would you like it if each time you update bash, all your scripts had
to stop working, unless you specify a specific compability flag in
scripts, in which case you miss (some) bugfixes?

Or if you found code snippets and they can't both work.  Why? Because
snippet 1 says request version 2.4 compatibility and another says use version 
2.5
compatibility, so you can't use both.

Each time you break command line compatibility, you
invalidate an unknown chunk of useful documentation somewhere.


 Yesterday Eduardo said it was about having a fixed version installed and
 therein switching from a legacy machine to a newer machine. In that case
 the dependencies existed ever since installation but were not
 immediately visible to the end user due to our stability guarantees.
 
 Andreas
 
 -- 
 SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
 GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
 21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Andreas Färber
Am 24.06.2015 um 16:19 schrieb Michael S. Tsirkin:
 On Wed, Jun 24, 2015 at 11:16:51AM -0300, Eduardo Habkost wrote:
 IMHO -M pc is not supposed to mean can break at any time.

 It means it may have new host-side requirements and may become runnable
 in your host (or require additional command-line flags to run) at any time.
 
 That would be pretty bad. I don't think we ever had such cases in practice.

Why is that bad or unexpected? If you install new software, it may have
new dependencies. QEMU would be no different from other software there.

Yesterday Eduardo said it was about having a fixed version installed and
therein switching from a legacy machine to a newer machine. In that case
the dependencies existed ever since installation but were not
immediately visible to the end user due to our stability guarantees.

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-24 Thread Paolo Bonzini


On 24/06/2015 17:58, Eduardo Habkost wrote:
  3) regarding enforce, there are indeed some cases where it would break:
  
  - Haswell/Broadwell CPU model after TSX removal
  
  - qemu64 with KVM
  
  - pretty much everything including qemu64 with TCG
 
 - Everything involving KVM CPUID features, that change depending on the
   kernel version.
 
 - Intel CPU models that had VMX enabled in older machine-types,
   and may or may not have VMX enabled depending on kernel VMX nesting
   configuration.
 
  So libvirt here could allow _now_ the user to specify enforce, either
  via XML or via qemu.conf (or via XML + a default specified via qemu.conf).
 
 More precisely, libvirt would emulate enforce mode by checking the
 filtered-features, because enforce error messages are not
 machine-friendly.

Even better, because it would let libvirt special-case stuff like TSX,
VMX and SVM.

Paolo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script is provided,
  that will generate a config file that can be loaded using -readconfig, 
  based on
  the -cpu and -machine options provided in the command-line.
  
  Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
  configuration data to libvirt, but now I think it actually makes sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
 That sounds like FUD to me. Any concrete data points where QEMU does not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
 for.

What Jiri is saying that the CPUs change depending on -mmachine, not
that the ABI is broken by a given machine.

The problem here is that libvirt needs to provide CPU models whose
runnability does not depend on the machine-type. If users have a VM that
is running in a host and the VM machine-type changes, the VM should be
still runnable in that host. QEMU doesn't provide that, our CPU models
may change when we introduce new machine-types, so we are giving them a
mechanism that allows libvirt to implement the policy they need.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
   To help libvirt in the transition, a x86-cpu-model-dump script is 
   provided,
   that will generate a config file that can be loaded using -readconfig, 
   based on
   the -cpu and -machine options provided in the command-line.
   
   Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
   configuration data to libvirt, but now I think it actually makes sense.
   We already have a partial copy of CPU model definitions in libvirt
   anyway, but as QEMU changes some CPU models in some machine types (and
   libvirt does not do that) we have no real control over the guest CPU
   configuration. While what we really want is full control to enforce
   stable guest ABI.
  
  That sounds like FUD to me. Any concrete data points where QEMU does not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
  for.
 
 What Jiri is saying that the CPUs change depending on -mmachine, not
 that the ABI is broken by a given machine.
 
 The problem here is that libvirt needs to provide CPU models whose
 runnability does not depend on the machine-type. If users have a VM that
 is running in a host and the VM machine-type changes,

How does it change, and why?

 the VM should be
 still runnable in that host. QEMU doesn't provide that, our CPU models
 may change when we introduce new machine-types, so we are giving them a
 mechanism that allows libvirt to implement the policy they need.

I don't mind wrt CPU specifically, but we absolutely do change guest ABI
in many ways when we change machine types.

 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
   To help libvirt in the transition, a x86-cpu-model-dump script is 
   provided,
   that will generate a config file that can be loaded using -readconfig, 
   based on
   the -cpu and -machine options provided in the command-line.
   
   Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
   configuration data to libvirt, but now I think it actually makes sense.
   We already have a partial copy of CPU model definitions in libvirt
   anyway, but as QEMU changes some CPU models in some machine types (and
   libvirt does not do that) we have no real control over the guest CPU
   configuration. While what we really want is full control to enforce
   stable guest ABI.
  
  That sounds like FUD to me. Any concrete data points where QEMU does not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
  for.
 
 What Jiri is saying that the CPUs change depending on -mmachine, not
 that the ABI is broken by a given machine.
 
 The problem here is that libvirt needs to provide CPU models whose
 runnability does not depend on the machine-type. If users have a VM that
 is running in a host and the VM machine-type changes, the VM should be
 still runnable in that host. QEMU doesn't provide that, our CPU models
 may change when we introduce new machine-types, so we are giving them a
 mechanism that allows libvirt to implement the policy they need.

Expanding on that, but tieing the CPU model to the machine type, QEMU
has in turn effectively tied the machine type to the host hardware.
eg, switching to a newer machine type, may then prevent the guest
from being able to launch on the hardware that it was previously
able to run on, due to some new requirement of the CPU model associated
with the machine type.

Libvirt wants the CPU models to be independant of the machine type,
so in general only the CPU model is dependant on hardware capabilities
and machine type is isolated from hardware.

Libvirt still intends to do versioning of the CPU models, but the
versioning will be separate from the versioning of the machine types,
and will be handled by libvirt itself.

This also allows us to get  further towards our goal which is to have a
consistent representation of CPU models across all libvirt hypervisors.
eg the same libvirt CPU model and versions can be made consistent across
kvm, xen, vmware, etc, as they're not longer changing behind our back
based on the qemu machine type.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script is provided,
 that will generate a config file that can be loaded using -readconfig, based 
 on
 the -cpu and -machine options provided in the command-line.
 
 Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
 configuration data to libvirt, but now I think it actually makes sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

That sounds like FUD to me. Any concrete data points where QEMU does not
have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
for.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
   Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
To help libvirt in the transition, a x86-cpu-model-dump script is 
provided,
that will generate a config file that can be loaded using 
-readconfig, based on
the -cpu and -machine options provided in the command-line.
   
Thanks Eduardo, I never was a big fan of moving (or copying) all 
the CPU
configuration data to libvirt, but now I think it actually makes 
sense.
We already have a partial copy of CPU model definitions in libvirt
anyway, but as QEMU changes some CPU models in some machine types 
(and
libvirt does not do that) we have no real control over the guest CPU
configuration. While what we really want is full control to enforce
stable guest ABI.
   
That sounds like FUD to me. Any concrete data points where QEMU does 
not
have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
machines
for.
   
What Jiri is saying that the CPUs change depending on -mmachine, not
that the ABI is broken by a given machine.
   
The problem here is that libvirt needs to provide CPU models whose
runnability does not depend on the machine-type. If users have a VM 
that
is running in a host and the VM machine-type changes,
   
How does it change, and why?

Sometimes we add features to a CPU model because they were not emulated 
by KVM
and now they are. Sometimes we remove or add features or change other 
fields
because we are fixing previous mistakes. Recently we we were going to 
remove
features from models because of an Intel CPU errata, but then decided 
to create
a new CPU model name instead.

See some examples at the end of this message.

   
the VM should be
still runnable in that host. QEMU doesn't provide that, our CPU models
may change when we introduce new machine-types, so we are giving them 
a
mechanism that allows libvirt to implement the policy they need.
   
I don't mind wrt CPU specifically, but we absolutely do change guest 
ABI
in many ways when we change machine types.

All the other ABI changes we introduce in QEMU don't affect runnability 
of the
VM in a given host, that's the problem we are trying to address here. 
ABI
changes are expected when changing to a new machine, runnability changes
aren't.


Examples of commits changing CPU models:
   [snip]
   
   I've always advocated remaining backwards-compatible and only making CPU
   model changes for new machines. You among others felt that was not
   always necessary, and now you're using the lack thereof as an argument
   to stop using QEMU's CPU models at all? That sounds convoluted...
  
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

Huh, I never said we wanted to bring back bugs. This is about allowing
libvirt to fix the CPU bugs in a way that is independant of the machine
types and portable across hypervisors we deal with. We're absolutely
still going to fix CPU model bugs and ensure stable guest ABI.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 18:32 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script is 
 provided,
 that will generate a config file that can be loaded using -readconfig, 
 based on
 the -cpu and -machine options provided in the command-line.

 Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
 configuration data to libvirt, but now I think it actually makes sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

 That sounds like FUD to me. Any concrete data points where QEMU does not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
 for.

 What Jiri is saying that the CPUs change depending on -mmachine, not
 that the ABI is broken by a given machine.

 The problem here is that libvirt needs to provide CPU models whose
 runnability does not depend on the machine-type. If users have a VM that
 is running in a host and the VM machine-type changes,

 How does it change, and why?

 Sometimes we add features to a CPU model because they were not emulated by 
 KVM
 and now they are. Sometimes we remove or add features or change other fields
 because we are fixing previous mistakes. Recently we we were going to remove
 features from models because of an Intel CPU errata, but then decided to 
 create
 a new CPU model name instead.

 See some examples at the end of this message.


 the VM should be
 still runnable in that host. QEMU doesn't provide that, our CPU models
 may change when we introduce new machine-types, so we are giving them a
 mechanism that allows libvirt to implement the policy they need.

 I don't mind wrt CPU specifically, but we absolutely do change guest ABI
 in many ways when we change machine types.

 All the other ABI changes we introduce in QEMU don't affect runnability of 
 the
 VM in a given host, that's the problem we are trying to address here. ABI
 changes are expected when changing to a new machine, runnability changes
 aren't.


 Examples of commits changing CPU models:
 [snip]

 I've always advocated remaining backwards-compatible and only making CPU
 model changes for new machines. You among others felt that was not
 always necessary, and now you're using the lack thereof as an argument
 to stop using QEMU's CPU models at all? That sounds convoluted...

 
 Uh? I don't remember anybody suggesting changing CPU models on existing
 machines. We always tried to keep existing machines compatible.

Yes, we try in general. And in a few cases I was overruled, possibly
related to TCG feature filtering or something. Thought that was the
problem here - apparently not. Explanations seem to be the culprit here!

 BTW your list does not answer my question. You would need examples where
 a CPU model changes between machines, and I am not aware of any example
 beyond the intentional -x.y variations. There are differences between
 KVM and TCG though, did you mean that? i440fx and q35 should be
 identical and isa-pc, too, and none anyway. None of this has anything to
 do with the host CPU.
 
 We are talking about the -x.y variations (that, yes, are intentional).
 But the fact that CPU features change (even the intentional ones in the
 -x.y machine variations) affect runnability of VMs (because enabling new
 CPU features in KVM require it to be supported by the host kernel code
 and by the host CPU).
 
 I was not thinking about the KVM and TCG differences, but this may also
 help libvirt deal with the KVM and TCG differences if necessary.
 
 I don't know what you mean by i440fx and q35 should be identical
 above.

In this thread there was a claim that CPU models varied between machine
types. I am saying that there should be no CPU model differences between
pc-i440fx-2.3 and pc-q35-2.3 etc. Thus the CPU model is not tied to one
machine, but to the version of QEMU, with -x.y matching the
corresponding release. No news to you, I would hope?

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

 I didn't take the time to count them, but I bet most of the commits I
 listed on my previous e-mail message are not bug fixes, but new
 features.

 Huh? Of course the latest machine model get new features. The point is
 that the previous ones don't and that's what we are providing them for -
 libvirt is expected to choose one machine and the contract with QEMU is
 that for that machine the CPU does *not* grow new features, and we're
 going at great lengths to achieve that. So this thread feels more and
 more weird...
 
 We are not talking about changes to existing machines. We are talking
 about having changes introduced in new machines (the one we did on
 purpose) affecting the runnability of the VM.

You are talking abstract!


Example 1:

Point A: Machine pc-i440fx-2.3 exists

Runs or runs not.

Point B: Machine pc-i440fx-2.3 still exists

Still runs or runs not due to guest ABI stability rules.


Example 2:

Point A: pc-i440fx-2.4 does not exist in 2.3

Does not run becomes it doesn't exist.

Point B: New pc-i440fx-2.4

Runs or does not run, and if so has more features than pc-i440fx-2.3.

There is no runnability problem - either it runs or it doesn't, but
there's no change over time.

This is what the machine -x.y versioning is all about.

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 07:10:11PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:53 schrieb Daniel P. Berrange:
  On Tue, Jun 23, 2015 at 06:40:19PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 18:25 schrieb Daniel P. Berrange:
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  This coupling is by design and we expect all KVM/QEMU users to adhere to
  it, including those that use the libvirt tool (which I assume is going
  to be the majority of KVM users). Either you want a certain
  backwards-compatible machine and CPU, or you want the latest and
  greatest - why in the world mix and match?!
  
  As mentioned, changes/fixes to the CPU model can affect the ability to
  launch the guest on a particular host, so we need the ability to control
  when those CPU changes are activated for a guest, separately from the
  machine type.
 
 Why? Today's libvirt with QEMU 2.3 resolves pc machine to
 pc-i440fx-2.3 and the guest XML stays that way. When we add new
 features for 2.4, 2.3 is guaranteed to stay compatible. Any change would
 involve the libvirt user actively switching from pc-i440fx-2.3 to a
 different machine such as upcoming pc-i440fx-2.4. Why do you need to
 change the CPU separately? Why would a user want to run 2.2's CPU with a
 2.3 machine? Or a 2.3 machine with a 2.4 CPU? That's nonsense. If you
 want to tweak features, you already have command line options available
 to do so on the basis of what the selected machine provides.

Because pure guest-side ABI changes are different from changes that also
have additional host-side requirements, so we want to untie both things.

About being able to tweak features today, that's true: we have
command-line options for most stuff and that's _almost_ enough for what
libvirt needs. What's missing is something to avoid silently getting new
features that libvirt aren't aware of (but may make the VM unrunnable).
The purpose of -cpu custom is to ensure no new host side feature
requirement will be introduced silently, even if choosing a different
machine.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 02:24:16PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 07:10:11PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 18:53 schrieb Daniel P. Berrange:
   On Tue, Jun 23, 2015 at 06:40:19PM +0200, Andreas Färber wrote:
   Am 23.06.2015 um 18:25 schrieb Daniel P. Berrange:
   Whether QEMU changed the CPU for existing machines, or only for new
   machines is actually not the core problem. Even if we only changed
   the CPU in new machines that would still be an unsatisfactory situation
   because we want to be able to be able to access different versions of
   the CPU without the machine type changing, and access different versions
   of the machine type, without the CPU changing. IOW it is the fact that 
   the
   changes in CPU are tied to changes in machine type that is the core
   problem.
  
   This coupling is by design and we expect all KVM/QEMU users to adhere to
   it, including those that use the libvirt tool (which I assume is going
   to be the majority of KVM users). Either you want a certain
   backwards-compatible machine and CPU, or you want the latest and
   greatest - why in the world mix and match?!
   
   As mentioned, changes/fixes to the CPU model can affect the ability to
   launch the guest on a particular host, so we need the ability to control
   when those CPU changes are activated for a guest, separately from the
   machine type.
  
  Why? Today's libvirt with QEMU 2.3 resolves pc machine to
  pc-i440fx-2.3 and the guest XML stays that way. When we add new
  features for 2.4, 2.3 is guaranteed to stay compatible. Any change would
  involve the libvirt user actively switching from pc-i440fx-2.3 to a
  different machine such as upcoming pc-i440fx-2.4. Why do you need to
  change the CPU separately? Why would a user want to run 2.2's CPU with a
  2.3 machine? Or a 2.3 machine with a 2.4 CPU? That's nonsense. If you
  want to tweak features, you already have command line options available
  to do so on the basis of what the selected machine provides.
 
 Because pure guest-side ABI changes are different from changes that also
 have additional host-side requirements, so we want to untie both things.
 
 About being able to tweak features today, that's true: we have
 command-line options for most stuff and that's _almost_ enough for what
 libvirt needs. What's missing is something to avoid silently getting new
 features that libvirt aren't aware of (but may make the VM unrunnable).
 The purpose of -cpu custom is to ensure no new host side feature
 requirement will be introduced silently, even if choosing a different
 machine.

It also has a positive impact on maintenance. By allowing libvirt to
control the CPU definitions, in cases where there is a need to push
out a CPU model bugfix, libvirt will often be able to make the update
available to users via a new CPU model version, without them having
to do a lock-step upgrade to QEMU at the same time.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
   Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
To help libvirt in the transition, a x86-cpu-model-dump script is 
provided,
that will generate a config file that can be loaded using -readconfig, 
based on
the -cpu and -machine options provided in the command-line.

Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
configuration data to libvirt, but now I think it actually makes sense.
We already have a partial copy of CPU model definitions in libvirt
anyway, but as QEMU changes some CPU models in some machine types (and
libvirt does not do that) we have no real control over the guest CPU
configuration. While what we really want is full control to enforce
stable guest ABI.
   
   That sounds like FUD to me. Any concrete data points where QEMU does not
   have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
   for.
  
  What Jiri is saying that the CPUs change depending on -mmachine, not
  that the ABI is broken by a given machine.
  
  The problem here is that libvirt needs to provide CPU models whose
  runnability does not depend on the machine-type. If users have a VM that
  is running in a host and the VM machine-type changes,
 
 How does it change, and why?

Sometimes we add features to a CPU model because they were not emulated by KVM
and now they are. Sometimes we remove or add features or change other fields
because we are fixing previous mistakes. Recently we we were going to remove
features from models because of an Intel CPU errata, but then decided to create
a new CPU model name instead.

See some examples at the end of this message.

 
  the VM should be
  still runnable in that host. QEMU doesn't provide that, our CPU models
  may change when we introduce new machine-types, so we are giving them a
  mechanism that allows libvirt to implement the policy they need.
 
 I don't mind wrt CPU specifically, but we absolutely do change guest ABI
 in many ways when we change machine types.

All the other ABI changes we introduce in QEMU don't affect runnability of the
VM in a given host, that's the problem we are trying to address here. ABI
changes are expected when changing to a new machine, runnability changes
aren't.


Examples of commits changing CPU models:

commit 726a8ff68677d8d5fba17eb0ffb85076bfb598dc
Author: Eduardo Habkost ehabk...@redhat.com
Date:   Fri Apr 10 14:45:00 2015 -0300

target-i386: Remove AMD feature flag aliases from CPU model table

When CPU vendor is AMD, the AMD feature alias bits on
CPUID[0x8001].EDX are already automatically copied from CPUID[1].EDX
on x86_cpu_realizefn(). When CPU vendor is Intel, those bits are
reserved and should be zero. On either case, those bits shouldn't be set
in the CPU model table.

Reviewed-by: Igor Mammedov imamm...@redhat.com
Signed-off-by: Eduardo Habkost ehabk...@redhat.com

commit 13704e4c455770d500d6b87b117e32f0d01252c9
Author: Eduardo Habkost ehabk...@redhat.com
Date:   Thu Jan 22 17:22:54 2015 -0200

target-i386: Disable HLE and RTM on Haswell  Broadwell

All Haswell CPUs and some Broadwell CPUs were updated by Intel to have
the HLE and RTM features disabled. This will prevent
-cpu Haswell,enforce and -cpu Broadwell,enforce from running out of
the box on those CPUs.

Disable those features by default on Broadwell and Haswell CPU models,
starting on pc-*-2.3. Users who want to use those features can enable
them explicitly on the command-line.

Signed-off-by: Eduardo Habkost ehabk...@redhat.com
Signed-off-by: Paolo Bonzini pbonz...@redhat.com

commit 78a611f1936b3eac8ed78a2be2146a742a85212c
Author: Paolo Bonzini pbonz...@redhat.com
Date:   Fri Dec 5 10:52:46 2014 +0100

target-i386: add f16c and rdrand to Haswell and Broadwell

Both were added in Ivy Bridge (for which we do not have a CPU model
yet!).

Reviewed-by: Eduardo Habkost ehabk...@redhat.com
Signed-off-by: Paolo Bonzini pbonz...@redhat.com

commit b3a4f0b1a072a467d003755ca0e55c5be38387cb
Author: Paolo Bonzini pbonz...@redhat.com
Date:   Wed Dec 10 14:12:41 2014 -0200

target-i386: add VME to all CPUs

vm86 mode extensions date back to the 486.  All models should have
them.

Signed-off-by: Paolo Bonzini pbonz...@redhat.com
Signed-off-by: Eduardo Habkost ehabk...@redhat.com
Signed-off-by: Paolo Bonzini pbonz...@redhat.com

commit 0bb0b2d2fe7f645ddaf1f0ff40ac669c9feb4aa1
Author: Paolo Bonzini pbonz...@redhat.com
Date:   Mon Nov 24 15:54:43 2014 +0100

target-i386: add feature flags for CPUID[EAX=0xd,ECX=1]

These represent xsave-related capabilities of the processor, and KVM may
or may not support them.

Add feature 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 04:51:00PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
   Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
To help libvirt in the transition, a x86-cpu-model-dump script is 
provided,
that will generate a config file that can be loaded using -readconfig, 
based on
the -cpu and -machine options provided in the command-line.

Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
configuration data to libvirt, but now I think it actually makes sense.
We already have a partial copy of CPU model definitions in libvirt
anyway, but as QEMU changes some CPU models in some machine types (and
libvirt does not do that) we have no real control over the guest CPU
configuration. While what we really want is full control to enforce
stable guest ABI.
   
   That sounds like FUD to me. Any concrete data points where QEMU does not
   have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
   for.
  
  What Jiri is saying that the CPUs change depending on -mmachine, not
  that the ABI is broken by a given machine.
  
  The problem here is that libvirt needs to provide CPU models whose
  runnability does not depend on the machine-type. If users have a VM that
  is running in a host and the VM machine-type changes, the VM should be
  still runnable in that host. QEMU doesn't provide that, our CPU models
  may change when we introduce new machine-types, so we are giving them a
  mechanism that allows libvirt to implement the policy they need.
 
 Expanding on that, but tieing the CPU model to the machine type, QEMU
 has in turn effectively tied the machine type to the host hardware.
 eg, switching to a newer machine type, may then prevent the guest
 from being able to launch on the hardware that it was previously
 able to run on, due to some new requirement of the CPU model associated
 with the machine type.

So why not keep machine type stable?

 Libvirt wants the CPU models to be independant of the machine type,
 so in general only the CPU model is dependant on hardware capabilities
 and machine type is isolated from hardware.
 
 Libvirt still intends to do versioning of the CPU models, but the
 versioning will be separate from the versioning of the machine types,
 and will be handled by libvirt itself.
 
 This also allows us to get  further towards our goal which is to have a
 consistent representation of CPU models across all libvirt hypervisors.
 eg the same libvirt CPU model and versions can be made consistent across
 kvm, xen, vmware, etc, as they're not longer changing behind our back
 based on the qemu machine type.
 
 Regards,
 Daniel
 -- 
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 18:42 schrieb Daniel P. Berrange:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script is 
 provided,
 that will generate a config file that can be loaded using 
 -readconfig, based on
 the -cpu and -machine options provided in the command-line.

 Thanks Eduardo, I never was a big fan of moving (or copying) all the 
 CPU
 configuration data to libvirt, but now I think it actually makes 
 sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

 That sounds like FUD to me. Any concrete data points where QEMU does 
 not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
 machines
 for.

 What Jiri is saying that the CPUs change depending on -mmachine, not
 that the ABI is broken by a given machine.

 The problem here is that libvirt needs to provide CPU models whose
 runnability does not depend on the machine-type. If users have a VM that
 is running in a host and the VM machine-type changes,

 How does it change, and why?

 Sometimes we add features to a CPU model because they were not emulated 
 by KVM
 and now they are. Sometimes we remove or add features or change other 
 fields
 because we are fixing previous mistakes. Recently we we were going to 
 remove
 features from models because of an Intel CPU errata, but then decided to 
 create
 a new CPU model name instead.

 See some examples at the end of this message.


 the VM should be
 still runnable in that host. QEMU doesn't provide that, our CPU models
 may change when we introduce new machine-types, so we are giving them a
 mechanism that allows libvirt to implement the policy they need.

 I don't mind wrt CPU specifically, but we absolutely do change guest ABI
 in many ways when we change machine types.

 All the other ABI changes we introduce in QEMU don't affect runnability 
 of the
 VM in a given host, that's the problem we are trying to address here. ABI
 changes are expected when changing to a new machine, runnability changes
 aren't.


 Examples of commits changing CPU models:
 [snip]

 I've always advocated remaining backwards-compatible and only making CPU
 model changes for new machines. You among others felt that was not
 always necessary, and now you're using the lack thereof as an argument
 to stop using QEMU's CPU models at all? That sounds convoluted...

 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?
 
 Huh, I never said we wanted to bring back bugs. This is about allowing
 libvirt to fix the CPU bugs in a way that is independant of the machine
 types and portable across hypervisors we deal with. We're absolutely
 still going to fix CPU model bugs and ensure stable guest ABI.

No, that's contradictory! Through the -x.y machines we leave bugs in the
old models *exactly* to assure a stable guest ABI. Fixes are only be
applied to new machines, thus I'm pointing out that you should not use a
new CPU model with an old machine type.

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 06:47:16PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:42 schrieb Daniel P. Berrange:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
  I've always advocated remaining backwards-compatible and only making CPU
  model changes for new machines. You among others felt that was not
  always necessary, and now you're using the lack thereof as an argument
  to stop using QEMU's CPU models at all? That sounds convoluted...
 
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
  
  Huh, I never said we wanted to bring back bugs. This is about allowing
  libvirt to fix the CPU bugs in a way that is independant of the machine
  types and portable across hypervisors we deal with. We're absolutely
  still going to fix CPU model bugs and ensure stable guest ABI.
 
 No, that's contradictory! Through the -x.y machines we leave bugs in the
 old models *exactly* to assure a stable guest ABI. Fixes are only be
 applied to new machines, thus I'm pointing out that you should not use a
 new CPU model with an old machine type.

I'm not saying that libvirt would ever allow a silent guest ABI change.
Given a libvirt XML config, the guest ABI will never change without an
explicit action on the part of the app/user to change the XML.

This is all about dealing with the case where the app / user conciously
needs/wants to opt-in to a guest ABI change for the guest. eg they wish
to make use of some bug fix or feature improvement in the new machine
type, but they do *not* wish to have the CPU model changed, because
of some CPU model change that is incompatible with their hosts' CPUs.
Conversely, they may wish to get access to a new CPU model, but not
wish to have the rest of the guest ABI change. In both cases the user
is explicitly opt-ing in the ABI change with knowledge about what
this might mean for the guest OS. Currently we are tieing users
hands by forcing CPU and machine types to change in lockstep.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 07:29:15PM +0200, Andreas Färber wrote:
 In summary you seem to be saying that all the years we have spent
 fiddling around with those mind-boggling compat_props in QEMU were in
 vain because libvirt now wants to start their own versioning system to
 give users more degrees of freedom even when you can't articulate a
 single concrete reason why users may want to do so.

I had a similar reaction when I learned about this libvirt
expectation/requirement I was never aware of. But we spent lots of
effort trying to do things differently doesn't seem like a valid
justification for design decision.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 18:25 schrieb Daniel P. Berrange:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

This coupling is by design and we expect all KVM/QEMU users to adhere to
it, including those that use the libvirt tool (which I assume is going
to be the majority of KVM users). Either you want a certain
backwards-compatible machine and CPU, or you want the latest and
greatest - why in the world mix and match?!

Would a qemu64-2.3 model help here that pc*-2.3 could use? I believe
that's been proposed in the past. I don't oppose the idea of a
fully-custom CPU, but this blatant attempt of ignoring QEMU's CPU
versioning by libvirt worries me.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 06:47:16PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:42 schrieb Daniel P. Berrange:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script is 
  provided,
  that will generate a config file that can be loaded using 
  -readconfig, based on
  the -cpu and -machine options provided in the command-line.
 
  Thanks Eduardo, I never was a big fan of moving (or copying) all 
  the CPU
  configuration data to libvirt, but now I think it actually makes 
  sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types 
  (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
  That sounds like FUD to me. Any concrete data points where QEMU does 
  not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
  machines
  for.
 
  What Jiri is saying that the CPUs change depending on -mmachine, not
  that the ABI is broken by a given machine.
 
  The problem here is that libvirt needs to provide CPU models whose
  runnability does not depend on the machine-type. If users have a VM 
  that
  is running in a host and the VM machine-type changes,
 
  How does it change, and why?
 
  Sometimes we add features to a CPU model because they were not emulated 
  by KVM
  and now they are. Sometimes we remove or add features or change other 
  fields
  because we are fixing previous mistakes. Recently we we were going to 
  remove
  features from models because of an Intel CPU errata, but then decided 
  to create
  a new CPU model name instead.
 
  See some examples at the end of this message.
 
 
  the VM should be
  still runnable in that host. QEMU doesn't provide that, our CPU models
  may change when we introduce new machine-types, so we are giving them 
  a
  mechanism that allows libvirt to implement the policy they need.
 
  I don't mind wrt CPU specifically, but we absolutely do change guest 
  ABI
  in many ways when we change machine types.
 
  All the other ABI changes we introduce in QEMU don't affect runnability 
  of the
  VM in a given host, that's the problem we are trying to address here. 
  ABI
  changes are expected when changing to a new machine, runnability changes
  aren't.
 
 
  Examples of commits changing CPU models:
  [snip]
 
  I've always advocated remaining backwards-compatible and only making CPU
  model changes for new machines. You among others felt that was not
  always necessary, and now you're using the lack thereof as an argument
  to stop using QEMU's CPU models at all? That sounds convoluted...
 
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
  
  Huh, I never said we wanted to bring back bugs. This is about allowing
  libvirt to fix the CPU bugs in a way that is independant of the machine
  types and portable across hypervisors we deal with. We're absolutely
  still going to fix CPU model bugs and ensure stable guest ABI.
 
 No, that's contradictory! Through the -x.y machines we leave bugs in the
 old models *exactly* to assure a stable guest ABI. Fixes are only be
 applied to new machines, thus I'm pointing out that you should not use a
 new CPU model with an old machine type.

They don't need to use a new model with an old machine-type (although I
don't see a reason to prevent that). They need to be able to change the
machine-type to a new one without getting any changes that would make
the VM not runnable in the same host. Even if it is a bug fix. If it is
a change that can make the VM unrunnable, it needs to be controlled by a
separate flag, not by the machine-type.

-- 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 07:18:06PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that 
  the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
 
  I didn't take the time to count them, but I bet most of the commits I
  listed on my previous e-mail message are not bug fixes, but new
  features.
 
  Huh? Of course the latest machine model get new features. The point is
  that the previous ones don't and that's what we are providing them for -
  libvirt is expected to choose one machine and the contract with QEMU is
  that for that machine the CPU does *not* grow new features, and we're
  going at great lengths to achieve that. So this thread feels more and
  more weird...
  
  We are not talking about changes to existing machines. We are talking
  about having changes introduced in new machines (the one we did on
  purpose) affecting the runnability of the VM.
 
 You are talking abstract!

I am just talking about a different problem, and I don't know if you are
purposely trying to ignore it, or are just denying that it is a problem.

 
 
 Example 1:
 
 Point A: Machine pc-i440fx-2.3 exists
 
 Runs or runs not.
 
 Point B: Machine pc-i440fx-2.3 still exists
 
 Still runs or runs not due to guest ABI stability rules.

If you didn't change the machine name, this is not the problem we are
talking about.

 
 
 Example 2:
 
 Point A: pc-i440fx-2.4 does not exist in 2.3
 
 Does not run becomes it doesn't exist.
 
 Point B: New pc-i440fx-2.4
 
 Runs or does not run, and if so has more features than pc-i440fx-2.3.

If you didn't change the machine name, this is not the problem we are
talking about.

 
 There is no runnability problem - either it runs or it doesn't, but
 there's no change over time.
 
 This is what the machine -x.y versioning is all about.

Let's try a concrete example:

* User is running a kernel that can't emulate x2apic
* User is running pc-i440fx-1.7
* User wants the gigabyte alignment change implemented by commit
  bb43d3839c29b17a2f5c122114cd4ca978065a18
* User changes machine to pc-i440fx-2.0
* x2apic is now enabled by default in all CPU models
* VM with the same configuration (just the machine change) is not
  runnable anymore in the same host

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 19:27 schrieb Daniel P. Berrange:
 On Tue, Jun 23, 2015 at 07:18:06PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that 
 the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

 I didn't take the time to count them, but I bet most of the commits I
 listed on my previous e-mail message are not bug fixes, but new
 features.

 Huh? Of course the latest machine model get new features. The point is
 that the previous ones don't and that's what we are providing them for -
 libvirt is expected to choose one machine and the contract with QEMU is
 that for that machine the CPU does *not* grow new features, and we're
 going at great lengths to achieve that. So this thread feels more and
 more weird...

 We are not talking about changes to existing machines. We are talking
 about having changes introduced in new machines (the one we did on
 purpose) affecting the runnability of the VM.

 You are talking abstract!


 Example 1:

 Point A: Machine pc-i440fx-2.3 exists

 Runs or runs not.

 Point B: Machine pc-i440fx-2.3 still exists

 Still runs or runs not due to guest ABI stability rules.


 Example 2:

 Point A: pc-i440fx-2.4 does not exist in 2.3

 Does not run becomes it doesn't exist.

 Point B: New pc-i440fx-2.4

 Runs or does not run, and if so has more features than pc-i440fx-2.3.

 There is no runnability problem - either it runs or it doesn't, but
 there's no change over time.

 This is what the machine -x.y versioning is all about.
 
 Consider a host currently running QEMU 2.3 with machine type
 pc-i440fx-2.3 used with SandyBridge.
 
 Now consider the def of SandyBridge was buggy and so in QEMU
 2.4 we add the missing CPU feature flag 'wizz', and only
 enable that new feature flag with pc-i440fx-2.4
 
 Now consider there was a bug in the virtio-scsi driver that
 we also fixed in QEMU 2.4 and thus pc-i440fx-2.4 includes
 that fix.
 
 Updating from pc-i440fx-2.3 to pc-i440fx-2.4 has a dependancy
 on the host CPU including the 'wizz' flag that was added. This
 new CPU feature can prevent the user from using the new machine
 type to get the virtio-scsi bug fix.

Thanks for this example! :)

Only if you use -cpu ...,enforce, no?

The KVM feature filtering should take care of dropping features that are
not available otherwise.

So we seem to be getting to the interesting case of the same machine
(different from what was said previously!) but different hosts.

The QOM property gives you insights into which feature bits are set for
the machine for the model (and for s390x I saw QMP extensions to the
same effect, I thought). That way you could discover features to
disable. However you'll only ever know which ones work once you've tried
it once, right?

I'm pretty sure that we've had discussions with Anthony and Avi on the
same topic ages ago. Alex also made the -cpu best proposal long ago.
In general, if you want to run on a group of hosts, then you need to
figure out a common denominator - a CPU model name and optionally
features to enable/disable.

If that is the whole problem here, then why not just add a global flag
to only enable explicitly requested KVM features? All other features
should not depend on the host, and the whole discussion about -x.y seems
like a distraction.

Regards,
Andreas

 Separately versioning CPU models still lets us preserve a stable
 guest ABI by default, but allows more flexibility when choosing
 to opt out of the ABI for a particular upgrade.
 
 Regards,
 Daniel

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script is 
  provided,
  that will generate a config file that can be loaded using -readconfig, 
  based on
  the -cpu and -machine options provided in the command-line.
 
  Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
  configuration data to libvirt, but now I think it actually makes sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
  That sounds like FUD to me. Any concrete data points where QEMU does not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
  for.
 
  What Jiri is saying that the CPUs change depending on -mmachine, not
  that the ABI is broken by a given machine.
 
  The problem here is that libvirt needs to provide CPU models whose
  runnability does not depend on the machine-type. If users have a VM that
  is running in a host and the VM machine-type changes,
 
  How does it change, and why?
  
  Sometimes we add features to a CPU model because they were not emulated by 
  KVM
  and now they are. Sometimes we remove or add features or change other fields
  because we are fixing previous mistakes. Recently we we were going to remove
  features from models because of an Intel CPU errata, but then decided to 
  create
  a new CPU model name instead.
  
  See some examples at the end of this message.
  
 
  the VM should be
  still runnable in that host. QEMU doesn't provide that, our CPU models
  may change when we introduce new machine-types, so we are giving them a
  mechanism that allows libvirt to implement the policy they need.
 
  I don't mind wrt CPU specifically, but we absolutely do change guest ABI
  in many ways when we change machine types.
  
  All the other ABI changes we introduce in QEMU don't affect runnability of 
  the
  VM in a given host, that's the problem we are trying to address here. ABI
  changes are expected when changing to a new machine, runnability changes
  aren't.
  
  
  Examples of commits changing CPU models:
 [snip]
 
 I've always advocated remaining backwards-compatible and only making CPU
 model changes for new machines. You among others felt that was not
 always necessary, and now you're using the lack thereof as an argument
 to stop using QEMU's CPU models at all? That sounds convoluted...

Whether QEMU changed the CPU for existing machines, or only for new
machines is actually not the core problem. Even if we only changed
the CPU in new machines that would still be an unsatisfactory situation
because we want to be able to be able to access different versions of
the CPU without the machine type changing, and access different versions
of the machine type, without the CPU changing. IOW it is the fact that the
changes in CPU are tied to changes in machine type that is the core
problem.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 05:00:54PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 05:56:35PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 04:51:00PM +0100, Daniel P. Berrange wrote:
   On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script is 
  provided,
  that will generate a config file that can be loaded using 
  -readconfig, based on
  the -cpu and -machine options provided in the command-line.
  
  Thanks Eduardo, I never was a big fan of moving (or copying) all 
  the CPU
  configuration data to libvirt, but now I think it actually makes 
  sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types 
  (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
 That sounds like FUD to me. Any concrete data points where QEMU does 
 not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
 machines
 for.

What Jiri is saying that the CPUs change depending on -mmachine, not
that the ABI is broken by a given machine.

The problem here is that libvirt needs to provide CPU models whose
runnability does not depend on the machine-type. If users have a VM that
is running in a host and the VM machine-type changes, the VM should be
still runnable in that host. QEMU doesn't provide that, our CPU models
may change when we introduce new machine-types, so we are giving them a
mechanism that allows libvirt to implement the policy they need.
   
   Expanding on that, but tieing the CPU model to the machine type, QEMU
   has in turn effectively tied the machine type to the host hardware.
   eg, switching to a newer machine type, may then prevent the guest
   from being able to launch on the hardware that it was previously
   able to run on, due to some new requirement of the CPU model associated
   with the machine type.
  
  So why not keep machine type stable?
 
 There are many reasons to choose a particular machine type - for
 example, to achieve migration compat between hosts with different
 QEMU versions,

This might make you use an old machine type.
It will never make you use a newer machine type
so you will never run into problems.

 or to enable access to some performance or bug
 fix in the machine type in question.

Performance/bugfixes is exactly why we change these though.

 Users / apps need to be free
 to make those decisions, without being restricted by changes in the
 CPU model which may affect what hardware the machine type can be
 used on. The current use of machine types for CPU model versioning
 is placing users between a rock  hard place, giving them impossible
 decisions about which bad behaviour/bug they're willing to accept.
 
 Regards,
 Daniel
 -- 
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
   Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
To help libvirt in the transition, a x86-cpu-model-dump script is 
provided,
that will generate a config file that can be loaded using 
-readconfig, based on
the -cpu and -machine options provided in the command-line.
   
Thanks Eduardo, I never was a big fan of moving (or copying) all 
the CPU
configuration data to libvirt, but now I think it actually makes 
sense.
We already have a partial copy of CPU model definitions in libvirt
anyway, but as QEMU changes some CPU models in some machine types 
(and
libvirt does not do that) we have no real control over the guest CPU
configuration. While what we really want is full control to enforce
stable guest ABI.
   
That sounds like FUD to me. Any concrete data points where QEMU does 
not
have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
machines
for.
   
What Jiri is saying that the CPUs change depending on -mmachine, not
that the ABI is broken by a given machine.
   
The problem here is that libvirt needs to provide CPU models whose
runnability does not depend on the machine-type. If users have a VM 
that
is running in a host and the VM machine-type changes,
   
How does it change, and why?

Sometimes we add features to a CPU model because they were not emulated 
by KVM
and now they are. Sometimes we remove or add features or change other 
fields
because we are fixing previous mistakes. Recently we we were going to 
remove
features from models because of an Intel CPU errata, but then decided 
to create
a new CPU model name instead.

See some examples at the end of this message.

   
the VM should be
still runnable in that host. QEMU doesn't provide that, our CPU models
may change when we introduce new machine-types, so we are giving them 
a
mechanism that allows libvirt to implement the policy they need.
   
I don't mind wrt CPU specifically, but we absolutely do change guest 
ABI
in many ways when we change machine types.

All the other ABI changes we introduce in QEMU don't affect runnability 
of the
VM in a given host, that's the problem we are trying to address here. 
ABI
changes are expected when changing to a new machine, runnability changes
aren't.


Examples of commits changing CPU models:
   [snip]
   
   I've always advocated remaining backwards-compatible and only making CPU
   model changes for new machines. You among others felt that was not
   always necessary, and now you're using the lack thereof as an argument
   to stop using QEMU's CPU models at all? That sounds convoluted...
  
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
  
  Regards,
  Daniel
 
 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

I didn't take the time to count them, but I bet most of the commits I
listed on my previous e-mail message are not bug fixes, but new
features.

Also, it doesn't matter if the change is a bug fix or a new feature: if
it affects runnability of the VM, it has more consequences than a simple
guest-side ABI change, and libvirt can't tie it to the machine-type so
it needs another flag to enable it.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?
 
 I didn't take the time to count them, but I bet most of the commits I
 listed on my previous e-mail message are not bug fixes, but new
 features.

Huh? Of course the latest machine model get new features. The point is
that the previous ones don't and that's what we are providing them for -
libvirt is expected to choose one machine and the contract with QEMU is
that for that machine the CPU does *not* grow new features, and we're
going at great lengths to achieve that. So this thread feels more and
more weird...

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 07:41:50PM +0200, Andreas Färber wrote:
[...]
 If that is the whole problem here, then why not just add a global flag
 to only enable explicitly requested KVM features? All other features
 should not depend on the host, and the whole discussion about -x.y seems
 like a distraction.

Now replace KVM features with CPU fatures, because all CPU features
are KVM features, as all of them depend on KVM code enabling them on
GET_SUPPORTED_CPUID.

Thus, the global flag to only enable explicitly request KVM features on
CPUs is -cpu custom, which doesn't enable any CPU feature at all.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
  
  I didn't take the time to count them, but I bet most of the commits I
  listed on my previous e-mail message are not bug fixes, but new
  features.
 
 Huh? Of course the latest machine model get new features. The point is
 that the previous ones don't and that's what we are providing them for -
 libvirt is expected to choose one machine and the contract with QEMU is
 that for that machine the CPU does *not* grow new features, and we're
 going at great lengths to achieve that. So this thread feels more and
 more weird...

We are not talking about changes to existing machines. We are talking
about having changes introduced in new machines (the one we did on
purpose) affecting the runnability of the VM.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 19:13 schrieb Daniel P. Berrange:
 On Tue, Jun 23, 2015 at 06:47:16PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:42 schrieb Daniel P. Berrange:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
 I've always advocated remaining backwards-compatible and only making CPU
 model changes for new machines. You among others felt that was not
 always necessary, and now you're using the lack thereof as an argument
 to stop using QEMU's CPU models at all? That sounds convoluted...

 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

 Huh, I never said we wanted to bring back bugs. This is about allowing
 libvirt to fix the CPU bugs in a way that is independant of the machine
 types and portable across hypervisors we deal with. We're absolutely
 still going to fix CPU model bugs and ensure stable guest ABI.

 No, that's contradictory! Through the -x.y machines we leave bugs in the
 old models *exactly* to assure a stable guest ABI. Fixes are only be
 applied to new machines, thus I'm pointing out that you should not use a
 new CPU model with an old machine type.
 
 I'm not saying that libvirt would ever allow a silent guest ABI change.
 Given a libvirt XML config, the guest ABI will never change without an
 explicit action on the part of the app/user to change the XML.
 
 This is all about dealing with the case where the app / user conciously
 needs/wants to opt-in to a guest ABI change for the guest. eg they wish
 to make use of some bug fix or feature improvement in the new machine
 type, but they do *not* wish to have the CPU model changed, because
 of some CPU model change that is incompatible with their hosts' CPUs.
 Conversely, they may wish to get access to a new CPU model, but not
 wish to have the rest of the guest ABI change. In both cases the user
 is explicitly opt-ing in the ABI change with knowledge about what
 this might mean for the guest OS. Currently we are tieing users
 hands by forcing CPU and machine types to change in lockstep.

Look, if you keep repeating that users may wish to do random
combinations if we give them the ability to do so, that is not helping
me understand. Right now they can't. If you don't have a single use case
of what CPU model, KVM feature, CPUID bit and guest may constitute a
valid reason to do so, I disagree with adding such a crazy feature.

In summary you seem to be saying that all the years we have spent
fiddling around with those mind-boggling compat_props in QEMU were in
vain because libvirt now wants to start their own versioning system to
give users more degrees of freedom even when you can't articulate a
single concrete reason why users may want to do so.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
  On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
  Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
  To help libvirt in the transition, a x86-cpu-model-dump script is 
  provided,
  that will generate a config file that can be loaded using -readconfig, 
  based on
  the -cpu and -machine options provided in the command-line.
 
  Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
  configuration data to libvirt, but now I think it actually makes sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
  That sounds like FUD to me. Any concrete data points where QEMU does not
  have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
  for.
 
  What Jiri is saying that the CPUs change depending on -mmachine, not
  that the ABI is broken by a given machine.
 
  The problem here is that libvirt needs to provide CPU models whose
  runnability does not depend on the machine-type. If users have a VM that
  is running in a host and the VM machine-type changes,
 
  How does it change, and why?
  
  Sometimes we add features to a CPU model because they were not emulated by 
  KVM
  and now they are. Sometimes we remove or add features or change other fields
  because we are fixing previous mistakes. Recently we we were going to remove
  features from models because of an Intel CPU errata, but then decided to 
  create
  a new CPU model name instead.
  
  See some examples at the end of this message.
  
 
  the VM should be
  still runnable in that host. QEMU doesn't provide that, our CPU models
  may change when we introduce new machine-types, so we are giving them a
  mechanism that allows libvirt to implement the policy they need.
 
  I don't mind wrt CPU specifically, but we absolutely do change guest ABI
  in many ways when we change machine types.
  
  All the other ABI changes we introduce in QEMU don't affect runnability of 
  the
  VM in a given host, that's the problem we are trying to address here. ABI
  changes are expected when changing to a new machine, runnability changes
  aren't.
  
  
  Examples of commits changing CPU models:
 [snip]
 
 I've always advocated remaining backwards-compatible and only making CPU
 model changes for new machines. You among others felt that was not
 always necessary, and now you're using the lack thereof as an argument
 to stop using QEMU's CPU models at all? That sounds convoluted...
 

Uh? I don't remember anybody suggesting changing CPU models on existing
machines. We always tried to keep existing machines compatible.

 BTW your list does not answer my question. You would need examples where
 a CPU model changes between machines, and I am not aware of any example
 beyond the intentional -x.y variations. There are differences between
 KVM and TCG though, did you mean that? i440fx and q35 should be
 identical and isa-pc, too, and none anyway. None of this has anything to
 do with the host CPU.

We are talking about the -x.y variations (that, yes, are intentional).
But the fact that CPU features change (even the intentional ones in the
-x.y machine variations) affect runnability of VMs (because enabling new
CPU features in KVM require it to be supported by the host kernel code
and by the host CPU).

I was not thinking about the KVM and TCG differences, but this may also
help libvirt deal with the KVM and TCG differences if necessary.

I don't know what you mean by i440fx and q35 should be identical
above.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
   On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
   On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
   On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
   Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
   To help libvirt in the transition, a x86-cpu-model-dump script is 
   provided,
   that will generate a config file that can be loaded using 
   -readconfig, based on
   the -cpu and -machine options provided in the command-line.
  
   Thanks Eduardo, I never was a big fan of moving (or copying) all the 
   CPU
   configuration data to libvirt, but now I think it actually makes 
   sense.
   We already have a partial copy of CPU model definitions in libvirt
   anyway, but as QEMU changes some CPU models in some machine types (and
   libvirt does not do that) we have no real control over the guest CPU
   configuration. While what we really want is full control to enforce
   stable guest ABI.
  
   That sounds like FUD to me. Any concrete data points where QEMU does 
   not
   have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
   machines
   for.
  
   What Jiri is saying that the CPUs change depending on -mmachine, not
   that the ABI is broken by a given machine.
  
   The problem here is that libvirt needs to provide CPU models whose
   runnability does not depend on the machine-type. If users have a VM that
   is running in a host and the VM machine-type changes,
  
   How does it change, and why?
   
   Sometimes we add features to a CPU model because they were not emulated 
   by KVM
   and now they are. Sometimes we remove or add features or change other 
   fields
   because we are fixing previous mistakes. Recently we we were going to 
   remove
   features from models because of an Intel CPU errata, but then decided to 
   create
   a new CPU model name instead.
   
   See some examples at the end of this message.
   
  
   the VM should be
   still runnable in that host. QEMU doesn't provide that, our CPU models
   may change when we introduce new machine-types, so we are giving them a
   mechanism that allows libvirt to implement the policy they need.
  
   I don't mind wrt CPU specifically, but we absolutely do change guest ABI
   in many ways when we change machine types.
   
   All the other ABI changes we introduce in QEMU don't affect runnability 
   of the
   VM in a given host, that's the problem we are trying to address here. ABI
   changes are expected when changing to a new machine, runnability changes
   aren't.
   
   
   Examples of commits changing CPU models:
  [snip]
  
  I've always advocated remaining backwards-compatible and only making CPU
  model changes for new machines. You among others felt that was not
  always necessary, and now you're using the lack thereof as an argument
  to stop using QEMU's CPU models at all? That sounds convoluted...
 
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.
 
 Regards,
 Daniel

But that's because we are fixing bugs.  If CPU X used to work on
hardware Y in machine type A and stopped in machine type B, this is
because we have determined that it's the right thing to do for the
guests and the users. We don't break stuff just for fun.
Why do you want to bring back the bugs we fixed?

 -- 
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 05:56:35PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 04:51:00PM +0100, Daniel P. Berrange wrote:
  On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
   On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script is 
 provided,
 that will generate a config file that can be loaded using 
 -readconfig, based on
 the -cpu and -machine options provided in the command-line.
 
 Thanks Eduardo, I never was a big fan of moving (or copying) all the 
 CPU
 configuration data to libvirt, but now I think it actually makes 
 sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

That sounds like FUD to me. Any concrete data points where QEMU does not
have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
for.
   
   What Jiri is saying that the CPUs change depending on -mmachine, not
   that the ABI is broken by a given machine.
   
   The problem here is that libvirt needs to provide CPU models whose
   runnability does not depend on the machine-type. If users have a VM that
   is running in a host and the VM machine-type changes, the VM should be
   still runnable in that host. QEMU doesn't provide that, our CPU models
   may change when we introduce new machine-types, so we are giving them a
   mechanism that allows libvirt to implement the policy they need.
  
  Expanding on that, but tieing the CPU model to the machine type, QEMU
  has in turn effectively tied the machine type to the host hardware.
  eg, switching to a newer machine type, may then prevent the guest
  from being able to launch on the hardware that it was previously
  able to run on, due to some new requirement of the CPU model associated
  with the machine type.
 
 So why not keep machine type stable?

There are many reasons to choose a particular machine type - for
example, to achieve migration compat between hosts with different
QEMU versions, or to enable access to some performance or bug
fix in the machine type in question. Users / apps need to be free
to make those decisions, without being restricted by changes in the
CPU model which may affect what hardware the machine type can be
used on. The current use of machine types for CPU model versioning
is placing users between a rock  hard place, giving them impossible
decisions about which bad behaviour/bug they're willing to accept.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script is 
 provided,
 that will generate a config file that can be loaded using -readconfig, 
 based on
 the -cpu and -machine options provided in the command-line.

 Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
 configuration data to libvirt, but now I think it actually makes sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

 That sounds like FUD to me. Any concrete data points where QEMU does not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y machines
 for.

 What Jiri is saying that the CPUs change depending on -mmachine, not
 that the ABI is broken by a given machine.

 The problem here is that libvirt needs to provide CPU models whose
 runnability does not depend on the machine-type. If users have a VM that
 is running in a host and the VM machine-type changes,

 How does it change, and why?
 
 Sometimes we add features to a CPU model because they were not emulated by KVM
 and now they are. Sometimes we remove or add features or change other fields
 because we are fixing previous mistakes. Recently we we were going to remove
 features from models because of an Intel CPU errata, but then decided to 
 create
 a new CPU model name instead.
 
 See some examples at the end of this message.
 

 the VM should be
 still runnable in that host. QEMU doesn't provide that, our CPU models
 may change when we introduce new machine-types, so we are giving them a
 mechanism that allows libvirt to implement the policy they need.

 I don't mind wrt CPU specifically, but we absolutely do change guest ABI
 in many ways when we change machine types.
 
 All the other ABI changes we introduce in QEMU don't affect runnability of the
 VM in a given host, that's the problem we are trying to address here. ABI
 changes are expected when changing to a new machine, runnability changes
 aren't.
 
 
 Examples of commits changing CPU models:
[snip]

I've always advocated remaining backwards-compatible and only making CPU
model changes for new machines. You among others felt that was not
always necessary, and now you're using the lack thereof as an argument
to stop using QEMU's CPU models at all? That sounds convoluted...

BTW your list does not answer my question. You would need examples where
a CPU model changes between machines, and I am not aware of any example
beyond the intentional -x.y variations. There are differences between
KVM and TCG though, did you mean that? i440fx and q35 should be
identical and isa-pc, too, and none anyway. None of this has anything to
do with the host CPU.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 06:40:19PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:25 schrieb Daniel P. Berrange:
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
 This coupling is by design and we expect all KVM/QEMU users to adhere to
 it, including those that use the libvirt tool (which I assume is going
 to be the majority of KVM users). Either you want a certain
 backwards-compatible machine and CPU, or you want the latest and
 greatest - why in the world mix and match?!

As mentioned, changes/fixes to the CPU model can affect the ability to
launch the guest on a particular host, so we need the ability to control
when those CPU changes are activated for a guest, separately from the
machine type.

 Would a qemu64-2.3 model help here that pc*-2.3 could use? I believe
 that's been proposed in the past. I don't oppose the idea of a
 fully-custom CPU, but this blatant attempt of ignoring QEMU's CPU
 versioning by libvirt worries me.

That is still tieing CPU model names to machine type versions, unless
I'm mis-understanding you. In general having QEMU models avialable
vary depending on the QEMU version is creating problems for apps
higher up the stack. By allowing libvirt to define the CPU model
policy, it can also provide a more consistent interface to applications
above, such as OpenStack, which will facilitate work in the schedular
when it picks hosts capable of deploying/migrating a VM, when there
are heterogenous QEMU versions across the hosts.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 18:53 schrieb Daniel P. Berrange:
 On Tue, Jun 23, 2015 at 06:40:19PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:25 schrieb Daniel P. Berrange:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 This coupling is by design and we expect all KVM/QEMU users to adhere to
 it, including those that use the libvirt tool (which I assume is going
 to be the majority of KVM users). Either you want a certain
 backwards-compatible machine and CPU, or you want the latest and
 greatest - why in the world mix and match?!
 
 As mentioned, changes/fixes to the CPU model can affect the ability to
 launch the guest on a particular host, so we need the ability to control
 when those CPU changes are activated for a guest, separately from the
 machine type.

Why? Today's libvirt with QEMU 2.3 resolves pc machine to
pc-i440fx-2.3 and the guest XML stays that way. When we add new
features for 2.4, 2.3 is guaranteed to stay compatible. Any change would
involve the libvirt user actively switching from pc-i440fx-2.3 to a
different machine such as upcoming pc-i440fx-2.4. Why do you need to
change the CPU separately? Why would a user want to run 2.2's CPU with a
2.3 machine? Or a 2.3 machine with a 2.4 CPU? That's nonsense. If you
want to tweak features, you already have command line options available
to do so on the basis of what the selected machine provides.

 Would a qemu64-2.3 model help here that pc*-2.3 could use? I believe
 that's been proposed in the past. I don't oppose the idea of a
 fully-custom CPU, but this blatant attempt of ignoring QEMU's CPU
 versioning by libvirt worries me.
 
 That is still tieing CPU model names to machine type versions, unless
 I'm mis-understanding you. In general having QEMU models avialable
 vary depending on the QEMU version is creating problems for apps
 higher up the stack. By allowing libvirt to define the CPU model
 policy, it can also provide a more consistent interface to applications
 above, such as OpenStack, which will facilitate work in the schedular
 when it picks hosts capable of deploying/migrating a VM, when there
 are heterogenous QEMU versions across the hosts.

If you have heterogeneous QEMUs across hosts, then you need a
common-denominator machine anyway because QEMU wouldn't start with a
machine it doesn't know.

Please give one concrete example of a real-world problem instead of
hypothetical abstract combinations users may or may not be able to
construct.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 07:18:06PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different versions
  of the machine type, without the CPU changing. IOW it is the fact that 
  the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
 
  I didn't take the time to count them, but I bet most of the commits I
  listed on my previous e-mail message are not bug fixes, but new
  features.
 
  Huh? Of course the latest machine model get new features. The point is
  that the previous ones don't and that's what we are providing them for -
  libvirt is expected to choose one machine and the contract with QEMU is
  that for that machine the CPU does *not* grow new features, and we're
  going at great lengths to achieve that. So this thread feels more and
  more weird...
  
  We are not talking about changes to existing machines. We are talking
  about having changes introduced in new machines (the one we did on
  purpose) affecting the runnability of the VM.
 
 You are talking abstract!
 
 
 Example 1:
 
 Point A: Machine pc-i440fx-2.3 exists
 
 Runs or runs not.
 
 Point B: Machine pc-i440fx-2.3 still exists
 
 Still runs or runs not due to guest ABI stability rules.
 
 
 Example 2:
 
 Point A: pc-i440fx-2.4 does not exist in 2.3
 
 Does not run becomes it doesn't exist.
 
 Point B: New pc-i440fx-2.4
 
 Runs or does not run, and if so has more features than pc-i440fx-2.3.
 
 There is no runnability problem - either it runs or it doesn't, but
 there's no change over time.
 
 This is what the machine -x.y versioning is all about.

Consider a host currently running QEMU 2.3 with machine type
pc-i440fx-2.3 used with SandyBridge.

Now consider the def of SandyBridge was buggy and so in QEMU
2.4 we add the missing CPU feature flag 'wizz', and only
enable that new feature flag with pc-i440fx-2.4

Now consider there was a bug in the virtio-scsi driver that
we also fixed in QEMU 2.4 and thus pc-i440fx-2.4 includes
that fix.

Updating from pc-i440fx-2.3 to pc-i440fx-2.4 has a dependancy
on the host CPU including the 'wizz' flag that was added. This
new CPU feature can prevent the user from using the new machine
type to get the virtio-scsi bug fix.

Separately versioning CPU models still lets us preserve a stable
guest ABI by default, but allows more flexibility when choosing
to opt out of the ABI for a particular upgrade.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 07:55:14PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:42 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 07:29:15PM +0200, Andreas Färber wrote:
  In summary you seem to be saying that all the years we have spent
  fiddling around with those mind-boggling compat_props in QEMU were in
  vain because libvirt now wants to start their own versioning system to
  give users more degrees of freedom even when you can't articulate a
  single concrete reason why users may want to do so.
  
  I had a similar reaction when I learned about this libvirt
  expectation/requirement I was never aware of. But we spent lots of
  effort trying to do things differently doesn't seem like a valid
  justification for design decision.
 
 True, but my expectation is that libvirt is a friendly wrapper around
 QEMU using the mechanisms like -x.y we implemented for it and does not
 stand up at random and says, like Dan, that it wants to have things
 differently from now on, breaking all past assumptions and guarantees.

FWIW, this isn't actually a change in opinion from the libvirt side.
We never wanted CPU representation tied to machine types, even years
back when the first qemu discussions in this area took place. We
always wanted to be able to control CPU definition separately from
the machine type.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 19:42 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 07:29:15PM +0200, Andreas Färber wrote:
 In summary you seem to be saying that all the years we have spent
 fiddling around with those mind-boggling compat_props in QEMU were in
 vain because libvirt now wants to start their own versioning system to
 give users more degrees of freedom even when you can't articulate a
 single concrete reason why users may want to do so.
 
 I had a similar reaction when I learned about this libvirt
 expectation/requirement I was never aware of. But we spent lots of
 effort trying to do things differently doesn't seem like a valid
 justification for design decision.

True, but my expectation is that libvirt is a friendly wrapper around
QEMU using the mechanisms like -x.y we implemented for it and does not
stand up at random and says, like Dan, that it wants to have things
differently from now on, breaking all past assumptions and guarantees.

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 07:41:50PM +0200, Andreas Färber wrote:
 Only if you use -cpu ...,enforce, no?
 
 The KVM feature filtering should take care of dropping features that are
 not available otherwise.
 
 So we seem to be getting to the interesting case of the same machine
 (different from what was said previously!) but different hosts.
 
 The QOM property gives you insights into which feature bits are set for
 the machine for the model (and for s390x I saw QMP extensions to the
 same effect, I thought). That way you could discover features to
 disable. However you'll only ever know which ones work once you've tried
 it once, right?

On this subject, a big problem in QOM in general is that it hasn't
included a distinction between object classes and object instances.
So there's no way for us to introspect what a machine type provides
without actually instantiating a guest with that machine type.

Again allowing libvirt to control the CPU model removes this problem
as libvirt will be able to determine what CPUs it can run on a given
host without having to probe all the different CPU - machine type
combinations for the QEMU on that host has.

This in turn simplifies life for apps using libvirt, as they will always
know that CPUFoo-X.Y will always mean the exact same thing on all the
hosts they have no matter what QEMU version/machine is in use.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 21:25 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 08:35:54PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:39 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 07:18:06PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory 
 situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different 
 versions
 of the machine type, without the CPU changing. IOW it is the fact 
 that the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

 I didn't take the time to count them, but I bet most of the commits I
 listed on my previous e-mail message are not bug fixes, but new
 features.

 Huh? Of course the latest machine model get new features. The point is
 that the previous ones don't and that's what we are providing them for -
 libvirt is expected to choose one machine and the contract with QEMU is
 that for that machine the CPU does *not* grow new features, and we're
 going at great lengths to achieve that. So this thread feels more and
 more weird...

 We are not talking about changes to existing machines. We are talking
 about having changes introduced in new machines (the one we did on
 purpose) affecting the runnability of the VM.

 You are talking abstract!

 I am just talking about a different problem, and I don't know if you are
 purposely trying to ignore it, or are just denying that it is a problem.

 So, are you and Dan talking about the same problem or different ones?
 
 The same one.
 
 I am not deliberately ignoring anything here, but I am denying there is
 a problem until either of you explains what a concrete problem is. Seems
 we are slowly getting there now.
 
 I hope so. :)
 

 Example 1:

 Point A: Machine pc-i440fx-2.3 exists

 Runs or runs not.

 Point B: Machine pc-i440fx-2.3 still exists

 Still runs or runs not due to guest ABI stability rules.

 If you didn't change the machine name, this is not the problem we are
 talking about.

 OK.

 Example 2:

 Point A: pc-i440fx-2.4 does not exist in 2.3

 Does not run becomes it doesn't exist.

 Point B: New pc-i440fx-2.4

 Runs or does not run, and if so has more features than pc-i440fx-2.3.

 If you didn't change the machine name, this is not the problem we are
 talking about.


 There is no runnability problem - either it runs or it doesn't, but
 there's no change over time.

 This is what the machine -x.y versioning is all about.

 Let's try a concrete example:

 * User is running a kernel that can't emulate x2apic
 * User is running pc-i440fx-1.7
 * User wants the gigabyte alignment change implemented by commit
   bb43d3839c29b17a2f5c122114cd4ca978065a18
 * User changes machine to pc-i440fx-2.0
 * x2apic is now enabled by default in all CPU models
 * VM with the same configuration (just the machine change) is not
   runnable anymore in the same host

 Then let's take a step back: In order to change the machine type, the
 user shuts the machine down (it does not run!), edits the XML and tries
 to boot it up again. That's where I've challenged your use of the term
 of changed runnability above. I acknowledged, it might happen that it
 does not run. But that has nothing to do with compatibility of QEMU
 versions v2.3.0 vs. v2.4.0 then, it is the user's active choice of
 options that are incompatible with her system and that never before
 worked there. That seems perfectly valid and unavoidable, just like
 adding a non-existing command-line option or an unknown XML element to
 the guest config.
 
 If you add a non-existing command-line option or unknown XML element,
 you are providing bad input. The most obvious way to handle it is an
 error.
 
 If you change -machine, you are providing good input, but the user won't
 have a good explanation why it can't run (because it is a perfectly
 valid machine name, reported as supported by QEMU). Or maybe they will
 see an explanation, but will have no idea what they need to change to
 make the new machine runnable.
 
 CPU model runnability, on the other hand, is well documented in the
 libvirt API, and would even allow a management system to automatically
 find a solution (because it can tell exactly 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 08:35:54PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:39 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 07:18:06PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
  Whether QEMU changed the CPU for existing machines, or only for new
  machines is actually not the core problem. Even if we only changed
  the CPU in new machines that would still be an unsatisfactory 
  situation
  because we want to be able to be able to access different versions of
  the CPU without the machine type changing, and access different 
  versions
  of the machine type, without the CPU changing. IOW it is the fact 
  that the
  changes in CPU are tied to changes in machine type that is the core
  problem.
 
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
 
  I didn't take the time to count them, but I bet most of the commits I
  listed on my previous e-mail message are not bug fixes, but new
  features.
 
  Huh? Of course the latest machine model get new features. The point is
  that the previous ones don't and that's what we are providing them for -
  libvirt is expected to choose one machine and the contract with QEMU is
  that for that machine the CPU does *not* grow new features, and we're
  going at great lengths to achieve that. So this thread feels more and
  more weird...
 
  We are not talking about changes to existing machines. We are talking
  about having changes introduced in new machines (the one we did on
  purpose) affecting the runnability of the VM.
 
  You are talking abstract!
  
  I am just talking about a different problem, and I don't know if you are
  purposely trying to ignore it, or are just denying that it is a problem.
 
 So, are you and Dan talking about the same problem or different ones?

The same one.

 I am not deliberately ignoring anything here, but I am denying there is
 a problem until either of you explains what a concrete problem is. Seems
 we are slowly getting there now.

I hope so. :)

 
  Example 1:
 
  Point A: Machine pc-i440fx-2.3 exists
 
  Runs or runs not.
 
  Point B: Machine pc-i440fx-2.3 still exists
 
  Still runs or runs not due to guest ABI stability rules.
  
  If you didn't change the machine name, this is not the problem we are
  talking about.
 
 OK.
 
  Example 2:
 
  Point A: pc-i440fx-2.4 does not exist in 2.3
 
  Does not run becomes it doesn't exist.
 
  Point B: New pc-i440fx-2.4
 
  Runs or does not run, and if so has more features than pc-i440fx-2.3.
  
  If you didn't change the machine name, this is not the problem we are
  talking about.
  
 
  There is no runnability problem - either it runs or it doesn't, but
  there's no change over time.
 
  This is what the machine -x.y versioning is all about.
  
  Let's try a concrete example:
  
  * User is running a kernel that can't emulate x2apic
  * User is running pc-i440fx-1.7
  * User wants the gigabyte alignment change implemented by commit
bb43d3839c29b17a2f5c122114cd4ca978065a18
  * User changes machine to pc-i440fx-2.0
  * x2apic is now enabled by default in all CPU models
  * VM with the same configuration (just the machine change) is not
runnable anymore in the same host
 
 Then let's take a step back: In order to change the machine type, the
 user shuts the machine down (it does not run!), edits the XML and tries
 to boot it up again. That's where I've challenged your use of the term
 of changed runnability above. I acknowledged, it might happen that it
 does not run. But that has nothing to do with compatibility of QEMU
 versions v2.3.0 vs. v2.4.0 then, it is the user's active choice of
 options that are incompatible with her system and that never before
 worked there. That seems perfectly valid and unavoidable, just like
 adding a non-existing command-line option or an unknown XML element to
 the guest config.

If you add a non-existing command-line option or unknown XML element,
you are providing bad input. The most obvious way to handle it is an
error.

If you change -machine, you are providing good input, but the user won't
have a good explanation why it can't run (because it is a perfectly
valid machine name, reported as supported by QEMU). Or maybe they will
see an explanation, but will have no idea what they need to change to
make the new machine runnable.

CPU model runnability, on the other hand, is well documented in the
libvirt API, and would even allow a management system to automatically
find a solution 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 19:45 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 07:41:50PM +0200, Andreas Färber wrote:
 [...]
 If that is the whole problem here, then why not just add a global flag
 to only enable explicitly requested KVM features? All other features
 should not depend on the host, and the whole discussion about -x.y seems
 like a distraction.
 
 Now replace KVM features with CPU fatures, because all CPU features
 are KVM features, as all of them depend on KVM code enabling them on
 GET_SUPPORTED_CPUID.
 
 Thus, the global flag to only enable explicitly request KVM features on
 CPUs is -cpu custom, which doesn't enable any CPU feature at all.

If libvirt wants to use an empty CPU model, then why export our models
to libvirt?

I don't mind there being an optional custom model, I mind our
compat_props getting ignored that way, which are unrelated to adding new
features, in fact they suppress just that for the -2.3 examples.

Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton; HRB
21284 (AG Nürnberg)



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Daniel P. Berrange
On Tue, Jun 23, 2015 at 07:58:44PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:45 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 07:41:50PM +0200, Andreas Färber wrote:
  [...]
  If that is the whole problem here, then why not just add a global flag
  to only enable explicitly requested KVM features? All other features
  should not depend on the host, and the whole discussion about -x.y seems
  like a distraction.
  
  Now replace KVM features with CPU fatures, because all CPU features
  are KVM features, as all of them depend on KVM code enabling them on
  GET_SUPPORTED_CPUID.
  
  Thus, the global flag to only enable explicitly request KVM features on
  CPUs is -cpu custom, which doesn't enable any CPU feature at all.
 
 If libvirt wants to use an empty CPU model, then why export our models
 to libvirt?
 
 I don't mind there being an optional custom model, I mind our
 compat_props getting ignored that way, which are unrelated to adding new
 features, in fact they suppress just that for the -2.3 examples.

If QEMU has a '-cpu custom' there isn't actually any need for QEMU to
export its CPU models to libvirt. Libvirt already has its own CPU
model database, so we'd just start to use our own database exclusively.

Libvirt would have to take care that we don't break ABI when updating
from a QEMU that pre-dates the '-cpu custom' feature - for that we
have done a one-time only extraction of details of all the historical
variations in QEMU CPUS models per machine type.

So apart from backcompat for existing QEMUs in the wild, libvirt would
no longer have any need to know about QEMU's CPU models.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 07:58:44PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:45 schrieb Eduardo Habkost:
  On Tue, Jun 23, 2015 at 07:41:50PM +0200, Andreas Färber wrote:
  [...]
  If that is the whole problem here, then why not just add a global flag
  to only enable explicitly requested KVM features? All other features
  should not depend on the host, and the whole discussion about -x.y seems
  like a distraction.
  
  Now replace KVM features with CPU fatures, because all CPU features
  are KVM features, as all of them depend on KVM code enabling them on
  GET_SUPPORTED_CPUID.
  
  Thus, the global flag to only enable explicitly request KVM features on
  CPUs is -cpu custom, which doesn't enable any CPU feature at all.
 
 If libvirt wants to use an empty CPU model, then why export our models
 to libvirt?

We wouldn't need it anymore. We would probably keep it just because old
libvirt versions (or maybe the non-x86 libvirt code) may use it, or
maybe libvirt will keep using the existing QEMU models for existing VMs
(because that would be simpler than converting existing -cpu Foo to
-cpu custom -readconfig).

 
 I don't mind there being an optional custom model, I mind our
 compat_props getting ignored that way, which are unrelated to adding new
 features, in fact they suppress just that for the -2.3 examples.

I am also not happy for having spent lots of time on compat_props for
CPUs when it is not going to be needed by libvirt anymore, but that's a
sunk cost.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Andreas Färber
Am 23.06.2015 um 19:39 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 07:18:06PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 19:08 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:44:57PM +0200, Andreas Färber wrote:
 Am 23.06.2015 um 18:38 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
 Whether QEMU changed the CPU for existing machines, or only for new
 machines is actually not the core problem. Even if we only changed
 the CPU in new machines that would still be an unsatisfactory situation
 because we want to be able to be able to access different versions of
 the CPU without the machine type changing, and access different versions
 of the machine type, without the CPU changing. IOW it is the fact that 
 the
 changes in CPU are tied to changes in machine type that is the core
 problem.

 But that's because we are fixing bugs.  If CPU X used to work on
 hardware Y in machine type A and stopped in machine type B, this is
 because we have determined that it's the right thing to do for the
 guests and the users. We don't break stuff just for fun.
 Why do you want to bring back the bugs we fixed?

 I didn't take the time to count them, but I bet most of the commits I
 listed on my previous e-mail message are not bug fixes, but new
 features.

 Huh? Of course the latest machine model get new features. The point is
 that the previous ones don't and that's what we are providing them for -
 libvirt is expected to choose one machine and the contract with QEMU is
 that for that machine the CPU does *not* grow new features, and we're
 going at great lengths to achieve that. So this thread feels more and
 more weird...

 We are not talking about changes to existing machines. We are talking
 about having changes introduced in new machines (the one we did on
 purpose) affecting the runnability of the VM.

 You are talking abstract!
 
 I am just talking about a different problem, and I don't know if you are
 purposely trying to ignore it, or are just denying that it is a problem.

So, are you and Dan talking about the same problem or different ones?
I am not deliberately ignoring anything here, but I am denying there is
a problem until either of you explains what a concrete problem is. Seems
we are slowly getting there now.

 Example 1:

 Point A: Machine pc-i440fx-2.3 exists

 Runs or runs not.

 Point B: Machine pc-i440fx-2.3 still exists

 Still runs or runs not due to guest ABI stability rules.
 
 If you didn't change the machine name, this is not the problem we are
 talking about.

OK.

 Example 2:

 Point A: pc-i440fx-2.4 does not exist in 2.3

 Does not run becomes it doesn't exist.

 Point B: New pc-i440fx-2.4

 Runs or does not run, and if so has more features than pc-i440fx-2.3.
 
 If you didn't change the machine name, this is not the problem we are
 talking about.
 

 There is no runnability problem - either it runs or it doesn't, but
 there's no change over time.

 This is what the machine -x.y versioning is all about.
 
 Let's try a concrete example:
 
 * User is running a kernel that can't emulate x2apic
 * User is running pc-i440fx-1.7
 * User wants the gigabyte alignment change implemented by commit
   bb43d3839c29b17a2f5c122114cd4ca978065a18
 * User changes machine to pc-i440fx-2.0
 * x2apic is now enabled by default in all CPU models
 * VM with the same configuration (just the machine change) is not
   runnable anymore in the same host

Then let's take a step back: In order to change the machine type, the
user shuts the machine down (it does not run!), edits the XML and tries
to boot it up again. That's where I've challenged your use of the term
of changed runnability above. I acknowledged, it might happen that it
does not run. But that has nothing to do with compatibility of QEMU
versions v2.3.0 vs. v2.4.0 then, it is the user's active choice of
options that are incompatible with her system and that never before
worked there. That seems perfectly valid and unavoidable, just like
adding a non-existing command-line option or an unknown XML element to
the guest config.

The difference of opinion seems to be that when there is a bug in QEMU,
I require that the user updates QEMU (not necessarily to a new version),
whereas you are proposing that libvirt should be the one to work around
bugs in QEMU by tweaking command line parameters.

In order to get a virtio-scsi or gigabyte alignment fix that varies
across -x.y machines, that feature can just as well be enabled via
global properties on the old machine. New machines are primarily for new
features.

If someone wants to use that new -2.0 machine, they need to pass the
correct options such as ,-x2apic in your example or use a CPU model
that does not enable such options by default. (FWIW in that concrete
example I remember Paolo(?) saying that that feature had been supported
for a really long time already.)
The user, who actively 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 09:41:51PM +0200, Andreas Färber wrote:
[...]
 I am going to stop arguing here and suggest you put this on the agenda
 for the next KVM call.

I am a bit confused. You said I don't mind there being an optional
custom model in a previous message.

If you have objections to libvirt API design decisions, I understand it,
but I suggest you take them to the libvir-list mailing list.

Now, if you have objections to having an optional custom model (i.e.
valid reasons to not apply patch 1/2), please let me know. I didn't see
a single argument to reject the patch, yet.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 06:13:24PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 06:47:16PM +0200, Andreas Färber wrote:
  Am 23.06.2015 um 18:42 schrieb Daniel P. Berrange:
   On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
   On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
   On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
   Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
   I've always advocated remaining backwards-compatible and only making 
   CPU
   model changes for new machines. You among others felt that was not
   always necessary, and now you're using the lack thereof as an argument
   to stop using QEMU's CPU models at all? That sounds convoluted...
  
   Whether QEMU changed the CPU for existing machines, or only for new
   machines is actually not the core problem. Even if we only changed
   the CPU in new machines that would still be an unsatisfactory situation
   because we want to be able to be able to access different versions of
   the CPU without the machine type changing, and access different versions
   of the machine type, without the CPU changing. IOW it is the fact that 
   the
   changes in CPU are tied to changes in machine type that is the core
   problem.
  
   But that's because we are fixing bugs.  If CPU X used to work on
   hardware Y in machine type A and stopped in machine type B, this is
   because we have determined that it's the right thing to do for the
   guests and the users. We don't break stuff just for fun.
   Why do you want to bring back the bugs we fixed?
   
   Huh, I never said we wanted to bring back bugs. This is about allowing
   libvirt to fix the CPU bugs in a way that is independant of the machine
   types and portable across hypervisors we deal with. We're absolutely
   still going to fix CPU model bugs and ensure stable guest ABI.
  
  No, that's contradictory! Through the -x.y machines we leave bugs in the
  old models *exactly* to assure a stable guest ABI. Fixes are only be
  applied to new machines, thus I'm pointing out that you should not use a
  new CPU model with an old machine type.
 
 I'm not saying that libvirt would ever allow a silent guest ABI change.
 Given a libvirt XML config, the guest ABI will never change without an
 explicit action on the part of the app/user to change the XML.
 
 This is all about dealing with the case where the app / user conciously
 needs/wants to opt-in to a guest ABI change for the guest. eg they wish
 to make use of some bug fix or feature improvement in the new machine
 type, but they do *not* wish to have the CPU model changed, because
 of some CPU model change that is incompatible with their hosts' CPUs.
 Conversely, they may wish to get access to a new CPU model, but not
 wish to have the rest of the guest ABI change. In both cases the user
 is explicitly opt-ing in the ABI change with knowledge about what
 this might mean for the guest OS. Currently we are tieing users
 hands by forcing CPU and machine types to change in lockstep.
 
 Regards,
 Daniel

Can we have a specific example please?  It's hard to understand the
facts based on such generic statements.


 -- 
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 05:26:38PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 09:41:51PM +0200, Andreas Färber wrote:
 [...]
  Given that we have had this versioning system for years and no problem
  specifically with 2.4 has been raised, I see this as 2.5+ material at
  this point.
 
 I see this on 2.4 schedule:
 
 2015-06-16   Soft feature freeze. All features
   should have patches on the list by this date; major features should have
   initial code committed.
 
 It is a 9-line patch (from which 6 are declarations, and 2 are actual
 code statements), it doesn't affect anybody who is not explicitly using
 -cpu custom, it is an useful feature, and it has been on the list
 since June 8 (and as RFC since April 13).
 
 Unless somebody gives me a good reason to consider it harmful (that's
 different from not accepting the explanations why it is useful, or
 disagreeing with users that want to use it), I would like to include it
 in 2.4 as long as I get Reviewed-by lines before hard freeze.

For the record I have nothing against the patch itself.
Might be useful for testing or something.
And how libvirt uses QEMU is, in the end, in the hands of
libvirt developers.

I do care about command line stability generally, and about
people not breaking existing documentation and tools
using QEMU directly as opposed to through libvirt,
that's the only reason I participated in this discussion.

 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 05:42:04PM +0100, Daniel P. Berrange wrote:
 On Tue, Jun 23, 2015 at 06:33:05PM +0200, Michael S. Tsirkin wrote:
  On Tue, Jun 23, 2015 at 05:25:55PM +0100, Daniel P. Berrange wrote:
   On Tue, Jun 23, 2015 at 06:15:51PM +0200, Andreas Färber wrote:
Am 23.06.2015 um 17:58 schrieb Eduardo Habkost:
 On Tue, Jun 23, 2015 at 05:32:42PM +0200, Michael S. Tsirkin wrote:
 On Tue, Jun 23, 2015 at 12:08:28PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 02:32:00PM +0200, Andreas Färber wrote:
 Am 08.06.2015 um 22:18 schrieb Jiri Denemark:
 To help libvirt in the transition, a x86-cpu-model-dump script 
 is provided,
 that will generate a config file that can be loaded using 
 -readconfig, based on
 the -cpu and -machine options provided in the command-line.

 Thanks Eduardo, I never was a big fan of moving (or copying) all 
 the CPU
 configuration data to libvirt, but now I think it actually makes 
 sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types 
 (and
 libvirt does not do that) we have no real control over the guest 
 CPU
 configuration. While what we really want is full control to 
 enforce
 stable guest ABI.

 That sounds like FUD to me. Any concrete data points where QEMU 
 does not
 have a stable ABI for x86 CPUs? That's what we have the pc*-x.y 
 machines
 for.

 What Jiri is saying that the CPUs change depending on -mmachine, not
 that the ABI is broken by a given machine.

 The problem here is that libvirt needs to provide CPU models whose
 runnability does not depend on the machine-type. If users have a VM 
 that
 is running in a host and the VM machine-type changes,

 How does it change, and why?
 
 Sometimes we add features to a CPU model because they were not 
 emulated by KVM
 and now they are. Sometimes we remove or add features or change other 
 fields
 because we are fixing previous mistakes. Recently we we were going to 
 remove
 features from models because of an Intel CPU errata, but then decided 
 to create
 a new CPU model name instead.
 
 See some examples at the end of this message.
 

 the VM should be
 still runnable in that host. QEMU doesn't provide that, our CPU 
 models
 may change when we introduce new machine-types, so we are giving 
 them a
 mechanism that allows libvirt to implement the policy they need.

 I don't mind wrt CPU specifically, but we absolutely do change guest 
 ABI
 in many ways when we change machine types.
 
 All the other ABI changes we introduce in QEMU don't affect 
 runnability of the
 VM in a given host, that's the problem we are trying to address here. 
 ABI
 changes are expected when changing to a new machine, runnability 
 changes
 aren't.
 
 
 Examples of commits changing CPU models:
[snip]

I've always advocated remaining backwards-compatible and only making CPU
model changes for new machines. You among others felt that was not
always necessary, and now you're using the lack thereof as an argument
to stop using QEMU's CPU models at all? That sounds convoluted...
   
   Whether QEMU changed the CPU for existing machines, or only for new
   machines is actually not the core problem. Even if we only changed
   the CPU in new machines that would still be an unsatisfactory situation
   because we want to be able to be able to access different versions of
   the CPU without the machine type changing, and access different versions
   of the machine type, without the CPU changing. IOW it is the fact that the
   changes in CPU are tied to changes in machine type that is the core
   problem.
  
  But that's because we are fixing bugs.  If CPU X used to work on
  hardware Y in machine type A and stopped in machine type B, this is
  because we have determined that it's the right thing to do for the
  guests and the users. We don't break stuff just for fun.
  Why do you want to bring back the bugs we fixed?
 
 Huh, I never said we wanted to bring back bugs. This is about allowing
 libvirt to fix the CPU bugs in a way that is independant of the machine
 types and portable across hypervisors we deal with. We're absolutely
 still going to fix CPU model bugs and ensure stable guest ABI.
 
 Regards,
 Daniel

So any single CPU flag now needs to be added in
- kvm
- qemu
- libvirt

Next thing libvirt will decide it's a policy thing and so
needs to be pushed up to openstack.

We should just figure out what you want to do and support it in QEMU.

Are there many examples where a single flag used to work and then
stopped? I would say such a change is problematic anyway:
not everyone uses libvirt, you are breaking things for people
that run 

Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 02:11:22PM -0300, Eduardo Habkost wrote:
 Even if it is a bug fix. If it is a change that can make the VM
 unrunnable, it needs to be controlled by a separate flag, not by the
 machine-type.

I agree - command line compatibility is important.  But we are supposed
to provide that.  I am surprised that libvirt suddenly wants to avoid
some command line flags because they are not stable. IMHO we did something
wrong here if so. Maybe there was a valid reason for it. But then won't
it apply to libvirt as well?

Now, if people want to update CPU models outside the QEMU binary,
that might be doable simply by moving them to a separate package,
with a text file that QEMU reads at startup.


 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Eduardo Habkost
On Tue, Jun 23, 2015 at 09:41:51PM +0200, Andreas Färber wrote:
[...]
 Given that we have had this versioning system for years and no problem
 specifically with 2.4 has been raised, I see this as 2.5+ material at
 this point.

I see this on 2.4 schedule:

2015-06-16 Soft feature freeze. All features
should have patches on the list by this date; major features should have
initial code committed.

It is a 9-line patch (from which 6 are declarations, and 2 are actual
code statements), it doesn't affect anybody who is not explicitly using
-cpu custom, it is an useful feature, and it has been on the list
since June 8 (and as RFC since April 13).

Unless somebody gives me a good reason to consider it harmful (that's
different from not accepting the explanations why it is useful, or
disagreeing with users that want to use it), I would like to include it
in 2.4 as long as I get Reviewed-by lines before hard freeze.

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-23 Thread Michael S. Tsirkin
On Tue, Jun 23, 2015 at 02:42:37PM -0300, Eduardo Habkost wrote:
 On Tue, Jun 23, 2015 at 07:29:15PM +0200, Andreas Färber wrote:
  In summary you seem to be saying that all the years we have spent
  fiddling around with those mind-boggling compat_props in QEMU were in
  vain because libvirt now wants to start their own versioning system to
  give users more degrees of freedom even when you can't articulate a
  single concrete reason why users may want to do so.
 
 I had a similar reaction when I learned about this libvirt
 expectation/requirement I was never aware of. But we spent lots of
 effort trying to do things differently doesn't seem like a valid
 justification for design decision.

Users will be hurt because they'll run untested configurations
seems like a valid reason.

 -- 
 Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-16 Thread Eduardo Habkost

Ping? Any feedback? I want to get this into 2.4.

On Mon, Jun 08, 2015 at 04:07:38PM -0300, Eduardo Habkost wrote:
 The problem:
 
 The existing libvirt APIs assume that if a given CPU model is runnable in a
 host kernel+hardware combination, it will be always runnable on that host even
 if the machine-type changes.
 
 That assumption is implied in some of libvirt interfaces, for example, at:
 
 1) Host capabilities, which let callers know the set of CPU models
that can run in a host:
https://libvirt.org/formatcaps.html#elementHost
 
virsh capabilities returns a CPU model name + CPU feature list, assuming
that a CPU model name has a meaning that's independent from the
machine-type.
 
 2) The function that checks if a given CPU model definition
is compatible with a host (virConnectCompareCPU()),
which does not take the machine-type as argument:
http://libvirt.org/html/libvirt-libvirt-host.html#virConnectCompareCPU
 
 But that assumption is not true, as QEMU changes CPU models in new
 machine-types when fixing bugs, or when new features (previously unsupported 
 by
 QEMU, TCG or KVM) get implemented.
 
 The solution:
 
 libvirt can solve this problem partially by making sure every feature in a CPU
 model is explicitly configured, instead of (incorrectly) expecting that a 
 named
 CPU model will never change in QEMU. But this doesn't solve the problem
 completely, because it is still possible that new features unknown to libvirt
 get enabled in the default CPU model in future machine-types (that's very
 likely to happen when we introduce new KVM features, for example).
 
 So, to make sure no new feature will be ever enabled without the knowledge of
 libvirt, add a -cpu custom mode, where no CPU model data is loaded at all,
 and everything needs to be configured explicitly using CPU properties. That
 means no CPU features will ever change depending on machine-type or 
 accelerator
 capabilities when using -cpu custom.
 
   * * *
 
 I know that this is basically the opposite of what we were aiming at in the
 last few month^Wyears, where we were struggling to implement probing 
 interfaces
 to expose machine-type-dependent data for libvirt. But, at least the fact that
 we could implement the new approach using a 9-line patch means we were still
 going in the right direction. :)
 
 To help libvirt in the transition, a x86-cpu-model-dump script is provided,
 that will generate a config file that can be loaded using -readconfig, based 
 on
 the -cpu and -machine options provided in the command-line.
 
   * * *
 
 This is basically the same version I sent as an RFC in April. A git tree is
 available at:
 
   git://github.com/ehabkost/qemu-hacks.git work/x86-cpu-custom-model
 
 Eduardo Habkost (2):
   target-i386: Introduce -cpu custom
   scripts: x86-cpu-model-dump script
 
  scripts/x86-cpu-model-dump | 322 
 +
  target-i386/cpu.c  |  10 +-
  2 files changed, 331 insertions(+), 1 deletion(-)
  create mode 100755 scripts/x86-cpu-model-dump
 
 -- 
 2.1.0
 
 

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-09 Thread Daniel P. Berrange
On Mon, Jun 08, 2015 at 10:18:35PM +0200, Jiri Denemark wrote:
 On Mon, Jun 08, 2015 at 16:07:38 -0300, Eduardo Habkost wrote:
 ...
  libvirt can solve this problem partially by making sure every feature in a 
  CPU
  model is explicitly configured, instead of (incorrectly) expecting that a 
  named
  CPU model will never change in QEMU. But this doesn't solve the problem
  completely, because it is still possible that new features unknown to 
  libvirt
  get enabled in the default CPU model in future machine-types (that's very
  likely to happen when we introduce new KVM features, for example).
  
  So, to make sure no new feature will be ever enabled without the knowledge 
  of
  libvirt, add a -cpu custom mode, where no CPU model data is loaded at all,
  and everything needs to be configured explicitly using CPU properties. That
  means no CPU features will ever change depending on machine-type or 
  accelerator
  capabilities when using -cpu custom.
  
* * *
  
  I know that this is basically the opposite of what we were aiming at in the
  last few month^Wyears, where we were struggling to implement probing 
  interfaces
  to expose machine-type-dependent data for libvirt. But, at least the fact 
  that
  we could implement the new approach using a 9-line patch means we were still
  going in the right direction. :)
  
  To help libvirt in the transition, a x86-cpu-model-dump script is provided,
  that will generate a config file that can be loaded using -readconfig, 
  based on
  the -cpu and -machine options provided in the command-line.
 
 Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
 configuration data to libvirt, but now I think it actually makes sense.
 We already have a partial copy of CPU model definitions in libvirt
 anyway, but as QEMU changes some CPU models in some machine types (and
 libvirt does not do that) we have no real control over the guest CPU
 configuration. While what we really want is full control to enforce
 stable guest ABI.

I meanwhile, always wanted the full CPU config data in libvirt, so that
we could ensure libvirt was able to use the exact same CPU model setup
on other hypervisors too - eg Xen, VMWare let us specify the CPUID masks
so we could re-use the libvirt data there.

 I will summarize my ideas on how libvirt should be using -cpu custom and
 send them to libvirt-list soon.

These patches are x86 obviously - is there anything we need to worry about
for non-x86 architectures I wonder ? IIRC, all the non-x86 archs have
traditionally only needed CPU model names and don't really have the same
level of per-flag CPUID mask configurability that x86 has.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-09 Thread Eduardo Habkost
On Tue, Jun 09, 2015 at 09:56:25AM +0100, Daniel P. Berrange wrote:
 On Mon, Jun 08, 2015 at 10:18:35PM +0200, Jiri Denemark wrote:
  On Mon, Jun 08, 2015 at 16:07:38 -0300, Eduardo Habkost wrote:
  ...
   libvirt can solve this problem partially by making sure every feature in 
   a CPU
   model is explicitly configured, instead of (incorrectly) expecting that a 
   named
   CPU model will never change in QEMU. But this doesn't solve the problem
   completely, because it is still possible that new features unknown to 
   libvirt
   get enabled in the default CPU model in future machine-types (that's very
   likely to happen when we introduce new KVM features, for example).
   
   So, to make sure no new feature will be ever enabled without the 
   knowledge of
   libvirt, add a -cpu custom mode, where no CPU model data is loaded at 
   all,
   and everything needs to be configured explicitly using CPU properties. 
   That
   means no CPU features will ever change depending on machine-type or 
   accelerator
   capabilities when using -cpu custom.
   
 * * *
   
   I know that this is basically the opposite of what we were aiming at in 
   the
   last few month^Wyears, where we were struggling to implement probing 
   interfaces
   to expose machine-type-dependent data for libvirt. But, at least the fact 
   that
   we could implement the new approach using a 9-line patch means we were 
   still
   going in the right direction. :)
   
   To help libvirt in the transition, a x86-cpu-model-dump script is 
   provided,
   that will generate a config file that can be loaded using -readconfig, 
   based on
   the -cpu and -machine options provided in the command-line.
  
  Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
  configuration data to libvirt, but now I think it actually makes sense.
  We already have a partial copy of CPU model definitions in libvirt
  anyway, but as QEMU changes some CPU models in some machine types (and
  libvirt does not do that) we have no real control over the guest CPU
  configuration. While what we really want is full control to enforce
  stable guest ABI.
 
 I meanwhile, always wanted the full CPU config data in libvirt, so that
 we could ensure libvirt was able to use the exact same CPU model setup
 on other hypervisors too - eg Xen, VMWare let us specify the CPUID masks
 so we could re-use the libvirt data there.
 
  I will summarize my ideas on how libvirt should be using -cpu custom and
  send them to libvirt-list soon.
 
 These patches are x86 obviously - is there anything we need to worry about
 for non-x86 architectures I wonder ? IIRC, all the non-x86 archs have
 traditionally only needed CPU model names and don't really have the same
 level of per-flag CPUID mask configurability that x86 has.

X86 started with opaque CPU model names hiding implementation details,
then moved to allow extra -cpu parameters. Then we noticed that the CPU
models were hiding too much from libvirt in the X86 case, and now those
parameters were converted to become QOM properties configurable using
-global.

I expect other architectures to follow a similar pattern and allow
things to be configured using QOM properties, but I am not sure they
would go to the extreme of making every single detail configurable using
QOM.

One thing you may need to worry about for all architectures is to check
if a CPU model is runnable in a host before starting or migrating a VM.
In this case, we're introducing a generic mechanism in
query-cpu-definitions for that. See [PATCH v6 15/17] target-s390x:
Extend arch specific QMP command query-cpu-definitions and related
patches (posted on 2015-04-27) on qemu-devel.

And in the case of runnability checks, x86 is a bit more complex, too
(because it is too configurable in QEMU and in libvirt): as libvirt
needs to know what exactly is blocking the CPU from running, we have the
filtered-features property (that libvirt can start using, now that we
have the qom-path field on query-cpus).

-- 
Eduardo



Re: [Qemu-devel] [PATCH 0/2] target-i386: custom CPU model + script to dump existing CPU models

2015-06-08 Thread Jiri Denemark
On Mon, Jun 08, 2015 at 16:07:38 -0300, Eduardo Habkost wrote:
...
 libvirt can solve this problem partially by making sure every feature in a CPU
 model is explicitly configured, instead of (incorrectly) expecting that a 
 named
 CPU model will never change in QEMU. But this doesn't solve the problem
 completely, because it is still possible that new features unknown to libvirt
 get enabled in the default CPU model in future machine-types (that's very
 likely to happen when we introduce new KVM features, for example).
 
 So, to make sure no new feature will be ever enabled without the knowledge of
 libvirt, add a -cpu custom mode, where no CPU model data is loaded at all,
 and everything needs to be configured explicitly using CPU properties. That
 means no CPU features will ever change depending on machine-type or 
 accelerator
 capabilities when using -cpu custom.
 
   * * *
 
 I know that this is basically the opposite of what we were aiming at in the
 last few month^Wyears, where we were struggling to implement probing 
 interfaces
 to expose machine-type-dependent data for libvirt. But, at least the fact that
 we could implement the new approach using a 9-line patch means we were still
 going in the right direction. :)
 
 To help libvirt in the transition, a x86-cpu-model-dump script is provided,
 that will generate a config file that can be loaded using -readconfig, based 
 on
 the -cpu and -machine options provided in the command-line.

Thanks Eduardo, I never was a big fan of moving (or copying) all the CPU
configuration data to libvirt, but now I think it actually makes sense.
We already have a partial copy of CPU model definitions in libvirt
anyway, but as QEMU changes some CPU models in some machine types (and
libvirt does not do that) we have no real control over the guest CPU
configuration. While what we really want is full control to enforce
stable guest ABI.

I will summarize my ideas on how libvirt should be using -cpu custom and
send them to libvirt-list soon.

Jirka