Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-12 Thread Kevin Wolf
Am 11.03.2013 um 19:03 hat Ján Tomko geschrieben:
 On 03/04/2013 04:40 PM, Kevin Wolf wrote:
  Am 04.03.2013 um 16:19 hat Daniel P. Berrange geschrieben:
  On Mon, Mar 04, 2013 at 04:05:50PM +0100, Kevin Wolf wrote:
 
  I'm not talking about the QEMU cli, but about qcow2 as the format as
  defined in the spec (which just happens to sit in qemu.git, but isn't
  qemu specific at all)
 
  So you're saying that you consider the format name to be qcow2 regardless
  of whether the version numer field is specified as 2 or 3.
  
  Yes.
 
  So in other words, if an application came along and required libvirt to
  set  format=qcow3 on its CLI, we could justifiably refuse to do that in
  libvirt claiming this is not in compliance with the spec ?
  
  No, you would just check which features this image uses (which, if I
  understood correctly, you need to save anyway), and if a version 3
  feature is among it (the basic version 3 could be represented by either
  a feature flags or zero clusters feature, which are what version 3
  really means), pass it the 'qcow3' command line option it wants.
  
 
 Since libvirt needs to know the feature names, I doesn't seem to be a
 problem to know what compat options they need. And the empty (but
 present) features element could just mean compat=1.1.

Sounds like an option if this isn't too much magic for you. If we ever
introduced a compat=2.0, would this be represented by a specific child
element of features?

 Would we still
 need to support creating compat=0.10 images with older qemu-img not
 understanding this option after the default gets changed in the current
 version?

If you want to create images that can be read by versions  1.1, you
need it. I think it's a reasonable expectation that libvirt allows this.

That said, I'm not sure when we should switch the default in qemu. It's
probably too early in 1.5, but we might consider it for 1.6.

  Of course, I would be disappointed that the tool thought it had to
  invent format names, but it's not really blocking any functionality.
  
  Just the same way it could happen that a tool uses different drivers for
  other features that we introduce. For example, imagine that we introduce
  a flag that modifies the L2 table structure to allow subclusters - a
  change that we've been discussing before and that would have a massive
  impact on the implementation, even though it's only a feature flag that
  has changed, and not the version number. Using a different driver for
  this looks much more likely than a different driver for version 2 and 3,
  which was really a quite small step.
 
 So, the format and the driver is still 'qcow2' now, there's no need to
 translate anything at the moment (apart from features-options when
 creating the image).
 
 But if the same 'qcow2' format would need different drivers based on the
 features, we need to use different set of values for driver types and
 image formats, if the features would not be in the domain XML.

What driver is used to deal with the format is an implementation detail,
it's not something mandated by the format. This is why some tools have
one driver for both qcow1 and qcow2, and others like qemu have separate
ones.

Kevin



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-11 Thread Ján Tomko
On 03/04/2013 04:40 PM, Kevin Wolf wrote:
 Am 04.03.2013 um 16:19 hat Daniel P. Berrange geschrieben:
 On Mon, Mar 04, 2013 at 04:05:50PM +0100, Kevin Wolf wrote:

 I'm not talking about the QEMU cli, but about qcow2 as the format as
 defined in the spec (which just happens to sit in qemu.git, but isn't
 qemu specific at all)

 So you're saying that you consider the format name to be qcow2 regardless
 of whether the version numer field is specified as 2 or 3.
 
 Yes.

 So in other words, if an application came along and required libvirt to
 set  format=qcow3 on its CLI, we could justifiably refuse to do that in
 libvirt claiming this is not in compliance with the spec ?
 
 No, you would just check which features this image uses (which, if I
 understood correctly, you need to save anyway), and if a version 3
 feature is among it (the basic version 3 could be represented by either
 a feature flags or zero clusters feature, which are what version 3
 really means), pass it the 'qcow3' command line option it wants.
 

Since libvirt needs to know the feature names, I doesn't seem to be a
problem to know what compat options they need. And the empty (but
present) features element could just mean compat=1.1. Would we still
need to support creating compat=0.10 images with older qemu-img not
understanding this option after the default gets changed in the current
version?

 Of course, I would be disappointed that the tool thought it had to
 invent format names, but it's not really blocking any functionality.
 
 Just the same way it could happen that a tool uses different drivers for
 other features that we introduce. For example, imagine that we introduce
 a flag that modifies the L2 table structure to allow subclusters - a
 change that we've been discussing before and that would have a massive
 impact on the implementation, even though it's only a feature flag that
 has changed, and not the version number. Using a different driver for
 this looks much more likely than a different driver for version 2 and 3,
 which was really a quite small step.

So, the format and the driver is still 'qcow2' now, there's no need to
translate anything at the moment (apart from features-options when
creating the image).

But if the same 'qcow2' format would need different drivers based on the
features, we need to use different set of values for driver types and
image formats, if the features would not be in the domain XML.

 
 So the main problem that I see is the assumption different version =
 big change, new feature flag = small change and as a conclusion from
 that different version = possibly new driver, new feature flag =
 definitely only old driver. This isn't true at all.
 
 Kevin
 





[Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Ján Tomko
Before posting another version of my patches [1], attempting to add
support for the new qcow format to libvirt, I would like to know if this
sounds reasonable:

A new format named 'qcow3' would be added, along with a features
sub-element for target.

volume
  nameqcow3test/name
  source
  /source
  capacity unit='GiB'8/capacity
  target
path/var/lib/libvirt/images/qcow3test/path
format type='qcow3'/
features
  lazy_refcounts/
/features
  /target
/volume

I think that libvirt shouldn't care if the features are compatible or
incompatible, as we don't know what features are supported by the
hypervisor. Would the features be any good as tri-state (on, off, default?).

While the qcow3 format is handled by the qcow2 driver in QEMU,
driver name='qemu' type='qcow2'/ should be enough for domains,
but in snapshot XML we treat the driver type as the format:

disk name='/path/to/old'
  driver type='qcow3'/
  source file='/path/to/new'/
  features
lazy_refcounts/
  /features
/disk

So I think we should allow the qcow3 driver type as well and translate
it to qcow2 for QEMU.

Jan

[1] v2 here:
https://www.redhat.com/archives/libvir-list/2013-February/msg00212.html



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Daniel P. Berrange
On Mon, Mar 04, 2013 at 01:58:12PM +0100, Ján Tomko wrote:
 Before posting another version of my patches [1], attempting to add
 support for the new qcow format to libvirt, I would like to know if this
 sounds reasonable:
 
 A new format named 'qcow3' would be added, along with a features
 sub-element for target.
 
 volume
   nameqcow3test/name
   source
   /source
   capacity unit='GiB'8/capacity
   target
 path/var/lib/libvirt/images/qcow3test/path
 format type='qcow3'/
 features
   lazy_refcounts/
 /features
   /target
 /volume
 
 I think that libvirt shouldn't care if the features are compatible or
 incompatible, as we don't know what features are supported by the
 hypervisor. Would the features be any good as tri-state (on, off, default?).
 
 While the qcow3 format is handled by the qcow2 driver in QEMU,
 driver name='qemu' type='qcow2'/ should be enough for domains,

We should use qcow3 everywhere IMHO, regardless of whether qcow2
technically works in this context.

 but in snapshot XML we treat the driver type as the format:
 
 disk name='/path/to/old'
   driver type='qcow3'/
   source file='/path/to/new'/
   features
 lazy_refcounts/
   /features
 /disk
 
 So I think we should allow the qcow3 driver type as well and translate
 it to qcow2 for QEMU.
 
 Jan
 
 [1] v2 here:
 https://www.redhat.com/archives/libvir-list/2013-February/msg00212.html
 

Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Kevin Wolf
Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben:
 On Mon, Mar 04, 2013 at 01:58:12PM +0100, Ján Tomko wrote:
  Before posting another version of my patches [1], attempting to add
  support for the new qcow format to libvirt, I would like to know if this
  sounds reasonable:
  
  A new format named 'qcow3' would be added, along with a features
  sub-element for target.
  
  volume
nameqcow3test/name
source
/source
capacity unit='GiB'8/capacity
target
  path/var/lib/libvirt/images/qcow3test/path
  format type='qcow3'/
  features
lazy_refcounts/
  /features
/target
  /volume
  
  I think that libvirt shouldn't care if the features are compatible or
  incompatible, as we don't know what features are supported by the
  hypervisor. Would the features be any good as tri-state (on, off, default?).
  
  While the qcow3 format is handled by the qcow2 driver in QEMU,
  driver name='qemu' type='qcow2'/ should be enough for domains,
 
 We should use qcow3 everywhere IMHO, regardless of whether qcow2
 technically works in this context.

I think it makes much more sense to deal with it the way qemu does
instead of inventing new names. This has much more of an (incompatible)
feature flag than of a different image format. So to fit it in your
proposed syntax:

target
  path/var/lib/libvirt/images/qcow3test/path
  format type='qcow2'/
  features
compat version=1.1 /
lazy_refcounts/
  /features
/target

Or if you really think that you should refer to the inner workings of
qcow2, you can make it version3/version.

But I guess you call all VMDKs just vmdk, despite the fact that they
are really just a collection of different subformats. Right?

Kevin



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Daniel P. Berrange
On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote:
 Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben:
  On Mon, Mar 04, 2013 at 01:58:12PM +0100, Ján Tomko wrote:
   Before posting another version of my patches [1], attempting to add
   support for the new qcow format to libvirt, I would like to know if this
   sounds reasonable:
   
   A new format named 'qcow3' would be added, along with a features
   sub-element for target.
   
   volume
 nameqcow3test/name
 source
 /source
 capacity unit='GiB'8/capacity
 target
   path/var/lib/libvirt/images/qcow3test/path
   format type='qcow3'/
   features
 lazy_refcounts/
   /features
 /target
   /volume
   
   I think that libvirt shouldn't care if the features are compatible or
   incompatible, as we don't know what features are supported by the
   hypervisor. Would the features be any good as tri-state (on, off, 
   default?).
   
   While the qcow3 format is handled by the qcow2 driver in QEMU,
   driver name='qemu' type='qcow2'/ should be enough for domains,
  
  We should use qcow3 everywhere IMHO, regardless of whether qcow2
  technically works in this context.
 
 I think it makes much more sense to deal with it the way qemu does
 instead of inventing new names. This has much more of an (incompatible)
 feature flag than of a different image format. So to fit it in your
 proposed syntax:

The issue is that QEMU is not the only thing that implements the qcow
format. There are a number of other impls out there, and we can't just
assume that they will all be providing a qcow2 driver that automagically
opens a qcow3 image format. Just in the same way we didn't assume that
a 'qcow' (version 1) driver would open a version 2 image.

It so happens that with QEMU if you specify format=qcow2 and give it
a qcow3 image, QEMU will open it, but libvirt can't assume that, since
this is a mere implementation detail. Hence libvirt must explicitly
refer to 'qcow3' in the XML and map it to qcow2 if applicable when
talking to QEMU.

 But I guess you call all VMDKs just vmdk, despite the fact that they
 are really just a collection of different subformats. Right?

Yes, but that is really a bug in our representation of vmdk.

Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Kevin Wolf
Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben:
 On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote:
  Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben:
   On Mon, Mar 04, 2013 at 01:58:12PM +0100, Ján Tomko wrote:
Before posting another version of my patches [1], attempting to add
support for the new qcow format to libvirt, I would like to know if this
sounds reasonable:

A new format named 'qcow3' would be added, along with a features
sub-element for target.

volume
  nameqcow3test/name
  source
  /source
  capacity unit='GiB'8/capacity
  target
path/var/lib/libvirt/images/qcow3test/path
format type='qcow3'/
features
  lazy_refcounts/
/features
  /target
/volume

I think that libvirt shouldn't care if the features are compatible or
incompatible, as we don't know what features are supported by the
hypervisor. Would the features be any good as tri-state (on, off, 
default?).

While the qcow3 format is handled by the qcow2 driver in QEMU,
driver name='qemu' type='qcow2'/ should be enough for domains,
   
   We should use qcow3 everywhere IMHO, regardless of whether qcow2
   technically works in this context.
  
  I think it makes much more sense to deal with it the way qemu does
  instead of inventing new names. This has much more of an (incompatible)
  feature flag than of a different image format. So to fit it in your
  proposed syntax:
 
 The issue is that QEMU is not the only thing that implements the qcow
 format. There are a number of other impls out there, and we can't just
 assume that they will all be providing a qcow2 driver that automagically
 opens a qcow3 image format. Just in the same way we didn't assume that
 a 'qcow' (version 1) driver would open a version 2 image.

That's true. Other implementation actually tend to have a 'qcow' driver
that deals with both qcow1 and qcow2. But these two are actually
different enough that calling them two different formats might be
acceptable.

In contrast, version 3 images share _exactly_ the same structure with
version 2 images, the just have additional header fields and support
some new flags in some structures (that were previously reserved).

If you call this a different image format, then scratch that whole
feature idea, because then each newly added feature is a new image
format by your standards.

 It so happens that with QEMU if you specify format=qcow2 and give it
 a qcow3 image, QEMU will open it, but libvirt can't assume that, since
 this is a mere implementation detail. Hence libvirt must explicitly
 refer to 'qcow3' in the XML and map it to qcow2 if applicable when
 talking to QEMU.

If you need this information, sure, save it. I'm just requesting that
you don't abuse the format name for it.

  But I guess you call all VMDKs just vmdk, despite the fact that they
  are really just a collection of different subformats. Right?
 
 Yes, but that is really a bug in our representation of vmdk.

How are you going to fix it? Do you think having ten different format
names all starting with vmdk will make tools user friendly?

Kevin



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Daniel P. Berrange
On Mon, Mar 04, 2013 at 03:38:54PM +0100, Kevin Wolf wrote:
 Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben:
  On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote:
   Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben:
   I think it makes much more sense to deal with it the way qemu does
   instead of inventing new names. This has much more of an (incompatible)
   feature flag than of a different image format. So to fit it in your
   proposed syntax:
  
  The issue is that QEMU is not the only thing that implements the qcow
  format. There are a number of other impls out there, and we can't just
  assume that they will all be providing a qcow2 driver that automagically
  opens a qcow3 image format. Just in the same way we didn't assume that
  a 'qcow' (version 1) driver would open a version 2 image.
 
 That's true. Other implementation actually tend to have a 'qcow' driver
 that deals with both qcow1 and qcow2. But these two are actually
 different enough that calling them two different formats might be
 acceptable.
 
 In contrast, version 3 images share _exactly_ the same structure with
 version 2 images, the just have additional header fields and support
 some new flags in some structures (that were previously reserved).
 
 If you call this a different image format, then scratch that whole
 feature idea, because then each newly added feature is a new image
 format by your standards.

No, that's not what I'm saying. The version 3 image format introduces
the ability to set a variety of features in an extensible way. Adding
new features to that list doesn't mean the version has changed.

 
  It so happens that with QEMU if you specify format=qcow2 and give it
  a qcow3 image, QEMU will open it, but libvirt can't assume that, since
  this is a mere implementation detail. Hence libvirt must explicitly
  refer to 'qcow3' in the XML and map it to qcow2 if applicable when
  talking to QEMU.
 
 If you need this information, sure, save it. I'm just requesting that
 you don't abuse the format name for it.

The key distinction is that libvirt XML is recording an generic image
format, while the QEMU cli args are referring to a specific driver
implementation, which are support multiple formats. Typically these
map 1-to-1, but there's no such requirement in general. Hence will
refer to 'qcow3' in all its XML descriptions, and map to 'qcow2' when
talking to QEMU, or even just to 'qcow' if talking to a different impl
which supports all 3 versions in one driver.

   But I guess you call all VMDKs just vmdk, despite the fact that they
   are really just a collection of different subformats. Right?
  
  Yes, but that is really a bug in our representation of vmdk.
 
 How are you going to fix it? Do you think having ten different format
 names all starting with vmdk will make tools user friendly?

Well we can't really fix it now, given we've got tools relying on
this naming, but use we ought to have numbered the vmdk formats in
retrospect.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Kevin Wolf
Am 04.03.2013 um 15:46 hat Daniel P. Berrange geschrieben:
 On Mon, Mar 04, 2013 at 03:38:54PM +0100, Kevin Wolf wrote:
  Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben:
   On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote:
Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben:
I think it makes much more sense to deal with it the way qemu does
instead of inventing new names. This has much more of an (incompatible)
feature flag than of a different image format. So to fit it in your
proposed syntax:
   
   The issue is that QEMU is not the only thing that implements the qcow
   format. There are a number of other impls out there, and we can't just
   assume that they will all be providing a qcow2 driver that automagically
   opens a qcow3 image format. Just in the same way we didn't assume that
   a 'qcow' (version 1) driver would open a version 2 image.
  
  That's true. Other implementation actually tend to have a 'qcow' driver
  that deals with both qcow1 and qcow2. But these two are actually
  different enough that calling them two different formats might be
  acceptable.
  
  In contrast, version 3 images share _exactly_ the same structure with
  version 2 images, the just have additional header fields and support
  some new flags in some structures (that were previously reserved).
  
  If you call this a different image format, then scratch that whole
  feature idea, because then each newly added feature is a new image
  format by your standards.
 
 No, that's not what I'm saying. The version 3 image format introduces
 the ability to set a variety of features in an extensible way. Adding
 new features to that list doesn't mean the version has changed.

Why does libvirt care whether a new feature is indicated by incrementing
one header field or by setting a bit in a different header field? These
are image format internals, not external interfaces.

   It so happens that with QEMU if you specify format=qcow2 and give it
   a qcow3 image, QEMU will open it, but libvirt can't assume that, since
   this is a mere implementation detail. Hence libvirt must explicitly
   refer to 'qcow3' in the XML and map it to qcow2 if applicable when
   talking to QEMU.
  
  If you need this information, sure, save it. I'm just requesting that
  you don't abuse the format name for it.
 
 The key distinction is that libvirt XML is recording an generic image
 format, while the QEMU cli args are referring to a specific driver
 implementation, which are support multiple formats. Typically these
 map 1-to-1, but there's no such requirement in general. Hence will
 refer to 'qcow3' in all its XML descriptions, and map to 'qcow2' when
 talking to QEMU, or even just to 'qcow' if talking to a different impl
 which supports all 3 versions in one driver.

I'm not talking about the QEMU cli, but about qcow2 as the format as
defined in the spec (which just happens to sit in qemu.git, but isn't
qemu specific at all)

But I guess you call all VMDKs just vmdk, despite the fact that they
are really just a collection of different subformats. Right?
   
   Yes, but that is really a bug in our representation of vmdk.
  
  How are you going to fix it? Do you think having ten different format
  names all starting with vmdk will make tools user friendly?
 
 Well we can't really fix it now, given we've got tools relying on
 this naming, but use we ought to have numbered the vmdk formats in
 retrospect.

Do you actually support more than one subformat today that they could
rely on? It's probably only important for creating images anyway as all
backends implement all VMDK subformats using the same driver.

Anyway, just imagine that these tools didn't exist yet. I still think
that giving the user ten different vmdk* formats to choose from would
be the wrong way - and isn't this how it would end up if libvirt treated
it as different formats? The user should select VMDK and then get
additional options that apply to this specific format. The same way the
user should select qcow2 (or even just qcow) and then all options
that are available for it, including version 3 images.

Kevin



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Daniel P. Berrange
On Mon, Mar 04, 2013 at 04:05:50PM +0100, Kevin Wolf wrote:
 Am 04.03.2013 um 15:46 hat Daniel P. Berrange geschrieben:
  On Mon, Mar 04, 2013 at 03:38:54PM +0100, Kevin Wolf wrote:
   Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben:
On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote:
 Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben:
 I think it makes much more sense to deal with it the way qemu does
 instead of inventing new names. This has much more of an 
 (incompatible)
 feature flag than of a different image format. So to fit it in your
 proposed syntax:

The issue is that QEMU is not the only thing that implements the qcow
format. There are a number of other impls out there, and we can't just
assume that they will all be providing a qcow2 driver that automagically
opens a qcow3 image format. Just in the same way we didn't assume that
a 'qcow' (version 1) driver would open a version 2 image.
   
   That's true. Other implementation actually tend to have a 'qcow' driver
   that deals with both qcow1 and qcow2. But these two are actually
   different enough that calling them two different formats might be
   acceptable.
   
   In contrast, version 3 images share _exactly_ the same structure with
   version 2 images, the just have additional header fields and support
   some new flags in some structures (that were previously reserved).
   
   If you call this a different image format, then scratch that whole
   feature idea, because then each newly added feature is a new image
   format by your standards.
  
  No, that's not what I'm saying. The version 3 image format introduces
  the ability to set a variety of features in an extensible way. Adding
  new features to that list doesn't mean the version has changed.
 
 Why does libvirt care whether a new feature is indicated by incrementing
 one header field or by setting a bit in a different header field? These
 are image format internals, not external interfaces.
 
It so happens that with QEMU if you specify format=qcow2 and give it
a qcow3 image, QEMU will open it, but libvirt can't assume that, since
this is a mere implementation detail. Hence libvirt must explicitly
refer to 'qcow3' in the XML and map it to qcow2 if applicable when
talking to QEMU.
   
   If you need this information, sure, save it. I'm just requesting that
   you don't abuse the format name for it.
  
  The key distinction is that libvirt XML is recording an generic image
  format, while the QEMU cli args are referring to a specific driver
  implementation, which are support multiple formats. Typically these
  map 1-to-1, but there's no such requirement in general. Hence will
  refer to 'qcow3' in all its XML descriptions, and map to 'qcow2' when
  talking to QEMU, or even just to 'qcow' if talking to a different impl
  which supports all 3 versions in one driver.
 
 I'm not talking about the QEMU cli, but about qcow2 as the format as
 defined in the spec (which just happens to sit in qemu.git, but isn't
 qemu specific at all)

So you're saying that you consider the format name to be qcow2 regardless
of whether the version numer field is specified as 2 or 3.

So in other words, if an application came along and required libvirt to
set  format=qcow3 on its CLI, we could justifiably refuse to do that in
libvirt claiming this is not in compliance with the spec ?  This is my
big concern.  If we go with 'format=qcow2' in the XML and we did ever
hit a case where we needed to distinguish versions, we'd not have enough
info in the XML todo that. If you are willing so that that such a scenario
is not spec compliant, then I'll be ok using just qcow2 in the libvirt XML
for this.

It would be nice if the spec explicitly stated that the format should be
referred to by any implementation as 'qcow2' regardles of version number
being 2 or 3.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|



Re: [Qemu-devel] [RFC] qcow3 format in libvirt

2013-03-04 Thread Kevin Wolf
Am 04.03.2013 um 16:19 hat Daniel P. Berrange geschrieben:
 On Mon, Mar 04, 2013 at 04:05:50PM +0100, Kevin Wolf wrote:
  Am 04.03.2013 um 15:46 hat Daniel P. Berrange geschrieben:
   On Mon, Mar 04, 2013 at 03:38:54PM +0100, Kevin Wolf wrote:
Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben:
 It so happens that with QEMU if you specify format=qcow2 and give it
 a qcow3 image, QEMU will open it, but libvirt can't assume that, since
 this is a mere implementation detail. Hence libvirt must explicitly
 refer to 'qcow3' in the XML and map it to qcow2 if applicable when
 talking to QEMU.

If you need this information, sure, save it. I'm just requesting that
you don't abuse the format name for it.
   
   The key distinction is that libvirt XML is recording an generic image
   format, while the QEMU cli args are referring to a specific driver
   implementation, which are support multiple formats. Typically these
   map 1-to-1, but there's no such requirement in general. Hence will
   refer to 'qcow3' in all its XML descriptions, and map to 'qcow2' when
   talking to QEMU, or even just to 'qcow' if talking to a different impl
   which supports all 3 versions in one driver.
  
  I'm not talking about the QEMU cli, but about qcow2 as the format as
  defined in the spec (which just happens to sit in qemu.git, but isn't
  qemu specific at all)
 
 So you're saying that you consider the format name to be qcow2 regardless
 of whether the version numer field is specified as 2 or 3.

Yes.

 So in other words, if an application came along and required libvirt to
 set  format=qcow3 on its CLI, we could justifiably refuse to do that in
 libvirt claiming this is not in compliance with the spec ?

No, you would just check which features this image uses (which, if I
understood correctly, you need to save anyway), and if a version 3
feature is among it (the basic version 3 could be represented by either
a feature flags or zero clusters feature, which are what version 3
really means), pass it the 'qcow3' command line option it wants.

Of course, I would be disappointed that the tool thought it had to
invent format names, but it's not really blocking any functionality.

Just the same way it could happen that a tool uses different drivers for
other features that we introduce. For example, imagine that we introduce
a flag that modifies the L2 table structure to allow subclusters - a
change that we've been discussing before and that would have a massive
impact on the implementation, even though it's only a feature flag that
has changed, and not the version number. Using a different driver for
this looks much more likely than a different driver for version 2 and 3,
which was really a quite small step.

So the main problem that I see is the assumption different version =
big change, new feature flag = small change and as a conclusion from
that different version = possibly new driver, new feature flag =
definitely only old driver. This isn't true at all.

Kevin