Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-24 Thread Arik Hadas
Hi Kevin,

That's the thread I told you about
Can you please sound your opinion on this?

Thanks,
Arik

- Original Message -
 
 
 - Original Message -
  On Sun, Jun 23, 2013 at 06:24:50AM -0400, Ayal Baron wrote:
   
   
   - Original Message -
On Sun, Jun 23, 2013 at 04:44:19AM -0400, Ayal Baron wrote:
 
 
 - Original Message -
  
  - Original Message -
   VDSM sync meeting Monday June 17th 2013
   ===
   
   - Dan requests reviews for pending patches from Mei Liu and
   himself
   (
 http://gerrit.ovirt.org/#/c/15356/8 ) for mom.
   
   - Toni to fix the problem with Mark's patch that Dan reviewed so
   that
   it
   can
 be merged together with the smaller following patches.
   
   - Federico has three high priority patches for features and two
   lower
   priority
 ones that need reviews in order to make it for the release
 deadline.
 They
 are
 related to disk extension and image upload/download (glance):
 + http://gerrit.ovirt.org/#/c/15442 - constants: unify the
 BLANK_UUID
 def...
 + http://gerrit.ovirt.org/#/c/14589 - volume: add the
 BLOCK_SIZE
 constant
 +1
 + http://gerrit.ovirt.org/#/c/14590 - volume: add the
 extendVolumeSize
 method
 + http://gerrit.ovirt.org/#/c/15614 - vm: add the live
 diskSizeExtend
 method +1
 + http://gerrit.ovirt.org/#/c/14955 - image: add support to
 upload/download
 images
   
   - Adam and others to review Gluster patches:
 + http://gerrit.ovirt.org/#/c/13785/
 + http://gerrit.ovirt.org/#/c/11094/
   
   - Is there a filing for Qemu image handling reported by virt
   team?
  
  Background about the problem:
  
  As part of the ram snapshots feature
  (http://wiki.ovirt.org/Features/RAM_Snapshots)
  we pass a path to file to libvirt, where the memory of the VM
  should
  be
  saved
  as
  part of the snapshot creation.
  the output file is identical to the one that is created to store
  the
  memory
  on
  hibernate command.
  
  When VM with snapshot(s) that has memory is being exported, we want
  the
  snapshot's
  memory file to be exported to the export domain as well. when
  trying
  to
  export such
  memory file, we saw that the file is being truncated to the closest
  size
  that
  is
  a multiple of block size (512). we found that the file is being
  truncated
  by
  qemu-img that is being used to copy the file to the export domain.
  
  We previously thought that it is a problem in qemu - not handling
  files
  with
  size
  that is not a multiple of block size, and a bug to qemu was opened
  (bz
  970559).
  
  We now understand that the issue described above might be a problem
  in
  qemu
  that
  needs to be solved, but it is not the problem in our case:
  treating the memory file as qemu image is wrong - the created file
  is
  not
  a
  qemu
  image. the file has a special format: it contains xml part at the
  beginning
  and
  the memory dump as a binary data right after that.
 
 This is not correct.
 qemu-img is a utility to manage images.

And here, we are trying to use it to copy something that is not a qemu
image.
   
   This is a file created by qemu/libvirt and represents part of the VM.
   If that is not a qemu image then I'm not sure what is.
  
  The file is created by libvirt, concatenating VM memory dump to its
  metadata. That something else to the qemu disk images which qemu-img was
  designed for (and named after).
 
 A raw file is a raw file, the data inside it is irrelevant.
 This is part of the VM.
 qemu-img was also not originally designed for block devices, nor was qemu...
 
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-23 Thread Eduardo Warszawski


- Original Message -
 On 06/18/2013 09:29 AM, Arik Hadas wrote:
 
  - Original Message -
  VDSM sync meeting Monday June 17th 2013
  ===
 
  - Dan requests reviews for pending patches from Mei Liu and himself (
 http://gerrit.ovirt.org/#/c/15356/8 ) for mom.
 
  - Toni to fix the problem with Mark's patch that Dan reviewed so that it
  can
 be merged together with the smaller following patches.
 
  - Federico has three high priority patches for features and two lower
  priority
 ones that need reviews in order to make it for the release deadline.
 They
 are
 related to disk extension and image upload/download (glance):
 + http://gerrit.ovirt.org/#/c/15442 - constants: unify the BLANK_UUID
 def...
 + http://gerrit.ovirt.org/#/c/14589 - volume: add the BLOCK_SIZE
 constant
 +1
 + http://gerrit.ovirt.org/#/c/14590 - volume: add the extendVolumeSize
 method
 + http://gerrit.ovirt.org/#/c/15614 - vm: add the live diskSizeExtend
 method +1
 + http://gerrit.ovirt.org/#/c/14955 - image: add support to
 upload/download
 images
 
  - Adam and others to review Gluster patches:
 + http://gerrit.ovirt.org/#/c/13785/
 + http://gerrit.ovirt.org/#/c/11094/
 
  - Is there a filing for Qemu image handling reported by virt team?
 
  Background about the problem:
 
  As part of the ram snapshots feature
  (http://wiki.ovirt.org/Features/RAM_Snapshots)
  we pass a path to file to libvirt, where the memory of the VM should be
  saved as
  part of the snapshot creation.
  the output file is identical to the one that is created to store the memory
  on
  hibernate command.
 
  When VM with snapshot(s) that has memory is being exported, we want the
  snapshot's
  memory file to be exported to the export domain as well. when trying to
  export such
  memory file, we saw that the file is being truncated to the closest size
  that is
  a multiple of block size (512). we found that the file is being truncated
  by
  qemu-img that is being used to copy the file to the export domain.
 
  We previously thought that it is a problem in qemu - not handling files
  with size
  that is not a multiple of block size, and a bug to qemu was opened (bz
  970559).
 
  We now understand that the issue described above might be a problem in qemu
  that
  needs to be solved, but it is not the problem in our case:
  treating the memory file as qemu image is wrong - the created file is not a
  qemu
  image. the file has a special format: it contains xml part at the beginning
  and
  the memory dump as a binary data right after that.
 
  So basically it seems that we need the following things in vdsm:
 
  1. The creation of the file that is expected to store the memory as qemu
  image
  seems to be redundant. instead, we need to have a way to create a file in a
  similar way to 'touch' command - just to verify that the file in the path
  we
  pass to libvirt is created with the right permissions. (it's probably true
  for
  the hibernation volume creation as well btw).
 
 how would this look for block devices?
It is working as expected on block devices (tested).
When dumping the memory the file, libvirt is doing O_TRUNC and the resultant
file have a random lenght which can be odd. This is erronously treated by the
qemu-img utility.

 
 
  2. The more important issue is that we need to have a way to export files
  which
  are not qemu images to the export domain, which means without using the
  qemu-img
  process, just copy the file.
qemu-img is capable to deal with raw images, as we are doing today, provided 
that
the files are a block size multiples, as can be expected for a file which is 
used 
like a volume.
Having two export paths can be very cumbersome, we should find a less 
disruptive 
solution.


 
 ayal - anything close to this today - i remember various qemu-img vs. dd
 patches in the past for copy operations?
 
 
 
  - Reminder to follow wiki feature page process closely to maximize changes
  to
 have features added to a release.
 
  - Multiple gateways are being worked on to fix dhcp races and comments by
 reviewers.
 
  - NetReloaded after multiple gateways. Try to get up to iproute2
  configurator.
 
  - Cross-distro support:
 + Netreloaded with iproute2 plus persistence would help.
 + init scripts separation (proposed by Yaniv to make ready for 3.3).
 
  - Cloud-init integration by Greg Padgett. There is a vdsm patch that
  should
  be
 taken care of in terms of reviewing http://gerrit.ovirt.org/#/c/14347/
  ___
  vdsm-devel mailing list
  vdsm-devel@lists.fedorahosted.org
  https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
 
  ___
  vdsm-devel mailing list
  vdsm-devel@lists.fedorahosted.org
  https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
 
 

Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-23 Thread Dan Kenigsberg
On Fri, Jun 21, 2013 at 02:27:39PM +0300, Itamar Heim wrote:
 On 06/18/2013 09:29 AM, Arik Hadas wrote:
 
 - Is there a filing for Qemu image handling reported by virt team?
 
 Background about the problem:
 
 As part of the ram snapshots feature 
 (http://wiki.ovirt.org/Features/RAM_Snapshots)
 we pass a path to file to libvirt, where the memory of the VM should be 
 saved as
 part of the snapshot creation.
 the output file is identical to the one that is created to store the memory 
 on
 hibernate command.
 
 When VM with snapshot(s) that has memory is being exported, we want the 
 snapshot's
 memory file to be exported to the export domain as well. when trying to 
 export such
 memory file, we saw that the file is being truncated to the closest size 
 that is
 a multiple of block size (512). we found that the file is being truncated by
 qemu-img that is being used to copy the file to the export domain.
 
 We previously thought that it is a problem in qemu - not handling files with 
 size
 that is not a multiple of block size, and a bug to qemu was opened (bz 
 970559).
 
 We now understand that the issue described above might be a problem in qemu 
 that
 needs to be solved, but it is not the problem in our case:
 treating the memory file as qemu image is wrong - the created file is not a 
 qemu
 image. the file has a special format: it contains xml part at the beginning 
 and
 the memory dump as a binary data right after that.
 
 So basically it seems that we need the following things in vdsm:
 
 1. The creation of the file that is expected to store the memory as qemu 
 image
 seems to be redundant. instead, we need to have a way to create a file in a
 similar way to 'touch' command - just to verify that the file in the path we
 pass to libvirt is created with the right permissions. (it's probably true 
 for
 the hibernation volume creation as well btw).
 
 how would this look for block devices?
 
 
 2. The more important issue is that we need to have a way to export files 
 which
 are not qemu images to the export domain, which means without using the 
 qemu-img
 process, just copy the file.
 
 ayal - anything close to this today - i remember various qemu-img
 vs. dd patches in the past for copy operations?

Indeed, the problematic qemu-img call was introduced by
http://gerrit.ovirt.org/#/c/10979/5/vdsm/storage/image.py

In my opinion, replacing it with cp (that implies --sparse=auto) would
do the right thing for all combinations of file/block
qemu-disk/snapshot-ram-data.

Regards,
Dan.
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-23 Thread Ayal Baron


- Original Message -
 On Fri, Jun 21, 2013 at 02:27:39PM +0300, Itamar Heim wrote:
  On 06/18/2013 09:29 AM, Arik Hadas wrote:
  
  - Is there a filing for Qemu image handling reported by virt team?
  
  Background about the problem:
  
  As part of the ram snapshots feature
  (http://wiki.ovirt.org/Features/RAM_Snapshots)
  we pass a path to file to libvirt, where the memory of the VM should be
  saved as
  part of the snapshot creation.
  the output file is identical to the one that is created to store the
  memory on
  hibernate command.
  
  When VM with snapshot(s) that has memory is being exported, we want the
  snapshot's
  memory file to be exported to the export domain as well. when trying to
  export such
  memory file, we saw that the file is being truncated to the closest size
  that is
  a multiple of block size (512). we found that the file is being truncated
  by
  qemu-img that is being used to copy the file to the export domain.
  
  We previously thought that it is a problem in qemu - not handling files
  with size
  that is not a multiple of block size, and a bug to qemu was opened (bz
  970559).
  
  We now understand that the issue described above might be a problem in
  qemu that
  needs to be solved, but it is not the problem in our case:
  treating the memory file as qemu image is wrong - the created file is not
  a qemu
  image. the file has a special format: it contains xml part at the
  beginning and
  the memory dump as a binary data right after that.
  
  So basically it seems that we need the following things in vdsm:
  
  1. The creation of the file that is expected to store the memory as qemu
  image
  seems to be redundant. instead, we need to have a way to create a file in
  a
  similar way to 'touch' command - just to verify that the file in the path
  we
  pass to libvirt is created with the right permissions. (it's probably true
  for
  the hibernation volume creation as well btw).
  
  how would this look for block devices?
  
  
  2. The more important issue is that we need to have a way to export files
  which
  are not qemu images to the export domain, which means without using the
  qemu-img
  process, just copy the file.
  
  ayal - anything close to this today - i remember various qemu-img
  vs. dd patches in the past for copy operations?
 
 Indeed, the problematic qemu-img call was introduced by
 http://gerrit.ovirt.org/#/c/10979/5/vdsm/storage/image.py
 
 In my opinion, replacing it with cp (that implies --sparse=auto) would
 do the right thing for all combinations of file/block
 qemu-disk/snapshot-ram-data.

No, for qcow images it would not dedup zeros (and reclaim space where relevant)
It would also not be able to collapse images.

 
 Regards,
 Dan.
 ___
 vdsm-devel mailing list
 vdsm-devel@lists.fedorahosted.org
 https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
 
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-23 Thread Dan Kenigsberg
On Sun, Jun 23, 2013 at 05:29:40AM -0400, Ayal Baron wrote:
 
 
 - Original Message -
  On Fri, Jun 21, 2013 at 02:27:39PM +0300, Itamar Heim wrote:
   On 06/18/2013 09:29 AM, Arik Hadas wrote:
   
   - Is there a filing for Qemu image handling reported by virt team?
   
   Background about the problem:
   
   As part of the ram snapshots feature
   (http://wiki.ovirt.org/Features/RAM_Snapshots)
   we pass a path to file to libvirt, where the memory of the VM should be
   saved as
   part of the snapshot creation.
   the output file is identical to the one that is created to store the
   memory on
   hibernate command.
   
   When VM with snapshot(s) that has memory is being exported, we want the
   snapshot's
   memory file to be exported to the export domain as well. when trying to
   export such
   memory file, we saw that the file is being truncated to the closest size
   that is
   a multiple of block size (512). we found that the file is being truncated
   by
   qemu-img that is being used to copy the file to the export domain.
   
   We previously thought that it is a problem in qemu - not handling files
   with size
   that is not a multiple of block size, and a bug to qemu was opened (bz
   970559).
   
   We now understand that the issue described above might be a problem in
   qemu that
   needs to be solved, but it is not the problem in our case:
   treating the memory file as qemu image is wrong - the created file is not
   a qemu
   image. the file has a special format: it contains xml part at the
   beginning and
   the memory dump as a binary data right after that.
   
   So basically it seems that we need the following things in vdsm:
   
   1. The creation of the file that is expected to store the memory as qemu
   image
   seems to be redundant. instead, we need to have a way to create a file in
   a
   similar way to 'touch' command - just to verify that the file in the path
   we
   pass to libvirt is created with the right permissions. (it's probably 
   true
   for
   the hibernation volume creation as well btw).
   
   how would this look for block devices?
   
   
   2. The more important issue is that we need to have a way to export files
   which
   are not qemu images to the export domain, which means without using the
   qemu-img
   process, just copy the file.
   
   ayal - anything close to this today - i remember various qemu-img
   vs. dd patches in the past for copy operations?
  
  Indeed, the problematic qemu-img call was introduced by
  http://gerrit.ovirt.org/#/c/10979/5/vdsm/storage/image.py
  
  In my opinion, replacing it with cp (that implies --sparse=auto) would
  do the right thing for all combinations of file/block
  qemu-disk/snapshot-ram-data.
 
 No, for qcow images it would not dedup zeros (and reclaim space where 
 relevant)
 It would also not be able to collapse images.

The relevant code runs for RAW file format. I.e., no qcow at the
source and no chains.
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-23 Thread Ayal Baron


- Original Message -
 On Sun, Jun 23, 2013 at 04:44:19AM -0400, Ayal Baron wrote:
  
  
  - Original Message -
   
   - Original Message -
VDSM sync meeting Monday June 17th 2013
===

- Dan requests reviews for pending patches from Mei Liu and himself (
  http://gerrit.ovirt.org/#/c/15356/8 ) for mom.

- Toni to fix the problem with Mark's patch that Dan reviewed so that
it
can
  be merged together with the smaller following patches.

- Federico has three high priority patches for features and two lower
priority
  ones that need reviews in order to make it for the release deadline.
  They
  are
  related to disk extension and image upload/download (glance):
  + http://gerrit.ovirt.org/#/c/15442 - constants: unify the BLANK_UUID
  def...
  + http://gerrit.ovirt.org/#/c/14589 - volume: add the BLOCK_SIZE
  constant
  +1
  + http://gerrit.ovirt.org/#/c/14590 - volume: add the
  extendVolumeSize
  method
  + http://gerrit.ovirt.org/#/c/15614 - vm: add the live diskSizeExtend
  method +1
  + http://gerrit.ovirt.org/#/c/14955 - image: add support to
  upload/download
  images

- Adam and others to review Gluster patches:
  + http://gerrit.ovirt.org/#/c/13785/
  + http://gerrit.ovirt.org/#/c/11094/

- Is there a filing for Qemu image handling reported by virt team?
   
   Background about the problem:
   
   As part of the ram snapshots feature
   (http://wiki.ovirt.org/Features/RAM_Snapshots)
   we pass a path to file to libvirt, where the memory of the VM should be
   saved
   as
   part of the snapshot creation.
   the output file is identical to the one that is created to store the
   memory
   on
   hibernate command.
   
   When VM with snapshot(s) that has memory is being exported, we want the
   snapshot's
   memory file to be exported to the export domain as well. when trying to
   export such
   memory file, we saw that the file is being truncated to the closest size
   that
   is
   a multiple of block size (512). we found that the file is being truncated
   by
   qemu-img that is being used to copy the file to the export domain.
   
   We previously thought that it is a problem in qemu - not handling files
   with
   size
   that is not a multiple of block size, and a bug to qemu was opened (bz
   970559).
   
   We now understand that the issue described above might be a problem in
   qemu
   that
   needs to be solved, but it is not the problem in our case:
   treating the memory file as qemu image is wrong - the created file is not
   a
   qemu
   image. the file has a special format: it contains xml part at the
   beginning
   and
   the memory dump as a binary data right after that.
  
  This is not correct.
  qemu-img is a utility to manage images.
 
 And here, we are trying to use it to copy something that is not a qemu
 image.

This is a file created by qemu/libvirt and represents part of the VM.  If that 
is not a qemu image then I'm not sure what is.


 
  There is no format limitation on images, and the content of this
  file/device is no exception and no reason to treat it as one.  This is a
  raw device/file, we don't and shouldn't care about whether it has xml,
  compression, encryption, qcow, whatever.
  This is an image we pass to libvirt to use in order to dump the memory.
  qemu-img manages raw files/devices and is used to keep sparseness, dedup
  zeros, etc.
 
 It should not be hard to fix qemu-img to work with binary files of any
 length. The only question is whether handling non-qemu-images is the
 job for qemu-img.
 
 Dan.
 
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-23 Thread Ayal Baron


- Original Message -
 On Sun, Jun 23, 2013 at 06:24:50AM -0400, Ayal Baron wrote:
  
  
  - Original Message -
   On Sun, Jun 23, 2013 at 04:44:19AM -0400, Ayal Baron wrote:


- Original Message -
 
 - Original Message -
  VDSM sync meeting Monday June 17th 2013
  ===
  
  - Dan requests reviews for pending patches from Mei Liu and himself
  (
http://gerrit.ovirt.org/#/c/15356/8 ) for mom.
  
  - Toni to fix the problem with Mark's patch that Dan reviewed so
  that
  it
  can
be merged together with the smaller following patches.
  
  - Federico has three high priority patches for features and two
  lower
  priority
ones that need reviews in order to make it for the release
deadline.
They
are
related to disk extension and image upload/download (glance):
+ http://gerrit.ovirt.org/#/c/15442 - constants: unify the
BLANK_UUID
def...
+ http://gerrit.ovirt.org/#/c/14589 - volume: add the BLOCK_SIZE
constant
+1
+ http://gerrit.ovirt.org/#/c/14590 - volume: add the
extendVolumeSize
method
+ http://gerrit.ovirt.org/#/c/15614 - vm: add the live
diskSizeExtend
method +1
+ http://gerrit.ovirt.org/#/c/14955 - image: add support to
upload/download
images
  
  - Adam and others to review Gluster patches:
+ http://gerrit.ovirt.org/#/c/13785/
+ http://gerrit.ovirt.org/#/c/11094/
  
  - Is there a filing for Qemu image handling reported by virt team?
 
 Background about the problem:
 
 As part of the ram snapshots feature
 (http://wiki.ovirt.org/Features/RAM_Snapshots)
 we pass a path to file to libvirt, where the memory of the VM should
 be
 saved
 as
 part of the snapshot creation.
 the output file is identical to the one that is created to store the
 memory
 on
 hibernate command.
 
 When VM with snapshot(s) that has memory is being exported, we want
 the
 snapshot's
 memory file to be exported to the export domain as well. when trying
 to
 export such
 memory file, we saw that the file is being truncated to the closest
 size
 that
 is
 a multiple of block size (512). we found that the file is being
 truncated
 by
 qemu-img that is being used to copy the file to the export domain.
 
 We previously thought that it is a problem in qemu - not handling
 files
 with
 size
 that is not a multiple of block size, and a bug to qemu was opened
 (bz
 970559).
 
 We now understand that the issue described above might be a problem
 in
 qemu
 that
 needs to be solved, but it is not the problem in our case:
 treating the memory file as qemu image is wrong - the created file is
 not
 a
 qemu
 image. the file has a special format: it contains xml part at the
 beginning
 and
 the memory dump as a binary data right after that.

This is not correct.
qemu-img is a utility to manage images.
   
   And here, we are trying to use it to copy something that is not a qemu
   image.
  
  This is a file created by qemu/libvirt and represents part of the VM.
  If that is not a qemu image then I'm not sure what is.
 
 The file is created by libvirt, concatenating VM memory dump to its
 metadata. That something else to the qemu disk images which qemu-img was
 designed for (and named after).

A raw file is a raw file, the data inside it is irrelevant.
This is part of the VM.
qemu-img was also not originally designed for block devices, nor was qemu...
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-21 Thread Itamar Heim

On 06/18/2013 09:29 AM, Arik Hadas wrote:


- Original Message -

VDSM sync meeting Monday June 17th 2013
===

- Dan requests reviews for pending patches from Mei Liu and himself (
   http://gerrit.ovirt.org/#/c/15356/8 ) for mom.

- Toni to fix the problem with Mark's patch that Dan reviewed so that it can
   be merged together with the smaller following patches.

- Federico has three high priority patches for features and two lower
priority
   ones that need reviews in order to make it for the release deadline. They
   are
   related to disk extension and image upload/download (glance):
   + http://gerrit.ovirt.org/#/c/15442 - constants: unify the BLANK_UUID
   def...
   + http://gerrit.ovirt.org/#/c/14589 - volume: add the BLOCK_SIZE constant
   +1
   + http://gerrit.ovirt.org/#/c/14590 - volume: add the extendVolumeSize
   method
   + http://gerrit.ovirt.org/#/c/15614 - vm: add the live diskSizeExtend
   method +1
   + http://gerrit.ovirt.org/#/c/14955 - image: add support to upload/download
   images

- Adam and others to review Gluster patches:
   + http://gerrit.ovirt.org/#/c/13785/
   + http://gerrit.ovirt.org/#/c/11094/

- Is there a filing for Qemu image handling reported by virt team?


Background about the problem:

As part of the ram snapshots feature 
(http://wiki.ovirt.org/Features/RAM_Snapshots)
we pass a path to file to libvirt, where the memory of the VM should be saved as
part of the snapshot creation.
the output file is identical to the one that is created to store the memory on
hibernate command.

When VM with snapshot(s) that has memory is being exported, we want the 
snapshot's
memory file to be exported to the export domain as well. when trying to export 
such
memory file, we saw that the file is being truncated to the closest size that is
a multiple of block size (512). we found that the file is being truncated by
qemu-img that is being used to copy the file to the export domain.

We previously thought that it is a problem in qemu - not handling files with 
size
that is not a multiple of block size, and a bug to qemu was opened (bz 970559).

We now understand that the issue described above might be a problem in qemu that
needs to be solved, but it is not the problem in our case:
treating the memory file as qemu image is wrong - the created file is not a qemu
image. the file has a special format: it contains xml part at the beginning and
the memory dump as a binary data right after that.

So basically it seems that we need the following things in vdsm:

1. The creation of the file that is expected to store the memory as qemu image
seems to be redundant. instead, we need to have a way to create a file in a
similar way to 'touch' command - just to verify that the file in the path we
pass to libvirt is created with the right permissions. (it's probably true for
the hibernation volume creation as well btw).


how would this look for block devices?



2. The more important issue is that we need to have a way to export files which
are not qemu images to the export domain, which means without using the qemu-img
process, just copy the file.


ayal - anything close to this today - i remember various qemu-img vs. dd 
patches in the past for copy operations?






- Reminder to follow wiki feature page process closely to maximize changes to
   have features added to a release.

- Multiple gateways are being worked on to fix dhcp races and comments by
   reviewers.

- NetReloaded after multiple gateways. Try to get up to iproute2
configurator.

- Cross-distro support:
   + Netreloaded with iproute2 plus persistence would help.
   + init scripts separation (proposed by Yaniv to make ready for 3.3).

- Cloud-init integration by Greg Padgett. There is a vdsm patch that should
be
   taken care of in terms of reviewing http://gerrit.ovirt.org/#/c/14347/
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


Re: [vdsm] vdsm sync meeting June 17th 2013

2013-06-18 Thread Arik Hadas

- Original Message -
 VDSM sync meeting Monday June 17th 2013
 ===
 
 - Dan requests reviews for pending patches from Mei Liu and himself (
   http://gerrit.ovirt.org/#/c/15356/8 ) for mom.
 
 - Toni to fix the problem with Mark's patch that Dan reviewed so that it can
   be merged together with the smaller following patches.
 
 - Federico has three high priority patches for features and two lower
 priority
   ones that need reviews in order to make it for the release deadline. They
   are
   related to disk extension and image upload/download (glance):
   + http://gerrit.ovirt.org/#/c/15442 - constants: unify the BLANK_UUID
   def...
   + http://gerrit.ovirt.org/#/c/14589 - volume: add the BLOCK_SIZE constant
   +1
   + http://gerrit.ovirt.org/#/c/14590 - volume: add the extendVolumeSize
   method
   + http://gerrit.ovirt.org/#/c/15614 - vm: add the live diskSizeExtend
   method +1
   + http://gerrit.ovirt.org/#/c/14955 - image: add support to upload/download
   images
 
 - Adam and others to review Gluster patches:
   + http://gerrit.ovirt.org/#/c/13785/
   + http://gerrit.ovirt.org/#/c/11094/
 
 - Is there a filing for Qemu image handling reported by virt team?

Background about the problem:

As part of the ram snapshots feature 
(http://wiki.ovirt.org/Features/RAM_Snapshots)
we pass a path to file to libvirt, where the memory of the VM should be saved as
part of the snapshot creation.
the output file is identical to the one that is created to store the memory on
hibernate command.

When VM with snapshot(s) that has memory is being exported, we want the 
snapshot's
memory file to be exported to the export domain as well. when trying to export 
such
memory file, we saw that the file is being truncated to the closest size that is
a multiple of block size (512). we found that the file is being truncated by
qemu-img that is being used to copy the file to the export domain.

We previously thought that it is a problem in qemu - not handling files with 
size
that is not a multiple of block size, and a bug to qemu was opened (bz 970559).

We now understand that the issue described above might be a problem in qemu that
needs to be solved, but it is not the problem in our case:
treating the memory file as qemu image is wrong - the created file is not a qemu
image. the file has a special format: it contains xml part at the beginning and
the memory dump as a binary data right after that.

So basically it seems that we need the following things in vdsm:

1. The creation of the file that is expected to store the memory as qemu image
seems to be redundant. instead, we need to have a way to create a file in a
similar way to 'touch' command - just to verify that the file in the path we
pass to libvirt is created with the right permissions. (it's probably true for
the hibernation volume creation as well btw).

2. The more important issue is that we need to have a way to export files which
are not qemu images to the export domain, which means without using the qemu-img
process, just copy the file.

 
 - Reminder to follow wiki feature page process closely to maximize changes to
   have features added to a release.
 
 - Multiple gateways are being worked on to fix dhcp races and comments by
   reviewers.
 
 - NetReloaded after multiple gateways. Try to get up to iproute2
 configurator.
 
 - Cross-distro support:
   + Netreloaded with iproute2 plus persistence would help.
   + init scripts separation (proposed by Yaniv to make ready for 3.3).
 
 - Cloud-init integration by Greg Padgett. There is a vdsm patch that should
 be
   taken care of in terms of reviewing http://gerrit.ovirt.org/#/c/14347/
 ___
 vdsm-devel mailing list
 vdsm-devel@lists.fedorahosted.org
 https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
 
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel


[vdsm] vdsm sync meeting June 17th 2013

2013-06-17 Thread Antoni Segura Puimedon
VDSM sync meeting Monday June 17th 2013
===

- Dan requests reviews for pending patches from Mei Liu and himself (
  http://gerrit.ovirt.org/#/c/15356/8 ) for mom.

- Toni to fix the problem with Mark's patch that Dan reviewed so that it can
  be merged together with the smaller following patches.

- Federico has three high priority patches for features and two lower priority
  ones that need reviews in order to make it for the release deadline. They are
  related to disk extension and image upload/download (glance):
  + http://gerrit.ovirt.org/#/c/15442 - constants: unify the BLANK_UUID def...
  + http://gerrit.ovirt.org/#/c/14589 - volume: add the BLOCK_SIZE constant +1
  + http://gerrit.ovirt.org/#/c/14590 - volume: add the extendVolumeSize method
  + http://gerrit.ovirt.org/#/c/15614 - vm: add the live diskSizeExtend method 
+1
  + http://gerrit.ovirt.org/#/c/14955 - image: add support to upload/download 
images

- Adam and others to review Gluster patches:
  + http://gerrit.ovirt.org/#/c/13785/
  + http://gerrit.ovirt.org/#/c/11094/

- Is there a filing for Qemu image handling reported by virt team?

- Reminder to follow wiki feature page process closely to maximize changes to
  have features added to a release.

- Multiple gateways are being worked on to fix dhcp races and comments by
  reviewers.

- NetReloaded after multiple gateways. Try to get up to iproute2 configurator.

- Cross-distro support:
  + Netreloaded with iproute2 plus persistence would help.
  + init scripts separation (proposed by Yaniv to make ready for 3.3).

- Cloud-init integration by Greg Padgett. There is a vdsm patch that should be
  taken care of in terms of reviewing http://gerrit.ovirt.org/#/c/14347/
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel