[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Richard W.M. Jones]

On Wed, May 30, 2018 at 03:35:11PM +0300, Nir Soffer wrote:
> This is not the flow we are looking for. We need a way to read qcow2 data
> from a pipe.

The flow you asked for:

> > > image in any format -> qemu-img -> [qcow2 byte stream] -> imageio http
> > > server -> http client

is exactly what rhv-upload-plugin.py does, except for "-> http client"
at the end which I don't understand.

Can you describe exactly what you're trying to do again?

[...]
> > But in any case you can just use the nbdkit tar plugin which already
> > does all of this.
> >
> 
> Can it work with a tar stream read from stdin, or it requires a tar file?

As above it may help to describe from the start exactly what you're
trying to do.  This email thread has gone on for days and it's hard to
keep track of everything.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JILUXYB2FPZMUZ347M5CMHR6LOSGZVK3/

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Nir Soffer]

On Wed, May 30, 2018 at 11:58 AM Richard W.M. Jones 
wrote:

> On Wed, May 30, 2018 at 12:11:21AM +0300, Nir Soffer wrote:
> > Exporting images or ova files:
> >
> > image in any format -> qemu-img -> [qcow2 byte stream] -> imageio http
> > server -> http client
>
> You can do this with nbdkit + plugin, it's exactly what we do today
> for virt-v2v:
>
>
> https://github.com/libguestfs/libguestfs/blob/master/v2v/rhv-upload-plugin.py


This is not the flow we are looking for. We need a way to read qcow2 data
from a pipe.

> Importing images or ova files:
> >
> > http client -> imageio http server -> [qcow2 byte stream] -> qemu-img ->
> > image in any format
>
> Also could be done with nbdkit + plugin, basically the reverse of the
> above.
>
> > > If you can create a tar file that reserves space for the image file
> > > without actually writing it, a possible workaround today would be using
> > > the offset/size runtime options of the raw driver to convert directly
> > > into a region inside the tar archive.
> > >
> >
> > What are the offset/size runtime options? I cannot find anything about
> > them in man qemu-img.
>
> See:
>
> https://github.com/libguestfs/libguestfs/blob/dd162d2cd56a2ecf4bcd40a7f463940eaac875b8/v2v/input_ova.ml#L161
>
> But in any case you can just use the nbdkit tar plugin which already
> does all of this.
>

Can it work with a tar stream read from stdin, or it requires a tar file?

Nir
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4WABP4VNDAAJM4I7OHXU6EK72EZKI6YZ/

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Richard W.M. Jones]

On Wed, May 30, 2018 at 12:11:21AM +0300, Nir Soffer wrote:
> Exporting images or ova files:
> 
> image in any format -> qemu-img -> [qcow2 byte stream] -> imageio http
> server -> http client

You can do this with nbdkit + plugin, it's exactly what we do today
for virt-v2v:

https://github.com/libguestfs/libguestfs/blob/master/v2v/rhv-upload-plugin.py

> Importing images or ova files:
> 
> http client -> imageio http server -> [qcow2 byte stream] -> qemu-img ->
> image in any format

Also could be done with nbdkit + plugin, basically the reverse of the
above.

> > If you can create a tar file that reserves space for the image file
> > without actually writing it, a possible workaround today would be using
> > the offset/size runtime options of the raw driver to convert directly
> > into a region inside the tar archive.
> >
> 
> What are the offset/size runtime options? I cannot find anything about
> them in man qemu-img.

See:
https://github.com/libguestfs/libguestfs/blob/dd162d2cd56a2ecf4bcd40a7f463940eaac875b8/v2v/input_ova.ml#L161

But in any case you can just use the nbdkit tar plugin which already
does all of this.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/PXRHKGNW7FTPT27XSGEDFWK7LRPNZMNL/

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Nir Soffer]

On Mon, May 28, 2018 at 2:38 PM Kevin Wolf  wrote:

> Am 28.05.2018 um 12:27 hat Arik Hadas geschrieben:
> > On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf  wrote:
> >
> > > [ Adding qemu-block ]
> > >
> > > Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:
> > > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer 
> wrote:
> > > >
> > > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <
> > > m.vrgo...@activevideo.com>
> > > > > wrote:
> > > > >
> > > > >> Dear Nir,
> > > > >>
> > > > >> Thank you for quick reply.
> > > > >>
> > > > >> Ok, why it will not work?
> > > > >>
> > > > >
> > > > > Because the image has a backing file which is not accessible to
> oVirt.
> > > > >
> > > > >
> > > > >> I used qemu+tcp connection, via import method through engine
> admin UI.
> > > > >>
> > > > >> Images was imported and converted according logs, still “backing
> file”
> > > > >> invalid entry remained.
> > > > >>
> > > > >> Also, I did use same method before, connecting to plain “libvirt
> kvm”
> > > > >> host, import and conversion went smooth, no backend file.
> > > > >>
> > > > >> Image format is qcow(2) which is supported by oVirt.
> > > > >>
> > > > >> What am I missing? Should I use different method?
> > > > >>
> > > > >
> > > > > I guess this is not a problem on your side, but a bug in our side.
> > > > >
> > > > > Either we should block the operation that cannot work, or fix the
> > > process
> > > > > so we don't refer to non-existing image.
> > > > >
> > > > > When importing we have 2 options:
> > > > >
> > > > > - import the entire chain,  importing all images in the chain,
> > > converting
> > > > >  each image to oVirt volume, and updating the backing file of each
> > > layer
> > > > > to point to the oVirt image.
> > > > >
> > > > > - import the current state of the image into a new image, using
> either
> > > raw
> > > > > or qcow2, but without any backing file.
> > > > >
> > > > > Arik, do you know why we create qcow2 file with invalid backing
> file?
> > > > >
> > > >
> > > > It seems to be a result of a bit naive behavior of the kvm2ovirt
> module
> > > > that tries to download only the top-level volume the VM uses,
> assuming
> > > each
> > > > of the disks to be imported is comprised of a single volume.
> > > >
> > > > Maybe it's time to finally asking QEMU guys to provide a way to
> consume
> > > the
> > > > 'collapsed' form of a chain of volumes as a stream if that's not
> > > available
> > > > yet? ;) It can also boost the recently added process of exporting
> VMs as
> > > > OVAs...
> > >
> > > Not sure which operation we're talking about on the QEMU level, but
> > > generally the "collapsed" view is the normal thing because that's what
> > > guests see.
> > >
> > > For example, if you use 'qemu-img convert', you have to pass options to
> > > specifically disable it and convert only a single layer if you want to
> > > keep using backing files instead of getting a standalone image that
> > > contains everything.
> > >
> >
> > Yeah, some context was missing. Sorry about that.
> >
> > Let me demonstrate briefly the flow for OVA:
> > Let's say that we have a VM that is based on a template and has one disk
> > and one snapshot, so its volume-chain would be:
> > T -> S -> V
> > (V is the volume the VM writes to, S is the backing file of V and T is
> the
> > backing file of S).
> > When exporting that VM to an OVA file we want the produced tar file to be
> > comprised of:
> > (1) OVF configuration
> > (2) single disk volume (preferably qcow).
> >
> > So we need to collapse T, S, V into a single volume.
> > Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> > (a) qemu-img convert produces a 'temporary' collapsed volume
> > (b) make a tar file of the OVf configuration and that 'temporary' volume
> > (c) delete the temporary volume
> >
> > But the fact that we produce that 'temporary' volume obviously slows down
> > the entire operation.
> > It would be much better if we could "open" a stream that we can read from
> > the 'collapsed' form of that chain and stream it directly into the
> > appropriate tar file entry, without extra writes to the storage device.
> >
> > Few months ago people from the oVirt-storage team checked the qemu
> toolset
> > and replied that this capability is not yet provided, therefore we
> > implemented the workaround described above. Apparently, the desired
> ability
> > can also be useful for the flow discussed in this thread so it worth
> asking
> > for it again :)
>
> I think real streaming is unlikely to happen because most image formats
> that QEMU supports aren't made that way. If there is a compelling
> reason, we can consider it, but it would work only with very few target
> formats and as such would have to be separate from existing commands.
>

Real streaming is exactly what we want, and we need it only for qcow2
format,
because it is our preferred way to pack images in ova.

We have 2 possible use cases:

Exporting images or ova files:

image in 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Kevin Wolf]

Am 29.05.2018 um 11:27 hat Richard W.M. Jones geschrieben:
> On Mon, May 28, 2018 at 01:27:21PM +0300, Arik Hadas wrote:
> > Let me demonstrate briefly the flow for OVA:
> > Let's say that we have a VM that is based on a template and has one disk
> > and one snapshot, so its volume-chain would be:
> > T -> S -> V
> > (V is the volume the VM writes to, S is the backing file of V and T is the
> > backing file of S).
> > When exporting that VM to an OVA file we want the produced tar file to be
> > comprised of:
> > (1) OVF configuration
> > (2) single disk volume (preferably qcow).
> > 
> > So we need to collapse T, S, V into a single volume.
> > Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> > (a) qemu-img convert produces a 'temporary' collapsed volume
> > (b) make a tar file of the OVf configuration and that 'temporary' volume
> > (c) delete the temporary volume
> > 
> > But the fact that we produce that 'temporary' volume obviously slows down
> > the entire operation.
> > It would be much better if we could "open" a stream that we can read from
> > the 'collapsed' form of that chain and stream it directly into the
> > appropriate tar file entry, without extra writes to the storage device.
> 
> A custom nbdkit plugin is possible here.  In fact it's almost possible
> using the existing nbdkit-tar-plugin[1], except that it doesn't
> support resizing the tarball so you'd need a way to predict the size
> of the final qcow2 file.

I think you can predict the size with 'qemu-img measure'.

But how do you create a tar archive that contains an empty file of the
right size without actually processing and writing gigabytes of zero
bytes? Is there an existing tool that can do that or would you have to
write your own?

> The main difficulty for modifying nbdkit-tar-plugin is working out how
> to resize tar files.  If you can do that then it's likely just a few
> lines of code.

This sounds impossible to do when the tar archive needs to stay
consistent at all times.

Kevin
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/UEWL6ATIGOAC7F6FG5HJG5CEXXSKEGYE/

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Richard W.M. Jones]

On Mon, May 28, 2018 at 01:27:21PM +0300, Arik Hadas wrote:
> Let me demonstrate briefly the flow for OVA:
> Let's say that we have a VM that is based on a template and has one disk
> and one snapshot, so its volume-chain would be:
> T -> S -> V
> (V is the volume the VM writes to, S is the backing file of V and T is the
> backing file of S).
> When exporting that VM to an OVA file we want the produced tar file to be
> comprised of:
> (1) OVF configuration
> (2) single disk volume (preferably qcow).
> 
> So we need to collapse T, S, V into a single volume.
> Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> (a) qemu-img convert produces a 'temporary' collapsed volume
> (b) make a tar file of the OVf configuration and that 'temporary' volume
> (c) delete the temporary volume
> 
> But the fact that we produce that 'temporary' volume obviously slows down
> the entire operation.
> It would be much better if we could "open" a stream that we can read from
> the 'collapsed' form of that chain and stream it directly into the
> appropriate tar file entry, without extra writes to the storage device.

A custom nbdkit plugin is possible here.  In fact it's almost possible
using the existing nbdkit-tar-plugin[1], except that it doesn't
support resizing the tarball so you'd need a way to predict the size
of the final qcow2 file.

The main difficulty for modifying nbdkit-tar-plugin is working out how
to resize tar files.  If you can do that then it's likely just a few
lines of code.

Rich.

[1] 
https://manpages.debian.org/testing/nbdkit-plugin-perl/nbdkit-tar-plugin.1.en.html
https://github.com/libguestfs/nbdkit/blob/master/plugins/tar/tar.pl

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/G34UH3QMVSGP4LMPJH7HWHXKNYHZ3D45/

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Kevin Wolf]

Am 28.05.2018 um 16:06 hat Tomáš Golembiovský geschrieben:
> 
> On Mon, 28 May 2018 13:37:59 +0200
> Kevin Wolf  wrote:
> 
> > Am 28.05.2018 um 12:27 hat Arik Hadas geschrieben:
> > > On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf  wrote:
> > >   
> > > > [ Adding qemu-block ]
> > > >
> > > > Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:  
> > > > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer  
> > > > > wrote:
> > > > >  
> > > > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <  
> > > > m.vrgo...@activevideo.com>  
> > > > > > wrote:
> > > > > >  
> > > > > >> Dear Nir,
> > > > > >>
> > > > > >> Thank you for quick reply.
> > > > > >>
> > > > > >> Ok, why it will not work?
> > > > > >>  
> > > > > >
> > > > > > Because the image has a backing file which is not accessible to 
> > > > > > oVirt.
> > > > > >
> > > > > >  
> > > > > >> I used qemu+tcp connection, via import method through engine admin 
> > > > > >> UI.
> > > > > >>
> > > > > >> Images was imported and converted according logs, still “backing 
> > > > > >> file”
> > > > > >> invalid entry remained.
> > > > > >>
> > > > > >> Also, I did use same method before, connecting to plain “libvirt 
> > > > > >> kvm”
> > > > > >> host, import and conversion went smooth, no backend file.
> > > > > >>
> > > > > >> Image format is qcow(2) which is supported by oVirt.
> > > > > >>
> > > > > >> What am I missing? Should I use different method?
> > > > > >>  
> > > > > >
> > > > > > I guess this is not a problem on your side, but a bug in our side.
> > > > > >
> > > > > > Either we should block the operation that cannot work, or fix the  
> > > > process  
> > > > > > so we don't refer to non-existing image.
> > > > > >
> > > > > > When importing we have 2 options:
> > > > > >
> > > > > > - import the entire chain,  importing all images in the chain,  
> > > > converting  
> > > > > >  each image to oVirt volume, and updating the backing file of each  
> > > > layer  
> > > > > > to point to the oVirt image.
> > > > > >
> > > > > > - import the current state of the image into a new image, using 
> > > > > > either  
> > > > raw  
> > > > > > or qcow2, but without any backing file.
> > > > > >
> > > > > > Arik, do you know why we create qcow2 file with invalid backing 
> > > > > > file?
> > > > > >  
> > > > >
> > > > > It seems to be a result of a bit naive behavior of the kvm2ovirt 
> > > > > module
> > > > > that tries to download only the top-level volume the VM uses, 
> > > > > assuming  
> > > > each  
> > > > > of the disks to be imported is comprised of a single volume.
> > > > >
> > > > > Maybe it's time to finally asking QEMU guys to provide a way to 
> > > > > consume  
> > > > the  
> > > > > 'collapsed' form of a chain of volumes as a stream if that's not  
> > > > available  
> > > > > yet? ;) It can also boost the recently added process of exporting VMs 
> > > > > as
> > > > > OVAs...  
> > > >
> > > > Not sure which operation we're talking about on the QEMU level, but
> > > > generally the "collapsed" view is the normal thing because that's what
> > > > guests see.
> > > >
> > > > For example, if you use 'qemu-img convert', you have to pass options to
> > > > specifically disable it and convert only a single layer if you want to
> > > > keep using backing files instead of getting a standalone image that
> > > > contains everything.
> > > >  
> > > 
> > > Yeah, some context was missing. Sorry about that.
> > > 
> > > Let me demonstrate briefly the flow for OVA:
> > > Let's say that we have a VM that is based on a template and has one disk
> > > and one snapshot, so its volume-chain would be:
> > > T -> S -> V
> > > (V is the volume the VM writes to, S is the backing file of V and T is the
> > > backing file of S).
> > > When exporting that VM to an OVA file we want the produced tar file to be
> > > comprised of:
> > > (1) OVF configuration
> > > (2) single disk volume (preferably qcow).
> > > 
> > > So we need to collapse T, S, V into a single volume.
> > > Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> > > (a) qemu-img convert produces a 'temporary' collapsed volume
> > > (b) make a tar file of the OVf configuration and that 'temporary' volume
> > > (c) delete the temporary volume
> > > 
> > > But the fact that we produce that 'temporary' volume obviously slows down
> > > the entire operation.
> > > It would be much better if we could "open" a stream that we can read from
> > > the 'collapsed' form of that chain and stream it directly into the
> > > appropriate tar file entry, without extra writes to the storage device.
> > > 
> > > Few months ago people from the oVirt-storage team checked the qemu toolset
> > > and replied that this capability is not yet provided, therefore we
> > > implemented the workaround described above. Apparently, the desired 
> > > ability
> > > can also be useful for the flow discussed in this thread so it worth 
> > > asking
> 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Tomáš Golembiovský]

On Mon, 28 May 2018 13:37:59 +0200
Kevin Wolf  wrote:

> Am 28.05.2018 um 12:27 hat Arik Hadas geschrieben:
> > On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf  wrote:
> >   
> > > [ Adding qemu-block ]
> > >
> > > Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:  
> > > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer  wrote:
> > > >  
> > > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <  
> > > m.vrgo...@activevideo.com>  
> > > > > wrote:
> > > > >  
> > > > >> Dear Nir,
> > > > >>
> > > > >> Thank you for quick reply.
> > > > >>
> > > > >> Ok, why it will not work?
> > > > >>  
> > > > >
> > > > > Because the image has a backing file which is not accessible to oVirt.
> > > > >
> > > > >  
> > > > >> I used qemu+tcp connection, via import method through engine admin 
> > > > >> UI.
> > > > >>
> > > > >> Images was imported and converted according logs, still “backing 
> > > > >> file”
> > > > >> invalid entry remained.
> > > > >>
> > > > >> Also, I did use same method before, connecting to plain “libvirt kvm”
> > > > >> host, import and conversion went smooth, no backend file.
> > > > >>
> > > > >> Image format is qcow(2) which is supported by oVirt.
> > > > >>
> > > > >> What am I missing? Should I use different method?
> > > > >>  
> > > > >
> > > > > I guess this is not a problem on your side, but a bug in our side.
> > > > >
> > > > > Either we should block the operation that cannot work, or fix the  
> > > process  
> > > > > so we don't refer to non-existing image.
> > > > >
> > > > > When importing we have 2 options:
> > > > >
> > > > > - import the entire chain,  importing all images in the chain,  
> > > converting  
> > > > >  each image to oVirt volume, and updating the backing file of each  
> > > layer  
> > > > > to point to the oVirt image.
> > > > >
> > > > > - import the current state of the image into a new image, using 
> > > > > either  
> > > raw  
> > > > > or qcow2, but without any backing file.
> > > > >
> > > > > Arik, do you know why we create qcow2 file with invalid backing file?
> > > > >  
> > > >
> > > > It seems to be a result of a bit naive behavior of the kvm2ovirt module
> > > > that tries to download only the top-level volume the VM uses, assuming  
> > > each  
> > > > of the disks to be imported is comprised of a single volume.
> > > >
> > > > Maybe it's time to finally asking QEMU guys to provide a way to consume 
> > > >  
> > > the  
> > > > 'collapsed' form of a chain of volumes as a stream if that's not  
> > > available  
> > > > yet? ;) It can also boost the recently added process of exporting VMs as
> > > > OVAs...  
> > >
> > > Not sure which operation we're talking about on the QEMU level, but
> > > generally the "collapsed" view is the normal thing because that's what
> > > guests see.
> > >
> > > For example, if you use 'qemu-img convert', you have to pass options to
> > > specifically disable it and convert only a single layer if you want to
> > > keep using backing files instead of getting a standalone image that
> > > contains everything.
> > >  
> > 
> > Yeah, some context was missing. Sorry about that.
> > 
> > Let me demonstrate briefly the flow for OVA:
> > Let's say that we have a VM that is based on a template and has one disk
> > and one snapshot, so its volume-chain would be:
> > T -> S -> V
> > (V is the volume the VM writes to, S is the backing file of V and T is the
> > backing file of S).
> > When exporting that VM to an OVA file we want the produced tar file to be
> > comprised of:
> > (1) OVF configuration
> > (2) single disk volume (preferably qcow).
> > 
> > So we need to collapse T, S, V into a single volume.
> > Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> > (a) qemu-img convert produces a 'temporary' collapsed volume
> > (b) make a tar file of the OVf configuration and that 'temporary' volume
> > (c) delete the temporary volume
> > 
> > But the fact that we produce that 'temporary' volume obviously slows down
> > the entire operation.
> > It would be much better if we could "open" a stream that we can read from
> > the 'collapsed' form of that chain and stream it directly into the
> > appropriate tar file entry, without extra writes to the storage device.
> > 
> > Few months ago people from the oVirt-storage team checked the qemu toolset
> > and replied that this capability is not yet provided, therefore we
> > implemented the workaround described above. Apparently, the desired ability
> > can also be useful for the flow discussed in this thread so it worth asking
> > for it again :)  
> 
> I think real streaming is unlikely to happen because most image formats
> that QEMU supports aren't made that way. If there is a compelling
> reason, we can consider it, but it would work only with very few target
> formats and as such would have to be separate from existing commands.
> 
> As for OVA files, I think it might be useful to have a tar block driver

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Vrgotic, Marko]

Bug submitted:

https://bugzilla.redhat.com/show_bug.cgi?id=1583176

Kind regards,

Marko


On 25/05/2018, 16:57, "Tomáš Golembiovský"  wrote:

On Fri, 25 May 2018 12:13:33 +
"Vrgotic, Marko"  wrote:

> 
> 
> On 25/05/2018, 12:29, "Tomáš Golembiovský"  wrote:
> 
> Hi,
> 
> On Fri, 25 May 2018 08:11:13 +
> "Vrgotic, Marko"  wrote:
> 
> > Dear Nir, Arik and Richard,
> > 
> > I hope discussion will continue somewhere where i am able to join 
as watcher at least.
> 
> please open a bug on VDSM. This is something we need to deal with 
during
> import -- or at least prevent users from importing.
>[Marko] Where? Email to users@ovirt.org? Do you need me to provide 
more information than in this email or additional information?
Here in Bugzilla: https://bugzilla.redhat.com/enter_bug.cgi?product=vdsm

Including the info you had in your first email should be enough for now.

>If possible, go with "deal with", instead of just 
preventing. My team very much enjoys oVirt platform and its functionality and 
we would love to see it grow further, internally and externally.

Could you please elaborate on your specific use-case here? Where did the
backing image came from? Snapshots, use of templates, ...?


> 
> > I have not seen any communication since Nir’s proposal. Please, if 
possible allow me to somehow track in which direction you are leaning.
> > 
> > In the meantime, as experienced engineers, do you have any 
suggestion how I could “workaround” current problem?
> > 
> > Rebasing image using qemu-img to remove backing file did not help 
(VM was able to start, but No Boot Device) and I think now that is due to image 
having functional dependency of base image.
> 
> What do you mean by rebasing? On which backing image did you rebase 
it?
>[Marko] It was using unsafe mode - just wanted to see what results 
will I get if I remove the backing file (inexperienced move). This action 
result in being able to start the VM, but ending up in No Bootable Device.

I guess you ended up with a disk full of holes in it and boot sector was
one of the missing pieces.

Tomas

> 
> I'm not too familiar with openstack, but I'd suggest doing a 
'qemu-img convert' on the disk in openstack to squash the backing change into 
new (and complete) image, assign this new disk to your VM and import it to 
oVirt.
> [Marko] Thank you. We will test it and check the result.
> 
> Tomas
> 
> > 
> > Like with VMs in oVrt, where template can not be deleted if there 
are still VMs existing, which were created from that template.
> > 
> > Please advise
> > 
> > — — —
> > Met vriendelijke groet / Best regards,
> > 
> > Marko Vrgotic
> > Sr. System Engineer
> > ActiveVideo
> > 
> > Tel. +31 (0)35 677 4131
> > email: m.vrgo...@activevideo.com
> > skype: av.mvrgotic.se
> > www.activevideo.com
> > 
> > From: Vrgotic, Marko
> > Sent: Thursday, May 24, 2018 5:26:30 PM
> > To: Nir Soffer
> > Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> > Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file 
after importing VM from OpenStack
> > 
> > Dear Nir,
> > 
> > I believe i understand now. The image imported is not base image, 
but required backing file to be able to work properly.
> > 
> > Maybe silly move, but i have tried to “solve/workaround” around the 
problem by rebasing image to remove backing file dependency, but it’s clear now 
why I than saw that “no bootable device found” during imported VM boot.
> > 
> > I support you suggestion to solve the import by either importing 
complete chain or recreating image so in a way it’s independent from former 
chain.
> > 
> > If you decide to go this way, please let me know which issue to 
track and if you need any more data provided from me.
> > 
> > I still need to solve problem with 200+ VM wanting to move to oVirt.
> > 
> > Kindly awaiting further updates.
> > 
> > — — —
> > Met vriendelijke groet / Best regards,
> > 
> > Marko Vrgotic
> > Sr. System Engineer
> > ActiveVideo
> > 
> > Tel. +31 (0)35 677 4131
> > email: m.vrgo...@activevideo.com
> > skype: av.mvrgotic.se
> > www.activevideo.com
> > 
> > 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Kevin Wolf]

Am 28.05.2018 um 12:27 hat Arik Hadas geschrieben:
> On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf  wrote:
> 
> > [ Adding qemu-block ]
> >
> > Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:
> > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer  wrote:
> > >
> > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <
> > m.vrgo...@activevideo.com>
> > > > wrote:
> > > >
> > > >> Dear Nir,
> > > >>
> > > >> Thank you for quick reply.
> > > >>
> > > >> Ok, why it will not work?
> > > >>
> > > >
> > > > Because the image has a backing file which is not accessible to oVirt.
> > > >
> > > >
> > > >> I used qemu+tcp connection, via import method through engine admin UI.
> > > >>
> > > >> Images was imported and converted according logs, still “backing file”
> > > >> invalid entry remained.
> > > >>
> > > >> Also, I did use same method before, connecting to plain “libvirt kvm”
> > > >> host, import and conversion went smooth, no backend file.
> > > >>
> > > >> Image format is qcow(2) which is supported by oVirt.
> > > >>
> > > >> What am I missing? Should I use different method?
> > > >>
> > > >
> > > > I guess this is not a problem on your side, but a bug in our side.
> > > >
> > > > Either we should block the operation that cannot work, or fix the
> > process
> > > > so we don't refer to non-existing image.
> > > >
> > > > When importing we have 2 options:
> > > >
> > > > - import the entire chain,  importing all images in the chain,
> > converting
> > > >  each image to oVirt volume, and updating the backing file of each
> > layer
> > > > to point to the oVirt image.
> > > >
> > > > - import the current state of the image into a new image, using either
> > raw
> > > > or qcow2, but without any backing file.
> > > >
> > > > Arik, do you know why we create qcow2 file with invalid backing file?
> > > >
> > >
> > > It seems to be a result of a bit naive behavior of the kvm2ovirt module
> > > that tries to download only the top-level volume the VM uses, assuming
> > each
> > > of the disks to be imported is comprised of a single volume.
> > >
> > > Maybe it's time to finally asking QEMU guys to provide a way to consume
> > the
> > > 'collapsed' form of a chain of volumes as a stream if that's not
> > available
> > > yet? ;) It can also boost the recently added process of exporting VMs as
> > > OVAs...
> >
> > Not sure which operation we're talking about on the QEMU level, but
> > generally the "collapsed" view is the normal thing because that's what
> > guests see.
> >
> > For example, if you use 'qemu-img convert', you have to pass options to
> > specifically disable it and convert only a single layer if you want to
> > keep using backing files instead of getting a standalone image that
> > contains everything.
> >
> 
> Yeah, some context was missing. Sorry about that.
> 
> Let me demonstrate briefly the flow for OVA:
> Let's say that we have a VM that is based on a template and has one disk
> and one snapshot, so its volume-chain would be:
> T -> S -> V
> (V is the volume the VM writes to, S is the backing file of V and T is the
> backing file of S).
> When exporting that VM to an OVA file we want the produced tar file to be
> comprised of:
> (1) OVF configuration
> (2) single disk volume (preferably qcow).
> 
> So we need to collapse T, S, V into a single volume.
> Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> (a) qemu-img convert produces a 'temporary' collapsed volume
> (b) make a tar file of the OVf configuration and that 'temporary' volume
> (c) delete the temporary volume
> 
> But the fact that we produce that 'temporary' volume obviously slows down
> the entire operation.
> It would be much better if we could "open" a stream that we can read from
> the 'collapsed' form of that chain and stream it directly into the
> appropriate tar file entry, without extra writes to the storage device.
> 
> Few months ago people from the oVirt-storage team checked the qemu toolset
> and replied that this capability is not yet provided, therefore we
> implemented the workaround described above. Apparently, the desired ability
> can also be useful for the flow discussed in this thread so it worth asking
> for it again :)

I think real streaming is unlikely to happen because most image formats
that QEMU supports aren't made that way. If there is a compelling
reason, we can consider it, but it would work only with very few target
formats and as such would have to be separate from existing commands.

As for OVA files, I think it might be useful to have a tar block driver
instead which would allow you to open a file inside a tar archive (you
could then also directly run an OVA without extracting it first). We
probably wouldn't be able to support resizing images there, but that
should be okay.

If you can create a tar file that reserves space for the image file
without actually writing it, a possible workaround today would be using
the offset/size runtime options of 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Arik Hadas]

On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf  wrote:

> [ Adding qemu-block ]
>
> Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:
> > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer  wrote:
> >
> > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <
> m.vrgo...@activevideo.com>
> > > wrote:
> > >
> > >> Dear Nir,
> > >>
> > >> Thank you for quick reply.
> > >>
> > >> Ok, why it will not work?
> > >>
> > >
> > > Because the image has a backing file which is not accessible to oVirt.
> > >
> > >
> > >> I used qemu+tcp connection, via import method through engine admin UI.
> > >>
> > >> Images was imported and converted according logs, still “backing file”
> > >> invalid entry remained.
> > >>
> > >> Also, I did use same method before, connecting to plain “libvirt kvm”
> > >> host, import and conversion went smooth, no backend file.
> > >>
> > >> Image format is qcow(2) which is supported by oVirt.
> > >>
> > >> What am I missing? Should I use different method?
> > >>
> > >
> > > I guess this is not a problem on your side, but a bug in our side.
> > >
> > > Either we should block the operation that cannot work, or fix the
> process
> > > so we don't refer to non-existing image.
> > >
> > > When importing we have 2 options:
> > >
> > > - import the entire chain,  importing all images in the chain,
> converting
> > >  each image to oVirt volume, and updating the backing file of each
> layer
> > > to point to the oVirt image.
> > >
> > > - import the current state of the image into a new image, using either
> raw
> > > or qcow2, but without any backing file.
> > >
> > > Arik, do you know why we create qcow2 file with invalid backing file?
> > >
> >
> > It seems to be a result of a bit naive behavior of the kvm2ovirt module
> > that tries to download only the top-level volume the VM uses, assuming
> each
> > of the disks to be imported is comprised of a single volume.
> >
> > Maybe it's time to finally asking QEMU guys to provide a way to consume
> the
> > 'collapsed' form of a chain of volumes as a stream if that's not
> available
> > yet? ;) It can also boost the recently added process of exporting VMs as
> > OVAs...
>
> Not sure which operation we're talking about on the QEMU level, but
> generally the "collapsed" view is the normal thing because that's what
> guests see.
>
> For example, if you use 'qemu-img convert', you have to pass options to
> specifically disable it and convert only a single layer if you want to
> keep using backing files instead of getting a standalone image that
> contains everything.
>

Yeah, some context was missing. Sorry about that.

Let me demonstrate briefly the flow for OVA:
Let's say that we have a VM that is based on a template and has one disk
and one snapshot, so its volume-chain would be:
T -> S -> V
(V is the volume the VM writes to, S is the backing file of V and T is the
backing file of S).
When exporting that VM to an OVA file we want the produced tar file to be
comprised of:
(1) OVF configuration
(2) single disk volume (preferably qcow).

So we need to collapse T, S, V into a single volume.
Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
(a) qemu-img convert produces a 'temporary' collapsed volume
(b) make a tar file of the OVf configuration and that 'temporary' volume
(c) delete the temporary volume

But the fact that we produce that 'temporary' volume obviously slows down
the entire operation.
It would be much better if we could "open" a stream that we can read from
the 'collapsed' form of that chain and stream it directly into the
appropriate tar file entry, without extra writes to the storage device.

Few months ago people from the oVirt-storage team checked the qemu toolset
and replied that this capability is not yet provided, therefore we
implemented the workaround described above. Apparently, the desired ability
can also be useful for the flow discussed in this thread so it worth asking
for it again :)


>
> Kevin
>
> >
> > >
> > > Nir
> > >
> > >
> > >>
> > >> Kindly awaiting your reply.
> > >>
> > >> — — —
> > >> Met vriendelijke groet / Best regards,
> > >>
> > >> Marko Vrgotic
> > >> Sr. System Engineer
> > >> ActiveVideo
> > >>
> > >> Tel. +31 (0)35 677 4131 <+31%2035%20677%204131>
> > >> email: m.vrgo...@activevideo.com
> > >> skype: av.mvrgotic.se
> > >> www.activevideo.com
> > >> --
> > >> *From:* Nir Soffer 
> > >> *Sent:* Thursday, May 24, 2018 4:09:40 PM
> > >> *To:* Vrgotic, Marko
> > >> *Cc:* users@ovirt.org; Richard W.M. Jones; Arik Hadas
> > >> *Subject:* Re: [ovirt-users] Libvirt ERROR cannot access backing file
> > >> after importing VM from OpenStack
> > >>
> > >>
> > >>
> > >> On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko <
> m.vrgo...@activevideo.com>
> > >> wrote:
> > >>
> > >> Dear oVirt team,
> > >>
> > >>
> > >>
> > >> When trying to start imported VM, it fails with following message:
> > >>
> > >>
> > >>
> > >> ERROR 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Vrgotic, Marko]

Apologies for delay,

I am reporting a bug atm.

Will send the bug number id afterwards.



On 25/05/2018, 16:57, "Tomáš Golembiovský"  wrote:

On Fri, 25 May 2018 12:13:33 +
"Vrgotic, Marko"  wrote:

> 
> 
> On 25/05/2018, 12:29, "Tomáš Golembiovský"  wrote:
> 
> Hi,
> 
> On Fri, 25 May 2018 08:11:13 +
> "Vrgotic, Marko"  wrote:
> 
> > Dear Nir, Arik and Richard,
> > 
> > I hope discussion will continue somewhere where i am able to join 
as watcher at least.
> 
> please open a bug on VDSM. This is something we need to deal with 
during
> import -- or at least prevent users from importing.
>[Marko] Where? Email to users@ovirt.org? Do you need me to provide 
more information than in this email or additional information?
Here in Bugzilla: https://bugzilla.redhat.com/enter_bug.cgi?product=vdsm

Including the info you had in your first email should be enough for now.

>If possible, go with "deal with", instead of just 
preventing. My team very much enjoys oVirt platform and its functionality and 
we would love to see it grow further, internally and externally.

Could you please elaborate on your specific use-case here? Where did the
backing image came from? Snapshots, use of templates, ...?


> 
> > I have not seen any communication since Nir’s proposal. Please, if 
possible allow me to somehow track in which direction you are leaning.
> > 
> > In the meantime, as experienced engineers, do you have any 
suggestion how I could “workaround” current problem?
> > 
> > Rebasing image using qemu-img to remove backing file did not help 
(VM was able to start, but No Boot Device) and I think now that is due to image 
having functional dependency of base image.
> 
> What do you mean by rebasing? On which backing image did you rebase 
it?
>[Marko] It was using unsafe mode - just wanted to see what results 
will I get if I remove the backing file (inexperienced move). This action 
result in being able to start the VM, but ending up in No Bootable Device.

I guess you ended up with a disk full of holes in it and boot sector was
one of the missing pieces.

Tomas

> 
> I'm not too familiar with openstack, but I'd suggest doing a 
'qemu-img convert' on the disk in openstack to squash the backing change into 
new (and complete) image, assign this new disk to your VM and import it to 
oVirt.
> [Marko] Thank you. We will test it and check the result.
> 
> Tomas
> 
> > 
> > Like with VMs in oVrt, where template can not be deleted if there 
are still VMs existing, which were created from that template.
> > 
> > Please advise
> > 
> > — — —
> > Met vriendelijke groet / Best regards,
> > 
> > Marko Vrgotic
> > Sr. System Engineer
> > ActiveVideo
> > 
> > Tel. +31 (0)35 677 4131
> > email: m.vrgo...@activevideo.com
> > skype: av.mvrgotic.se
> > www.activevideo.com
> > 
> > From: Vrgotic, Marko
> > Sent: Thursday, May 24, 2018 5:26:30 PM
> > To: Nir Soffer
> > Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> > Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file 
after importing VM from OpenStack
> > 
> > Dear Nir,
> > 
> > I believe i understand now. The image imported is not base image, 
but required backing file to be able to work properly.
> > 
> > Maybe silly move, but i have tried to “solve/workaround” around the 
problem by rebasing image to remove backing file dependency, but it’s clear now 
why I than saw that “no bootable device found” during imported VM boot.
> > 
> > I support you suggestion to solve the import by either importing 
complete chain or recreating image so in a way it’s independent from former 
chain.
> > 
> > If you decide to go this way, please let me know which issue to 
track and if you need any more data provided from me.
> > 
> > I still need to solve problem with 200+ VM wanting to move to oVirt.
> > 
> > Kindly awaiting further updates.
> > 
> > — — —
> > Met vriendelijke groet / Best regards,
> > 
> > Marko Vrgotic
> > Sr. System Engineer
> > ActiveVideo
> > 
> > Tel. +31 (0)35 677 4131
> > email: m.vrgo...@activevideo.com
> > skype: av.mvrgotic.se
> > www.activevideo.com
> > 
> > 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Kevin Wolf]

[ Adding qemu-block ]

Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:
> On Thu, May 24, 2018 at 6:13 PM, Nir Soffer  wrote:
> 
> > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> > wrote:
> >
> >> Dear Nir,
> >>
> >> Thank you for quick reply.
> >>
> >> Ok, why it will not work?
> >>
> >
> > Because the image has a backing file which is not accessible to oVirt.
> >
> >
> >> I used qemu+tcp connection, via import method through engine admin UI.
> >>
> >> Images was imported and converted according logs, still “backing file”
> >> invalid entry remained.
> >>
> >> Also, I did use same method before, connecting to plain “libvirt kvm”
> >> host, import and conversion went smooth, no backend file.
> >>
> >> Image format is qcow(2) which is supported by oVirt.
> >>
> >> What am I missing? Should I use different method?
> >>
> >
> > I guess this is not a problem on your side, but a bug in our side.
> >
> > Either we should block the operation that cannot work, or fix the process
> > so we don't refer to non-existing image.
> >
> > When importing we have 2 options:
> >
> > - import the entire chain,  importing all images in the chain, converting
> >  each image to oVirt volume, and updating the backing file of each layer
> > to point to the oVirt image.
> >
> > - import the current state of the image into a new image, using either raw
> > or qcow2, but without any backing file.
> >
> > Arik, do you know why we create qcow2 file with invalid backing file?
> >
> 
> It seems to be a result of a bit naive behavior of the kvm2ovirt module
> that tries to download only the top-level volume the VM uses, assuming each
> of the disks to be imported is comprised of a single volume.
> 
> Maybe it's time to finally asking QEMU guys to provide a way to consume the
> 'collapsed' form of a chain of volumes as a stream if that's not available
> yet? ;) It can also boost the recently added process of exporting VMs as
> OVAs...

Not sure which operation we're talking about on the QEMU level, but
generally the "collapsed" view is the normal thing because that's what
guests see.

For example, if you use 'qemu-img convert', you have to pass options to
specifically disable it and convert only a single layer if you want to
keep using backing files instead of getting a standalone image that
contains everything.

Kevin

> 
> >
> > Nir
> >
> >
> >>
> >> Kindly awaiting your reply.
> >>
> >> — — —
> >> Met vriendelijke groet / Best regards,
> >>
> >> Marko Vrgotic
> >> Sr. System Engineer
> >> ActiveVideo
> >>
> >> Tel. +31 (0)35 677 4131 <+31%2035%20677%204131>
> >> email: m.vrgo...@activevideo.com
> >> skype: av.mvrgotic.se
> >> www.activevideo.com
> >> --
> >> *From:* Nir Soffer 
> >> *Sent:* Thursday, May 24, 2018 4:09:40 PM
> >> *To:* Vrgotic, Marko
> >> *Cc:* users@ovirt.org; Richard W.M. Jones; Arik Hadas
> >> *Subject:* Re: [ovirt-users] Libvirt ERROR cannot access backing file
> >> after importing VM from OpenStack
> >>
> >>
> >>
> >> On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
> >> wrote:
> >>
> >> Dear oVirt team,
> >>
> >>
> >>
> >> When trying to start imported VM, it fails with following message:
> >>
> >>
> >>
> >> ERROR 
> >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> >> (ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM
> >> instance-0673 is down with error. Exit message: Cannot access backing
> >> file 
> >> '/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce'
> >> of storage file '/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.
> >> lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/
> >> 816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba'
> >> (as uid:107, gid:107): No such file or directory.
> >>
> >>
> >>
> >> Platform details:
> >>
> >> Ovirt SHE
> >>
> >> Version 4.2.2.6-1.el7.centos
> >>
> >> GlusterFS, unmanaged by oVirt.
> >>
> >>
> >>
> >> VM is imported & converted from OpenStack, according to log files,
> >> successfully (one WARN, related to different MAC address):
> >>
> >> 2018-05-24 12:03:31,028+02 INFO  [org.ovirt.engine.core.
> >> vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand] (default
> >> task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH,
> >> GetVmsNamesFromExternalProviderVDSCommand, return: [VM
> >> [instance-0001f94c], VM [instance-00078f6a], VM [instance-0814], VM
> >> [instance-0001f9ac], VM [instance-01ff], VM [instance-0001f718], VM
> >> [instance-0673], VM [instance-0001ecf2], VM [instance-00078d38]], log
> >> id: 7f178a5e
> >>
> >> 2018-05-24 12:48:33,722+02 INFO  [org.ovirt.engine.core.
> >> vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand] (default
> >> task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH,
> >> GetVmsNamesFromExternalProviderVDSCommand, return: [VM
> >> [instance-0001f94c], VM [instance-00078f6a], VM 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Arik Hadas]

On Thu, May 24, 2018 at 6:13 PM, Nir Soffer  wrote:

> On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> wrote:
>
>> Dear Nir,
>>
>> Thank you for quick reply.
>>
>> Ok, why it will not work?
>>
>
> Because the image has a backing file which is not accessible to oVirt.
>
>
>> I used qemu+tcp connection, via import method through engine admin UI.
>>
>> Images was imported and converted according logs, still “backing file”
>> invalid entry remained.
>>
>> Also, I did use same method before, connecting to plain “libvirt kvm”
>> host, import and conversion went smooth, no backend file.
>>
>> Image format is qcow(2) which is supported by oVirt.
>>
>> What am I missing? Should I use different method?
>>
>
> I guess this is not a problem on your side, but a bug in our side.
>
> Either we should block the operation that cannot work, or fix the process
> so we don't refer to non-existing image.
>
> When importing we have 2 options:
>
> - import the entire chain,  importing all images in the chain, converting
>  each image to oVirt volume, and updating the backing file of each layer
> to point to the oVirt image.
>
> - import the current state of the image into a new image, using either raw
> or qcow2, but without any backing file.
>
> Arik, do you know why we create qcow2 file with invalid backing file?
>

It seems to be a result of a bit naive behavior of the kvm2ovirt module
that tries to download only the top-level volume the VM uses, assuming each
of the disks to be imported is comprised of a single volume.

Maybe it's time to finally asking QEMU guys to provide a way to consume the
'collapsed' form of a chain of volumes as a stream if that's not available
yet? ;) It can also boost the recently added process of exporting VMs as
OVAs...


>
> Nir
>
>
>>
>> Kindly awaiting your reply.
>>
>> — — —
>> Met vriendelijke groet / Best regards,
>>
>> Marko Vrgotic
>> Sr. System Engineer
>> ActiveVideo
>>
>> Tel. +31 (0)35 677 4131 <+31%2035%20677%204131>
>> email: m.vrgo...@activevideo.com
>> skype: av.mvrgotic.se
>> www.activevideo.com
>> --
>> *From:* Nir Soffer 
>> *Sent:* Thursday, May 24, 2018 4:09:40 PM
>> *To:* Vrgotic, Marko
>> *Cc:* users@ovirt.org; Richard W.M. Jones; Arik Hadas
>> *Subject:* Re: [ovirt-users] Libvirt ERROR cannot access backing file
>> after importing VM from OpenStack
>>
>>
>>
>> On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
>> wrote:
>>
>> Dear oVirt team,
>>
>>
>>
>> When trying to start imported VM, it fails with following message:
>>
>>
>>
>> ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM
>> instance-0673 is down with error. Exit message: Cannot access backing
>> file '/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce'
>> of storage file '/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.
>> lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/
>> 816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba'
>> (as uid:107, gid:107): No such file or directory.
>>
>>
>>
>> Platform details:
>>
>> Ovirt SHE
>>
>> Version 4.2.2.6-1.el7.centos
>>
>> GlusterFS, unmanaged by oVirt.
>>
>>
>>
>> VM is imported & converted from OpenStack, according to log files,
>> successfully (one WARN, related to different MAC address):
>>
>> 2018-05-24 12:03:31,028+02 INFO  [org.ovirt.engine.core.
>> vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand] (default
>> task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH,
>> GetVmsNamesFromExternalProviderVDSCommand, return: [VM
>> [instance-0001f94c], VM [instance-00078f6a], VM [instance-0814], VM
>> [instance-0001f9ac], VM [instance-01ff], VM [instance-0001f718], VM
>> [instance-0673], VM [instance-0001ecf2], VM [instance-00078d38]], log
>> id: 7f178a5e
>>
>> 2018-05-24 12:48:33,722+02 INFO  [org.ovirt.engine.core.
>> vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand] (default
>> task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH,
>> GetVmsNamesFromExternalProviderVDSCommand, return: [VM
>> [instance-0001f94c], VM [instance-00078f6a], VM [instance-0814], VM
>> [instance-0001f9ac], VM [instance-01ff], VM [instance-0001f718], VM
>> [instance-0673], VM [instance-0001ecf2], VM [instance-00078d38]], log
>> id: 3aa178c5
>>
>> 2018-05-24 12:48:47,291+02 INFO  [org.ovirt.engine.core.
>> vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
>> (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] START,
>> GetVmsFullInfoFromExternalProviderVDSCommand(HostName = aws-ovhv-01,
>> GetVmsFromExternalProviderParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d',
>> url='qemu+tcp://root@172.19.0.12/system', username='null',
>> originType='KVM', namesOfVms='[instance-0673]'}), log id: 4c445109
>>
>> 2018-05-24 12:48:47,318+02 INFO  

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Tomáš Golembiovský]

On Fri, 25 May 2018 12:13:33 +
"Vrgotic, Marko"  wrote:

> 
> 
> On 25/05/2018, 12:29, "Tomáš Golembiovský"  wrote:
> 
> Hi,
> 
> On Fri, 25 May 2018 08:11:13 +
> "Vrgotic, Marko"  wrote:
> 
> > Dear Nir, Arik and Richard,
> > 
> > I hope discussion will continue somewhere where i am able to join as 
> watcher at least.
> 
> please open a bug on VDSM. This is something we need to deal with during
> import -- or at least prevent users from importing.
>[Marko] Where? Email to users@ovirt.org? Do you need me to provide 
> more information than in this email or additional information?
Here in Bugzilla: https://bugzilla.redhat.com/enter_bug.cgi?product=vdsm

Including the info you had in your first email should be enough for now.

>If possible, go with "deal with", instead of just 
> preventing. My team very much enjoys oVirt platform and its functionality and 
> we would love to see it grow further, internally and externally.

Could you please elaborate on your specific use-case here? Where did the
backing image came from? Snapshots, use of templates, ...?


> 
> > I have not seen any communication since Nir’s proposal. Please, if 
> possible allow me to somehow track in which direction you are leaning.
> > 
> > In the meantime, as experienced engineers, do you have any suggestion 
> how I could “workaround” current problem?
> > 
> > Rebasing image using qemu-img to remove backing file did not help (VM 
> was able to start, but No Boot Device) and I think now that is due to image 
> having functional dependency of base image.
> 
> What do you mean by rebasing? On which backing image did you rebase it?
>[Marko] It was using unsafe mode - just wanted to see what results 
> will I get if I remove the backing file (inexperienced move). This action 
> result in being able to start the VM, but ending up in No Bootable Device.

I guess you ended up with a disk full of holes in it and boot sector was
one of the missing pieces.

Tomas

> 
> I'm not too familiar with openstack, but I'd suggest doing a 'qemu-img 
> convert' on the disk in openstack to squash the backing change into new (and 
> complete) image, assign this new disk to your VM and import it to oVirt.
> [Marko] Thank you. We will test it and check the result.
> 
> Tomas
> 
> > 
> > Like with VMs in oVrt, where template can not be deleted if there are 
> still VMs existing, which were created from that template.
> > 
> > Please advise
> > 
> > — — —
> > Met vriendelijke groet / Best regards,
> > 
> > Marko Vrgotic
> > Sr. System Engineer
> > ActiveVideo
> > 
> > Tel. +31 (0)35 677 4131
> > email: m.vrgo...@activevideo.com
> > skype: av.mvrgotic.se
> > www.activevideo.com
> > 
> > From: Vrgotic, Marko
> > Sent: Thursday, May 24, 2018 5:26:30 PM
> > To: Nir Soffer
> > Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> > Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file 
> after importing VM from OpenStack
> > 
> > Dear Nir,
> > 
> > I believe i understand now. The image imported is not base image, but 
> required backing file to be able to work properly.
> > 
> > Maybe silly move, but i have tried to “solve/workaround” around the 
> problem by rebasing image to remove backing file dependency, but it’s clear 
> now why I than saw that “no bootable device found” during imported VM boot.
> > 
> > I support you suggestion to solve the import by either importing 
> complete chain or recreating image so in a way it’s independent from former 
> chain.
> > 
> > If you decide to go this way, please let me know which issue to track 
> and if you need any more data provided from me.
> > 
> > I still need to solve problem with 200+ VM wanting to move to oVirt.
> > 
> > Kindly awaiting further updates.
> > 
> > — — —
> > Met vriendelijke groet / Best regards,
> > 
> > Marko Vrgotic
> > Sr. System Engineer
> > ActiveVideo
> > 
> > Tel. +31 (0)35 677 4131
> > email: m.vrgo...@activevideo.com
> > skype: av.mvrgotic.se
> > www.activevideo.com
> > 
> > From: Nir Soffer 
> > Sent: Thursday, May 24, 2018 5:13:47 PM
> > To: Vrgotic, Marko
> > Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> > Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file 
> after importing VM from OpenStack
> > 
> > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> > wrote:
> > Dear Nir,
> > 
> > Thank you for quick reply.
> > 
> > Ok, why it will not work?

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Vrgotic, Marko]

On 25/05/2018, 12:29, "Tomáš Golembiovský"  wrote:

Hi,

On Fri, 25 May 2018 08:11:13 +
"Vrgotic, Marko"  wrote:

> Dear Nir, Arik and Richard,
> 
> I hope discussion will continue somewhere where i am able to join as 
watcher at least.

please open a bug on VDSM. This is something we need to deal with during
import -- or at least prevent users from importing.
   [Marko] Where? Email to users@ovirt.org? Do you need me to provide more 
information than in this email or additional information?
   If possible, go with "deal with", instead of just 
preventing. My team very much enjoys oVirt platform and its functionality and 
we would love to see it grow further, internally and externally.

> I have not seen any communication since Nir’s proposal. Please, if 
possible allow me to somehow track in which direction you are leaning.
> 
> In the meantime, as experienced engineers, do you have any suggestion how 
I could “workaround” current problem?
> 
> Rebasing image using qemu-img to remove backing file did not help (VM was 
able to start, but No Boot Device) and I think now that is due to image having 
functional dependency of base image.

What do you mean by rebasing? On which backing image did you rebase it?
   [Marko] It was using unsafe mode - just wanted to see what results will 
I get if I remove the backing file (inexperienced move). This action result in 
being able to start the VM, but ending up in No Bootable Device.

I'm not too familiar with openstack, but I'd suggest doing a 'qemu-img 
convert' on the disk in openstack to squash the backing change into new (and 
complete) image, assign this new disk to your VM and import it to oVirt.
[Marko] Thank you. We will test it and check the result.

Tomas

> 
> Like with VMs in oVrt, where template can not be deleted if there are 
still VMs existing, which were created from that template.
> 
> Please advise
> 
> — — —
> Met vriendelijke groet / Best regards,
> 
> Marko Vrgotic
> Sr. System Engineer
> ActiveVideo
> 
> Tel. +31 (0)35 677 4131
> email: m.vrgo...@activevideo.com
> skype: av.mvrgotic.se
> www.activevideo.com
> 
> From: Vrgotic, Marko
> Sent: Thursday, May 24, 2018 5:26:30 PM
> To: Nir Soffer
> Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack
> 
> Dear Nir,
> 
> I believe i understand now. The image imported is not base image, but 
required backing file to be able to work properly.
> 
> Maybe silly move, but i have tried to “solve/workaround” around the 
problem by rebasing image to remove backing file dependency, but it’s clear now 
why I than saw that “no bootable device found” during imported VM boot.
> 
> I support you suggestion to solve the import by either importing complete 
chain or recreating image so in a way it’s independent from former chain.
> 
> If you decide to go this way, please let me know which issue to track and 
if you need any more data provided from me.
> 
> I still need to solve problem with 200+ VM wanting to move to oVirt.
> 
> Kindly awaiting further updates.
> 
> — — —
> Met vriendelijke groet / Best regards,
> 
> Marko Vrgotic
> Sr. System Engineer
> ActiveVideo
> 
> Tel. +31 (0)35 677 4131
> email: m.vrgo...@activevideo.com
> skype: av.mvrgotic.se
> www.activevideo.com
> 
> From: Nir Soffer 
> Sent: Thursday, May 24, 2018 5:13:47 PM
> To: Vrgotic, Marko
> Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack
> 
> On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> wrote:
> Dear Nir,
> 
> Thank you for quick reply.
> 
> Ok, why it will not work?
> 
> Because the image has a backing file which is not accessible to oVirt.
> 
> I used qemu+tcp connection, via import method through engine admin UI.
> 
> Images was imported and converted according logs, still “backing file” 
invalid entry remained.
> 
> Also, I did use same method before, connecting to plain “libvirt kvm” 
host, import and conversion went smooth, no backend file.
> 
> Image format is qcow(2) which is supported by oVirt.
> 
> What am I missing? Should I use different method?
> 
> I guess this is not a problem on your side, but a bug in our side.
> 
> Either we should block the operation that cannot work, or fix the process
 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Tomáš Golembiovský]

Hi,

On Fri, 25 May 2018 08:11:13 +
"Vrgotic, Marko"  wrote:

> Dear Nir, Arik and Richard,
> 
> I hope discussion will continue somewhere where i am able to join as watcher 
> at least.

please open a bug on VDSM. This is something we need to deal with during
import -- or at least prevent users from importing.


> I have not seen any communication since Nir’s proposal. Please, if possible 
> allow me to somehow track in which direction you are leaning.
> 
> In the meantime, as experienced engineers, do you have any suggestion how I 
> could “workaround” current problem?
> 
> Rebasing image using qemu-img to remove backing file did not help (VM was 
> able to start, but No Boot Device) and I think now that is due to image 
> having functional dependency of base image.

What do you mean by rebasing? On which backing image did you rebase it?

I'm not too familiar with openstack, but I'd suggest doing a 'qemu-img convert' 
on the disk in openstack to squash the backing change into new (and complete) 
image, assign this new disk to your VM and import it to oVirt.

Tomas

> 
> Like with VMs in oVrt, where template can not be deleted if there are still 
> VMs existing, which were created from that template.
> 
> Please advise
> 
> — — —
> Met vriendelijke groet / Best regards,
> 
> Marko Vrgotic
> Sr. System Engineer
> ActiveVideo
> 
> Tel. +31 (0)35 677 4131
> email: m.vrgo...@activevideo.com
> skype: av.mvrgotic.se
> www.activevideo.com
> 
> From: Vrgotic, Marko
> Sent: Thursday, May 24, 2018 5:26:30 PM
> To: Nir Soffer
> Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
> importing VM from OpenStack
> 
> Dear Nir,
> 
> I believe i understand now. The image imported is not base image, but 
> required backing file to be able to work properly.
> 
> Maybe silly move, but i have tried to “solve/workaround” around the problem 
> by rebasing image to remove backing file dependency, but it’s clear now why I 
> than saw that “no bootable device found” during imported VM boot.
> 
> I support you suggestion to solve the import by either importing complete 
> chain or recreating image so in a way it’s independent from former chain.
> 
> If you decide to go this way, please let me know which issue to track and if 
> you need any more data provided from me.
> 
> I still need to solve problem with 200+ VM wanting to move to oVirt.
> 
> Kindly awaiting further updates.
> 
> — — —
> Met vriendelijke groet / Best regards,
> 
> Marko Vrgotic
> Sr. System Engineer
> ActiveVideo
> 
> Tel. +31 (0)35 677 4131
> email: m.vrgo...@activevideo.com
> skype: av.mvrgotic.se
> www.activevideo.com
> 
> From: Nir Soffer 
> Sent: Thursday, May 24, 2018 5:13:47 PM
> To: Vrgotic, Marko
> Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
> importing VM from OpenStack
> 
> On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> > wrote:
> Dear Nir,
> 
> Thank you for quick reply.
> 
> Ok, why it will not work?
> 
> Because the image has a backing file which is not accessible to oVirt.
> 
> I used qemu+tcp connection, via import method through engine admin UI.
> 
> Images was imported and converted according logs, still “backing file” 
> invalid entry remained.
> 
> Also, I did use same method before, connecting to plain “libvirt kvm” host, 
> import and conversion went smooth, no backend file.
> 
> Image format is qcow(2) which is supported by oVirt.
> 
> What am I missing? Should I use different method?
> 
> I guess this is not a problem on your side, but a bug in our side.
> 
> Either we should block the operation that cannot work, or fix the process
> so we don't refer to non-existing image.
> 
> When importing we have 2 options:
> 
> - import the entire chain,  importing all images in the chain, converting
>  each image to oVirt volume, and updating the backing file of each layer
> to point to the oVirt image.
> 
> - import the current state of the image into a new image, using either raw
> or qcow2, but without any backing file.
> 
> Arik, do you know why we create qcow2 file with invalid backing file?
> 
> Nir
> 
> 
> Kindly awaiting your reply.
> 
> — — —
> Met vriendelijke groet / Best regards,
> 
> Marko Vrgotic
> Sr. System Engineer
> ActiveVideo
> 
> Tel. +31 (0)35 677 4131
> email: m.vrgo...@activevideo.com
> skype: av.mvrgotic.se
> www.activevideo.com
> 
> From: Nir Soffer >
> Sent: Thursday, May 24, 2018 4:09:40 PM
> To: Vrgotic, Marko
> Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
> 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Vrgotic, Marko]

Dear Nir, Arik and Richard,

I hope discussion will continue somewhere where i am able to join as watcher at 
least.

I have not seen any communication since Nir’s proposal. Please, if possible 
allow me to somehow track in which direction you are leaning.

In the meantime, as experienced engineers, do you have any suggestion how I 
could “workaround” current problem?

Rebasing image using qemu-img to remove backing file did not help (VM was able 
to start, but No Boot Device) and I think now that is due to image having 
functional dependency of base image.

Like with VMs in oVrt, where template can not be deleted if there are still VMs 
existing, which were created from that template.

Please advise

— — —
Met vriendelijke groet / Best regards,

Marko Vrgotic
Sr. System Engineer
ActiveVideo

Tel. +31 (0)35 677 4131
email: m.vrgo...@activevideo.com
skype: av.mvrgotic.se
www.activevideo.com

From: Vrgotic, Marko
Sent: Thursday, May 24, 2018 5:26:30 PM
To: Nir Soffer
Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack

Dear Nir,

I believe i understand now. The image imported is not base image, but required 
backing file to be able to work properly.

Maybe silly move, but i have tried to “solve/workaround” around the problem by 
rebasing image to remove backing file dependency, but it’s clear now why I than 
saw that “no bootable device found” during imported VM boot.

I support you suggestion to solve the import by either importing complete chain 
or recreating image so in a way it’s independent from former chain.

If you decide to go this way, please let me know which issue to track and if 
you need any more data provided from me.

I still need to solve problem with 200+ VM wanting to move to oVirt.

Kindly awaiting further updates.

— — —
Met vriendelijke groet / Best regards,

Marko Vrgotic
Sr. System Engineer
ActiveVideo

Tel. +31 (0)35 677 4131
email: m.vrgo...@activevideo.com
skype: av.mvrgotic.se
www.activevideo.com

From: Nir Soffer 
Sent: Thursday, May 24, 2018 5:13:47 PM
To: Vrgotic, Marko
Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack

On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> wrote:
Dear Nir,

Thank you for quick reply.

Ok, why it will not work?

Because the image has a backing file which is not accessible to oVirt.

I used qemu+tcp connection, via import method through engine admin UI.

Images was imported and converted according logs, still “backing file” invalid 
entry remained.

Also, I did use same method before, connecting to plain “libvirt kvm” host, 
import and conversion went smooth, no backend file.

Image format is qcow(2) which is supported by oVirt.

What am I missing? Should I use different method?

I guess this is not a problem on your side, but a bug in our side.

Either we should block the operation that cannot work, or fix the process
so we don't refer to non-existing image.

When importing we have 2 options:

- import the entire chain,  importing all images in the chain, converting
 each image to oVirt volume, and updating the backing file of each layer
to point to the oVirt image.

- import the current state of the image into a new image, using either raw
or qcow2, but without any backing file.

Arik, do you know why we create qcow2 file with invalid backing file?

Nir


Kindly awaiting your reply.

— — —
Met vriendelijke groet / Best regards,

Marko Vrgotic
Sr. System Engineer
ActiveVideo

Tel. +31 (0)35 677 4131
email: m.vrgo...@activevideo.com
skype: av.mvrgotic.se
www.activevideo.com

From: Nir Soffer >
Sent: Thursday, May 24, 2018 4:09:40 PM
To: Vrgotic, Marko
Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack



On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
> wrote:

Dear oVirt team,



When trying to start imported VM, it fails with following message:



ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM instance-0673 
is down with error. Exit message: Cannot access backing file 
'/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of 
storage file 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Vrgotic, Marko]

Dear Nir,

I believe i understand now. The image imported is not base image, but required 
backing file to be able to work properly.

Maybe silly move, but i have tried to “solve/workaround” around the problem by 
rebasing image to remove backing file dependency, but it’s clear now why I than 
saw that “no bootable device found” during imported VM boot.

I support you suggestion to solve the import by either importing complete chain 
or recreating image so in a way it’s independent from former chain.

If you decide to go this way, please let me know which issue to track and if 
you need any more data provided from me.

I still need to solve problem with 200+ VM wanting to move to oVirt.

Kindly awaiting further updates.

— — —
Met vriendelijke groet / Best regards,

Marko Vrgotic
Sr. System Engineer
ActiveVideo

Tel. +31 (0)35 677 4131
email: m.vrgo...@activevideo.com
skype: av.mvrgotic.se
www.activevideo.com

From: Nir Soffer 
Sent: Thursday, May 24, 2018 5:13:47 PM
To: Vrgotic, Marko
Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack

On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
> wrote:
Dear Nir,

Thank you for quick reply.

Ok, why it will not work?

Because the image has a backing file which is not accessible to oVirt.

I used qemu+tcp connection, via import method through engine admin UI.

Images was imported and converted according logs, still “backing file” invalid 
entry remained.

Also, I did use same method before, connecting to plain “libvirt kvm” host, 
import and conversion went smooth, no backend file.

Image format is qcow(2) which is supported by oVirt.

What am I missing? Should I use different method?

I guess this is not a problem on your side, but a bug in our side.

Either we should block the operation that cannot work, or fix the process
so we don't refer to non-existing image.

When importing we have 2 options:

- import the entire chain,  importing all images in the chain, converting
 each image to oVirt volume, and updating the backing file of each layer
to point to the oVirt image.

- import the current state of the image into a new image, using either raw
or qcow2, but without any backing file.

Arik, do you know why we create qcow2 file with invalid backing file?

Nir


Kindly awaiting your reply.

— — —
Met vriendelijke groet / Best regards,

Marko Vrgotic
Sr. System Engineer
ActiveVideo

Tel. +31 (0)35 677 4131
email: m.vrgo...@activevideo.com
skype: av.mvrgotic.se
www.activevideo.com

From: Nir Soffer >
Sent: Thursday, May 24, 2018 4:09:40 PM
To: Vrgotic, Marko
Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack



On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
> wrote:

Dear oVirt team,



When trying to start imported VM, it fails with following message:



ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM instance-0673 
is down with error. Exit message: Cannot access backing file 
'/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of 
storage file 
'/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba'
 (as uid:107, gid:107): No such file or directory.



Platform details:

Ovirt SHE

Version 4.2.2.6-1.el7.centos

GlusterFS, unmanaged by oVirt.



VM is imported & converted from OpenStack, according to log files, successfully 
(one WARN, related to different MAC address):

2018-05-24 12:03:31,028+02 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
 (default task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH, 
GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c], VM 
[instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM 
[instance-01ff], VM [instance-0001f718], VM [instance-0673], VM 
[instance-0001ecf2], VM [instance-00078d38]], log id: 7f178a5e

2018-05-24 12:48:33,722+02 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
 (default task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH, 
GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c], VM 
[instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM 
[instance-01ff], VM [instance-0001f718], VM [instance-0673], VM 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Nir Soffer]

On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko 
wrote:

> Dear Nir,
>
> Thank you for quick reply.
>
> Ok, why it will not work?
>

Because the image has a backing file which is not accessible to oVirt.


> I used qemu+tcp connection, via import method through engine admin UI.
>
> Images was imported and converted according logs, still “backing file”
> invalid entry remained.
>
> Also, I did use same method before, connecting to plain “libvirt kvm”
> host, import and conversion went smooth, no backend file.
>
> Image format is qcow(2) which is supported by oVirt.
>
> What am I missing? Should I use different method?
>

I guess this is not a problem on your side, but a bug in our side.

Either we should block the operation that cannot work, or fix the process
so we don't refer to non-existing image.

When importing we have 2 options:

- import the entire chain,  importing all images in the chain, converting
 each image to oVirt volume, and updating the backing file of each layer
to point to the oVirt image.

- import the current state of the image into a new image, using either raw
or qcow2, but without any backing file.

Arik, do you know why we create qcow2 file with invalid backing file?

Nir


>
> Kindly awaiting your reply.
>
> — — —
> Met vriendelijke groet / Best regards,
>
> Marko Vrgotic
> Sr. System Engineer
> ActiveVideo
>
> Tel. +31 (0)35 677 4131 <+31%2035%20677%204131>
> email: m.vrgo...@activevideo.com
> skype: av.mvrgotic.se
> www.activevideo.com
> --
> *From:* Nir Soffer 
> *Sent:* Thursday, May 24, 2018 4:09:40 PM
> *To:* Vrgotic, Marko
> *Cc:* users@ovirt.org; Richard W.M. Jones; Arik Hadas
> *Subject:* Re: [ovirt-users] Libvirt ERROR cannot access backing file
> after importing VM from OpenStack
>
>
>
> On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
> wrote:
>
> Dear oVirt team,
>
>
>
> When trying to start imported VM, it fails with following message:
>
>
>
> ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM
> instance-0673 is down with error. Exit message: Cannot access backing
> file
> '/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of
> storage file
> '/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba'
> (as uid:107, gid:107): No such file or directory.
>
>
>
> Platform details:
>
> Ovirt SHE
>
> Version 4.2.2.6-1.el7.centos
>
> GlusterFS, unmanaged by oVirt.
>
>
>
> VM is imported & converted from OpenStack, according to log files,
> successfully (one WARN, related to different MAC address):
>
> 2018-05-24 12:03:31,028+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
> (default task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH,
> GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c],
> VM [instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM
> [instance-01ff], VM [instance-0001f718], VM [instance-0673], VM
> [instance-0001ecf2], VM [instance-00078d38]], log id: 7f178a5e
>
> 2018-05-24 12:48:33,722+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
> (default task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH,
> GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c],
> VM [instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM
> [instance-01ff], VM [instance-0001f718], VM [instance-0673], VM
> [instance-0001ecf2], VM [instance-00078d38]], log id: 3aa178c5
>
> 2018-05-24 12:48:47,291+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
> (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] START,
> GetVmsFullInfoFromExternalProviderVDSCommand(HostName = aws-ovhv-01,
> GetVmsFromExternalProviderParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d',
> url='qemu+tcp://root@172.19.0.12/system', username='null',
> originType='KVM', namesOfVms='[instance-0673]'}), log id: 4c445109
>
> 2018-05-24 12:48:47,318+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
> (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] FINISH,
> GetVmsFullInfoFromExternalProviderVDSCommand, return: [VM
> [instance-0673]], log id: 4c445109
>
> 2018-05-24 12:49:20,466+02 INFO
> [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand]
> (default task-41) [14edb003-b4a0-4355-b3de-da2b68774fe3] Lock Acquired to
> object 'EngineLock:{exclusiveLocks='[instance-0673=VM_NAME,
> 1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}'
>
> 2018-05-24 12:49:20,586+02 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> 

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Vrgotic, Marko]

Dear Nir,

Thank you for quick reply.

Ok, why it will not work?

I used qemu+tcp connection, via import method through engine admin UI.

Images was imported and converted according logs, still “backing file” invalid 
entry remained.

Also, I did use same method before, connecting to plain “libvirt kvm” host, 
import and conversion went smooth, no backend file.

Image format is qcow(2) which is supported by oVirt.

What am I missing? Should I use different method?

Kindly awaiting your reply.

— — —
Met vriendelijke groet / Best regards,

Marko Vrgotic
Sr. System Engineer
ActiveVideo

Tel. +31 (0)35 677 4131
email: m.vrgo...@activevideo.com
skype: av.mvrgotic.se
www.activevideo.com

From: Nir Soffer 
Sent: Thursday, May 24, 2018 4:09:40 PM
To: Vrgotic, Marko
Cc: users@ovirt.org; Richard W.M. Jones; Arik Hadas
Subject: Re: [ovirt-users] Libvirt ERROR cannot access backing file after 
importing VM from OpenStack



On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
> wrote:

Dear oVirt team,



When trying to start imported VM, it fails with following message:



ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM instance-0673 
is down with error. Exit message: Cannot access backing file 
'/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of 
storage file 
'/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba'
 (as uid:107, gid:107): No such file or directory.



Platform details:

Ovirt SHE

Version 4.2.2.6-1.el7.centos

GlusterFS, unmanaged by oVirt.



VM is imported & converted from OpenStack, according to log files, successfully 
(one WARN, related to different MAC address):

2018-05-24 12:03:31,028+02 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
 (default task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH, 
GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c], VM 
[instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM 
[instance-01ff], VM [instance-0001f718], VM [instance-0673], VM 
[instance-0001ecf2], VM [instance-00078d38]], log id: 7f178a5e

2018-05-24 12:48:33,722+02 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
 (default task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH, 
GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c], VM 
[instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM 
[instance-01ff], VM [instance-0001f718], VM [instance-0673], VM 
[instance-0001ecf2], VM [instance-00078d38]], log id: 3aa178c5

2018-05-24 12:48:47,291+02 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
 (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] START, 
GetVmsFullInfoFromExternalProviderVDSCommand(HostName = aws-ovhv-01, 
GetVmsFromExternalProviderParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d',
 url='qemu+tcp://root@172.19.0.12/system', 
username='null', originType='KVM', namesOfVms='[instance-0673]'}), log id: 
4c445109

2018-05-24 12:48:47,318+02 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
 (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] FINISH, 
GetVmsFullInfoFromExternalProviderVDSCommand, return: [VM [instance-0673]], 
log id: 4c445109

2018-05-24 12:49:20,466+02 INFO  
[org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand] 
(default task-41) [14edb003-b4a0-4355-b3de-da2b68774fe3] Lock Acquired to 
object 'EngineLock:{exclusiveLocks='[instance-0673=VM_NAME, 
1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}'

2018-05-24 12:49:20,586+02 WARN  
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(EE-ManagedThreadFactory-engine-Thread-653408) 
[14edb003-b4a0-4355-b3de-da2b68774fe3] EVENT_ID: MAC_ADDRESS_IS_EXTERNAL(925), 
VM instance-0673 has MAC address(es) fa:16:3e:74:18:50, which is/are out of 
its MAC pool definitions.

2018-05-24 12:49:21,021+02 INFO  
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(EE-ManagedThreadFactory-engine-Thread-653408) 
[14edb003-b4a0-4355-b3de-da2b68774fe3] EVENT_ID: 
IMPORTEXPORT_STARTING_IMPORT_VM(1,165), Starting to import Vm instance-0673 
to Data Center AVEUNL, Cluster AWSEUOPS

2018-05-24 12:49:28,816+02 INFO  
[org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand] 
(EE-ManagedThreadFactory-engine-Thread-653407) [] Lock freed to object 
'EngineLock:{exclusiveLocks='[instance-0673=VM_NAME, 
1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}'

[ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Nir Soffer]

On Thu, May 24, 2018 at 5:05 PM Vrgotic, Marko 
wrote:

> Dear oVirt team,
>
>
>
> When trying to start imported VM, it fails with following message:
>
>
>
> ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM
> instance-0673 is down with error. Exit message: Cannot access backing
> file
> '/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of
> storage file
> '/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba'
> (as uid:107, gid:107): No such file or directory.
>
>
>
> Platform details:
>
> Ovirt SHE
>
> Version 4.2.2.6-1.el7.centos
>
> GlusterFS, unmanaged by oVirt.
>
>
>
> VM is imported & converted from OpenStack, according to log files,
> successfully (one WARN, related to different MAC address):
>
> 2018-05-24 12:03:31,028+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
> (default task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH,
> GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c],
> VM [instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM
> [instance-01ff], VM [instance-0001f718], VM [instance-0673], VM
> [instance-0001ecf2], VM [instance-00078d38]], log id: 7f178a5e
>
> 2018-05-24 12:48:33,722+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand]
> (default task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH,
> GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c],
> VM [instance-00078f6a], VM [instance-0814], VM [instance-0001f9ac], VM
> [instance-01ff], VM [instance-0001f718], VM [instance-0673], VM
> [instance-0001ecf2], VM [instance-00078d38]], log id: 3aa178c5
>
> 2018-05-24 12:48:47,291+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
> (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] START,
> GetVmsFullInfoFromExternalProviderVDSCommand(HostName = aws-ovhv-01,
> GetVmsFromExternalProviderParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d',
> url='qemu+tcp://root@172.19.0.12/system', username='null',
> originType='KVM', namesOfVms='[instance-0673]'}), log id: 4c445109
>
> 2018-05-24 12:48:47,318+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand]
> (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] FINISH,
> GetVmsFullInfoFromExternalProviderVDSCommand, return: [VM
> [instance-0673]], log id: 4c445109
>
> 2018-05-24 12:49:20,466+02 INFO
> [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand]
> (default task-41) [14edb003-b4a0-4355-b3de-da2b68774fe3] Lock Acquired to
> object 'EngineLock:{exclusiveLocks='[instance-0673=VM_NAME,
> 1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}'
>
> 2018-05-24 12:49:20,586+02 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-engine-Thread-653408)
> [14edb003-b4a0-4355-b3de-da2b68774fe3] EVENT_ID:
> MAC_ADDRESS_IS_EXTERNAL(925), VM instance-0673 has MAC address(es)
> fa:16:3e:74:18:50, which is/are out of its MAC pool definitions.
>
> 2018-05-24 12:49:21,021+02 INFO
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-engine-Thread-653408)
> [14edb003-b4a0-4355-b3de-da2b68774fe3] EVENT_ID:
> IMPORTEXPORT_STARTING_IMPORT_VM(1,165), Starting to import Vm
> instance-0673 to Data Center AVEUNL, Cluster AWSEUOPS
>
> 2018-05-24 12:49:28,816+02 INFO
> [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand]
> (EE-ManagedThreadFactory-engine-Thread-653407) [] Lock freed to object
> 'EngineLock:{exclusiveLocks='[instance-0673=VM_NAME,
> 1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}'
>
> 2018-05-24 12:49:28,911+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConvertVmVDSCommand]
> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [2047673e] START,
> ConvertVmVDSCommand(HostName = aws-ovhv-01,
> ConvertVmVDSParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d',
> url='qemu+tcp://root@172.19.0.12/system', username='null',
> vmId='1f0b608f-7cfc-4b27-a876-b5d8073011a1', vmName='instance-0673',
> storageDomainId='2607c265-248c-40ad-b020-f3756454839e',
> storagePoolId='5a5de92c-0120-0167-03cb-038a', virtioIsoPath='null',
> compatVersion='null', Disk0='816ac00f-ba98-4827-b5c8-42a8ba496089'}), log
> id: 53408517
>
> 2018-05-24 12:49:29,010+02 INFO
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [2047673e] EVENT_ID:
> IMPORTEXPORT_STARTING_CONVERT_VM(1,193), Starting to convert Vm
> instance-0673
>
> 2018-05-24