[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
Dear oVirt gurus, There was a lot of discussion and ideas regarding solving this issue, which I absolutely respect. Still as product consumer, administrator, what is the status of it? Can we and in which version expect to see and test solution or part of it? Is there an approach you could recommend us to try with, while waiting for solution? Would it be possible to me to track it somewhere? Kindly awaiting your reply. Marko Vrgotic ActiveVideo From: Arik Hadas Date: Wednesday, 11 July 2018 at 10:30 To: "Wolf, Kevin" , Eric Blake , "Vrgotic, Marko" , Nir Soffer , Richard Jones , qemu-block Cc: users Subject: Re: [Qemu-block] [ovirt-users] Libvirt ERROR cannot access backing file after importing VM from OpenStack Better late than never - thank you all for the input, it was very useful! With the ability to measure the collapsed form of a volume chain in the qcow2 format, we managed to simplify and improve the process of creating an OVA significantly. What we do now is: 1. Measure the collapsed qcow2 volumes that would be written to the OVA 2. Create the OVA file accordingly 3. Write the "metadata" of the OVA - headers and OVF, while skipping the reserved places for disks 4. Mount each reserved place for a disk as a loopback device and convert the volume-chain directly to it [1] This saves us from allocating temporary disks and extra copies. That change would hopefully land in one of the next updates of oVirt 4.2. [1] https://github.com/oVirt/ovirt-engine/commit/23230a131af33bfa55a3fe828660c32f7fff04d7 On Wed, May 30, 2018 at 7:39 PM Kevin Wolf mailto:kw...@redhat.com>> wrote: Am 30.05.2018 um 18:14 hat Arik Hadas geschrieben: > On Wed, May 30, 2018 at 6:33 PM, Kevin Wolf > mailto:kw...@redhat.com>> wrote: > > I think the problem is that we're talking about two different things in > > one thread. If I understand correctly, what oVirt does today is: > > > > 1. qemu-img convert to create a temporary qcow2 image that merges the > >whole backing chain in a single file > > > > 2. tar to create an temporary OVA archive that contains, amongst others, > >the temporary qcow2 image. This is a second temporary file. > > > > 3. Stream this temporary OVA archive over HTTP > > > > Well, today we suggest users to mount a shared storage to multiple hosts > that reside in different oVirt/RHV deployments so they could export > VMs/templates as OVAs to that shared storage and import these OVAs from the > shared storage to a destination deployment. This process involves only #1 > and #2. > > The technique you proposed earlier for writing disks directly into an OVA, > assuming that the target size can be retrieved with 'qemu-img measure', > sounds like a nice approach to accelerate this process. I think we should > really consider doing that if that's as easy as it sounds. Writing the image to a given offset in a file is the example that I gave further down in the mail: > > You added another host into the mix, which just receives the image > > content via NBD and then re-exports it as HTTP. Does this host actually > > exist or is it the same host where the original images are located? > > > > Because if you stay local for this step, there is no need to use NBD at > > all: > > > > $ ./qemu-img measure -O qcow2 ~/images/hd.img > > required size: 67436544 > > fully allocated size: 67436544 > > $ ./qemu-img create -f file /tmp/test.qcow2 67436544 > > Formatting '/tmp/test.qcow2', fmt=file size=67436544 > > $ ./qemu-img convert -n --target-image-opts ~/images/hd.img > > driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536 > > > > hexdump verifies that this does the expected thing. > But #3 is definitely something we are interested in because we expect the > next step to be exporting the OVAs to a remote instance of Glance that > serves as a shared repository for the different deployments. Being able to > stream the collapsed form of a volume chain without writing anything to the > storage device would be fantastic. I think that even at the expense of > iterating the chain twice - once to map the structure of the jump tables > (right?) and once to stream the whole data. If the target is not a stupid web browser, but something actually virt-related like Glance, I'm sure it can offer a more suitable protocol than HTTP? If you could talk NBD to Glance, you'd get rid of the streaming requirement. I think it would make more sense to invest the effort there. Kevin ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PUWIUUENVIPFBG7QIQWJXJCO6LTCWXWP/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On Wed, Jul 11, 2018 at 11:30:19AM +0300, Arik Hadas wrote: > 4. Mount each reserved place for a disk as a loopback device and convert > the volume-chain directly to it [1] nbdkit tar plugin can overwrite a single file inside a tarball, all in userspace and non-root. https://github.com/libguestfs/nbdkit/tree/master/plugins/tar Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/GOESXWCB4JEIMWHQBA2T3CPTDOCDHHD4/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
Better late than never - thank you all for the input, it was very useful! With the ability to measure the collapsed form of a volume chain in the qcow2 format, we managed to simplify and improve the process of creating an OVA significantly. What we do now is: 1. Measure the collapsed qcow2 volumes that would be written to the OVA 2. Create the OVA file accordingly 3. Write the "metadata" of the OVA - headers and OVF, while skipping the reserved places for disks 4. Mount each reserved place for a disk as a loopback device and convert the volume-chain directly to it [1] This saves us from allocating temporary disks and extra copies. That change would hopefully land in one of the next updates of oVirt 4.2. [1] https://github.com/oVirt/ovirt-engine/commit/23230a131af33bfa55a3fe828660c32f7fff04d7 On Wed, May 30, 2018 at 7:39 PM Kevin Wolf wrote: > Am 30.05.2018 um 18:14 hat Arik Hadas geschrieben: > > On Wed, May 30, 2018 at 6:33 PM, Kevin Wolf wrote: > > > I think the problem is that we're talking about two different things in > > > one thread. If I understand correctly, what oVirt does today is: > > > > > > 1. qemu-img convert to create a temporary qcow2 image that merges the > > >whole backing chain in a single file > > > > > > 2. tar to create an temporary OVA archive that contains, amongst > others, > > >the temporary qcow2 image. This is a second temporary file. > > > > > > 3. Stream this temporary OVA archive over HTTP > > > > > > > Well, today we suggest users to mount a shared storage to multiple hosts > > that reside in different oVirt/RHV deployments so they could export > > VMs/templates as OVAs to that shared storage and import these OVAs from > the > > shared storage to a destination deployment. This process involves only #1 > > and #2. > > > > The technique you proposed earlier for writing disks directly into an > OVA, > > assuming that the target size can be retrieved with 'qemu-img measure', > > sounds like a nice approach to accelerate this process. I think we should > > really consider doing that if that's as easy as it sounds. > > Writing the image to a given offset in a file is the example that I gave > further down in the mail: > > > > You added another host into the mix, which just receives the image > > > content via NBD and then re-exports it as HTTP. Does this host actually > > > exist or is it the same host where the original images are located? > > > > > > Because if you stay local for this step, there is no need to use NBD at > > > all: > > > > > > $ ./qemu-img measure -O qcow2 ~/images/hd.img > > > required size: 67436544 > > > fully allocated size: 67436544 > > > $ ./qemu-img create -f file /tmp/test.qcow2 67436544 > > > Formatting '/tmp/test.qcow2', fmt=file size=67436544 > > > $ ./qemu-img convert -n --target-image-opts ~/images/hd.img > > > driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536 > > > > > > hexdump verifies that this does the expected thing. > > > But #3 is definitely something we are interested in because we expect the > > next step to be exporting the OVAs to a remote instance of Glance that > > serves as a shared repository for the different deployments. Being able > to > > stream the collapsed form of a volume chain without writing anything to > the > > storage device would be fantastic. I think that even at the expense of > > iterating the chain twice - once to map the structure of the jump tables > > (right?) and once to stream the whole data. > > If the target is not a stupid web browser, but something actually > virt-related like Glance, I'm sure it can offer a more suitable protocol > than HTTP? If you could talk NBD to Glance, you'd get rid of the > streaming requirement. I think it would make more sense to invest the > effort there. > > Kevin > ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FNTUC4PPI4VKYYFL5OH3ALOJWYMGEI2Z/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
Am 30.05.2018 um 18:14 hat Arik Hadas geschrieben: > On Wed, May 30, 2018 at 6:33 PM, Kevin Wolf wrote: > > I think the problem is that we're talking about two different things in > > one thread. If I understand correctly, what oVirt does today is: > > > > 1. qemu-img convert to create a temporary qcow2 image that merges the > >whole backing chain in a single file > > > > 2. tar to create an temporary OVA archive that contains, amongst others, > >the temporary qcow2 image. This is a second temporary file. > > > > 3. Stream this temporary OVA archive over HTTP > > > > Well, today we suggest users to mount a shared storage to multiple hosts > that reside in different oVirt/RHV deployments so they could export > VMs/templates as OVAs to that shared storage and import these OVAs from the > shared storage to a destination deployment. This process involves only #1 > and #2. > > The technique you proposed earlier for writing disks directly into an OVA, > assuming that the target size can be retrieved with 'qemu-img measure', > sounds like a nice approach to accelerate this process. I think we should > really consider doing that if that's as easy as it sounds. Writing the image to a given offset in a file is the example that I gave further down in the mail: > > You added another host into the mix, which just receives the image > > content via NBD and then re-exports it as HTTP. Does this host actually > > exist or is it the same host where the original images are located? > > > > Because if you stay local for this step, there is no need to use NBD at > > all: > > > > $ ./qemu-img measure -O qcow2 ~/images/hd.img > > required size: 67436544 > > fully allocated size: 67436544 > > $ ./qemu-img create -f file /tmp/test.qcow2 67436544 > > Formatting '/tmp/test.qcow2', fmt=file size=67436544 > > $ ./qemu-img convert -n --target-image-opts ~/images/hd.img > > driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536 > > > > hexdump verifies that this does the expected thing. > But #3 is definitely something we are interested in because we expect the > next step to be exporting the OVAs to a remote instance of Glance that > serves as a shared repository for the different deployments. Being able to > stream the collapsed form of a volume chain without writing anything to the > storage device would be fantastic. I think that even at the expense of > iterating the chain twice - once to map the structure of the jump tables > (right?) and once to stream the whole data. If the target is not a stupid web browser, but something actually virt-related like Glance, I'm sure it can offer a more suitable protocol than HTTP? If you could talk NBD to Glance, you'd get rid of the streaming requirement. I think it would make more sense to invest the effort there. Kevin ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AGEZDBNTOSTEFN7SIZ63U6CSIXHL547G/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On Wed, May 30, 2018 at 6:33 PM, Kevin Wolf wrote: > Am 30.05.2018 um 17:05 hat Eric Blake geschrieben: > > If I understood the question, we start with a local: > > > > T (any format) <- S (qcow2) <- V (qcow2) > > > > and want to create a remote tar file: > > > > dest.tar == | header ... | qcow2 image | > > > > where we write a single collapsed view of the T<-S<-V chain as a qcow2 > image > > in the subset of the remote tar file. > > I think the problem is that we're talking about two different things in > one thread. If I understand correctly, what oVirt does today is: > > 1. qemu-img convert to create a temporary qcow2 image that merges the >whole backing chain in a single file > > 2. tar to create an temporary OVA archive that contains, amongst others, >the temporary qcow2 image. This is a second temporary file. > > 3. Stream this temporary OVA archive over HTTP > Well, today we suggest users to mount a shared storage to multiple hosts that reside in different oVirt/RHV deployments so they could export VMs/templates as OVAs to that shared storage and import these OVAs from the shared storage to a destination deployment. This process involves only #1 and #2. The technique you proposed earlier for writing disks directly into an OVA, assuming that the target size can be retrieved with 'qemu-img measure', sounds like a nice approach to accelerate this process. I think we should really consider doing that if that's as easy as it sounds. But #3 is definitely something we are interested in because we expect the next step to be exporting the OVAs to a remote instance of Glance that serves as a shared repository for the different deployments. Being able to stream the collapsed form of a volume chain without writing anything to the storage device would be fantastic. I think that even at the expense of iterating the chain twice - once to map the structure of the jump tables (right?) and once to stream the whole data. > > Your proposal is about getting rid of the temporary file from step 1, > but keeping the temporary file from step 2. I was kind of ignoring > step 2 and answering how you can avoid a temporary file by creating and > streaming a qcow2 file in a single step, but if you already have the > code to create a qcow2 image as a stream, adding a tar header as well > shouldn't be that hard... > > I think Nir was talking about both. > > Ideally, we'd somehow get rid of HTTP, which introduces the requirement > of a non-seekable stream. > > > So, first use qemu-img to learn how big to size the collapsed qcow2 > image, > > and by extension, the overall tar image > > $ qemu-img measure -f qcow2 -O qcow2 V > > > > then pre-create a large enough tar file on the destination > > $ create header > > $ truncate --size=XXX dest.qcow2 > > $ tar cf dest.tar header dest.qcow2 > > > > (note that I explicitly did NOT use tar --sparse; dest.qcow2 is sparse > and > > occupies practically no disk space, but dest.tar must NOT be sparse > because > > neither tar nor NBD work well with after-the-fact resizing) > > > > then set up an NBD server on the destination that can write to the > subset of > > the tar file: > > > > $ learn the offset of dest.qcow2 within dest.tar (probably a multiple of > > 10240, given default GNU tar options) > > $ qemu-nbd --image-opts > > driver=raw,offset=YYY,size=XXX,file.driver=file,file.filename=dest.tar > > > > (I'm not sure if I got the --image-opts syntax exactly correct. nbdkit > has > > more examples of learning offsets within a tar file, and may be a better > > option as a server than qemu-nbd - but the point remains: serve up the > > subset of the dest.tar file as raw bytes) > > > > finally set up qemu as an NBD client on the source: > > $ qemu-img convert -f qcow2 V -O qcow2 nbd://remote > > > > (now the client collapses the qcow2 chain onto the source, and writes > that > > into a qcow2 subset of the tar file on the destination, where the > > destination was already sized large enough to hold the qcow2 image, and > > where no other temporary storage was needed other than the sparse > dest.qcow2 > > used in creating a large enough tar file) > > You added another host into the mix, which just receives the image > content via NBD and then re-exports it as HTTP. Does this host actually > exist or is it the same host where the original images are located? > > Because if you stay local for this step, there is no need to use NBD at > all: > > $ ./qemu-img measure -O qcow2 ~/images/hd.img > required size: 67436544 > fully allocated size: 67436544 > $ ./qemu-img create -f file /tmp/test.qcow2 67436544 > Formatting '/tmp/test.qcow2', fmt=file size=67436544 > $ ./qemu-img convert -n --target-image-opts ~/images/hd.img > driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536 > > hexdump verifies that this does the expected thing. > > > > Exporting to a stream is possible if we're allowed to make two passes > > > over the source, but the existing QEMU code is useless for
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
Am 30.05.2018 um 17:05 hat Eric Blake geschrieben: > If I understood the question, we start with a local: > > T (any format) <- S (qcow2) <- V (qcow2) > > and want to create a remote tar file: > > dest.tar == | header ... | qcow2 image | > > where we write a single collapsed view of the T<-S<-V chain as a qcow2 image > in the subset of the remote tar file. I think the problem is that we're talking about two different things in one thread. If I understand correctly, what oVirt does today is: 1. qemu-img convert to create a temporary qcow2 image that merges the whole backing chain in a single file 2. tar to create an temporary OVA archive that contains, amongst others, the temporary qcow2 image. This is a second temporary file. 3. Stream this temporary OVA archive over HTTP Your proposal is about getting rid of the temporary file from step 1, but keeping the temporary file from step 2. I was kind of ignoring step 2 and answering how you can avoid a temporary file by creating and streaming a qcow2 file in a single step, but if you already have the code to create a qcow2 image as a stream, adding a tar header as well shouldn't be that hard... I think Nir was talking about both. Ideally, we'd somehow get rid of HTTP, which introduces the requirement of a non-seekable stream. > So, first use qemu-img to learn how big to size the collapsed qcow2 image, > and by extension, the overall tar image > $ qemu-img measure -f qcow2 -O qcow2 V > > then pre-create a large enough tar file on the destination > $ create header > $ truncate --size=XXX dest.qcow2 > $ tar cf dest.tar header dest.qcow2 > > (note that I explicitly did NOT use tar --sparse; dest.qcow2 is sparse and > occupies practically no disk space, but dest.tar must NOT be sparse because > neither tar nor NBD work well with after-the-fact resizing) > > then set up an NBD server on the destination that can write to the subset of > the tar file: > > $ learn the offset of dest.qcow2 within dest.tar (probably a multiple of > 10240, given default GNU tar options) > $ qemu-nbd --image-opts > driver=raw,offset=YYY,size=XXX,file.driver=file,file.filename=dest.tar > > (I'm not sure if I got the --image-opts syntax exactly correct. nbdkit has > more examples of learning offsets within a tar file, and may be a better > option as a server than qemu-nbd - but the point remains: serve up the > subset of the dest.tar file as raw bytes) > > finally set up qemu as an NBD client on the source: > $ qemu-img convert -f qcow2 V -O qcow2 nbd://remote > > (now the client collapses the qcow2 chain onto the source, and writes that > into a qcow2 subset of the tar file on the destination, where the > destination was already sized large enough to hold the qcow2 image, and > where no other temporary storage was needed other than the sparse dest.qcow2 > used in creating a large enough tar file) You added another host into the mix, which just receives the image content via NBD and then re-exports it as HTTP. Does this host actually exist or is it the same host where the original images are located? Because if you stay local for this step, there is no need to use NBD at all: $ ./qemu-img measure -O qcow2 ~/images/hd.img required size: 67436544 fully allocated size: 67436544 $ ./qemu-img create -f file /tmp/test.qcow2 67436544 Formatting '/tmp/test.qcow2', fmt=file size=67436544 $ ./qemu-img convert -n --target-image-opts ~/images/hd.img driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536 hexdump verifies that this does the expected thing. > > Exporting to a stream is possible if we're allowed to make two passes > > over the source, but the existing QEMU code is useless for that because > > it inherently requires seeking. I think if I had to get something like > > this, I'd probably implement such an exporter as a script external to > > QEMU. > > Wait. What are we trying to stream? A qcow2 file, or what the guest would > see? If you stream just what the guest sees, then 'qemu-img map' tells you > which portions of which source files to read in order to reconstruct data in > the order it would be seen by the guest. I think the requirement was that the HTTP client downloads a qcow2 image. Did I get this wrong? > But yeah, an external exporter that takes a raw file, learns its size > and where the holes are, and then writes a trivial qcow2 header and > appends L1/L2/refcount tables on the end to convert the raw file into > a slightly-larger qcow2 file, might be a valid way to create a qcow2 > file from a two-pass read. Right. It may have to calculate the size of the L1 and refcount table first so it can write the right offsets into the header, so maybe it's easiest to precreate the whole metadata. But that's an implementation detail. Anyway, I don't think the existing QEMU code helps you with this. Kevin ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On 05/30/2018 09:16 AM, Kevin Wolf wrote: Am 30.05.2018 um 15:44 hat Eric Blake geschrieben: On 05/29/2018 04:18 PM, Nir Soffer wrote: You CAN get a logically collapsed view of storage (that is, what the guest would see), by using an NBD export of volume V. Reading from that volume will then pull sectors from whichever portion of the chain you need. You can use either qemu-nbd (if no guest is writing to the chain), or within a running qemu, you can use nbd-server-start and nbd-server-add (over QMP) to get such an NBD server running. NBD expose the guest data, but we want the qcow2 stream - without creating a new image. NBD can do both. You choose whether it exposes the guest data or the qcow2 data, by whether the client or the server is interpreting qcow2 data. But if I understand correctly, it doesn't result in the image Nir wants. You would only export an existing qcow2 file, i.e. a single layer in the backing chain, this way. The question was about a collapsed image, i.e. the disk content as the guest sees it. The problem is that qcow2 just isn't made to be streamable. Importing a qcow2 stream without saving it into a temporary file (or a memory buffer as large as the image file) simply isn't possible in the general case. If I understood the question, we start with a local: T (any format) <- S (qcow2) <- V (qcow2) and want to create a remote tar file: dest.tar == | header ... | qcow2 image | where we write a single collapsed view of the T<-S<-V chain as a qcow2 image in the subset of the remote tar file. So, first use qemu-img to learn how big to size the collapsed qcow2 image, and by extension, the overall tar image $ qemu-img measure -f qcow2 -O qcow2 V then pre-create a large enough tar file on the destination $ create header $ truncate --size=XXX dest.qcow2 $ tar cf dest.tar header dest.qcow2 (note that I explicitly did NOT use tar --sparse; dest.qcow2 is sparse and occupies practically no disk space, but dest.tar must NOT be sparse because neither tar nor NBD work well with after-the-fact resizing) then set up an NBD server on the destination that can write to the subset of the tar file: $ learn the offset of dest.qcow2 within dest.tar (probably a multiple of 10240, given default GNU tar options) $ qemu-nbd --image-opts driver=raw,offset=YYY,size=XXX,file.driver=file,file.filename=dest.tar (I'm not sure if I got the --image-opts syntax exactly correct. nbdkit has more examples of learning offsets within a tar file, and may be a better option as a server than qemu-nbd - but the point remains: serve up the subset of the dest.tar file as raw bytes) finally set up qemu as an NBD client on the source: $ qemu-img convert -f qcow2 V -O qcow2 nbd://remote (now the client collapses the qcow2 chain onto the source, and writes that into a qcow2 subset of the tar file on the destination, where the destination was already sized large enough to hold the qcow2 image, and where no other temporary storage was needed other than the sparse dest.qcow2 used in creating a large enough tar file) Exporting to a stream is possible if we're allowed to make two passes over the source, but the existing QEMU code is useless for that because it inherently requires seeking. I think if I had to get something like this, I'd probably implement such an exporter as a script external to QEMU. Wait. What are we trying to stream? A qcow2 file, or what the guest would see? If you stream just what the guest sees, then 'qemu-img map' tells you which portions of which source files to read in order to reconstruct data in the order it would be seen by the guest. But yeah, an external exporter that takes a raw file, learns its size and where the holes are, and then writes a trivial qcow2 header and appends L1/L2/refcount tables on the end to convert the raw file into a slightly-larger qcow2 file, might be a valid way to create a qcow2 file from a two-pass read. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UTMXG4DZLWRDCIVZ5KPNAYFJUWSSKFIJ/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
Am 30.05.2018 um 15:44 hat Eric Blake geschrieben: > On 05/29/2018 04:18 PM, Nir Soffer wrote: > > > You CAN get a logically collapsed view of storage (that is, what the > > > guest would see), by using an NBD export of volume V. Reading from that > > > volume will then pull sectors from whichever portion of the chain you > > > need. You can use either qemu-nbd (if no guest is writing to the > > > chain), or within a running qemu, you can use nbd-server-start and > > > nbd-server-add (over QMP) to get such an NBD server running. > > > > > > NBD expose the guest data, but we want the qcow2 stream - without > > creating a new image. > > NBD can do both. You choose whether it exposes the guest data or the qcow2 > data, by whether the client or the server is interpreting qcow2 data. But if I understand correctly, it doesn't result in the image Nir wants. You would only export an existing qcow2 file, i.e. a single layer in the backing chain, this way. The question was about a collapsed image, i.e. the disk content as the guest sees it. The problem is that qcow2 just isn't made to be streamable. Importing a qcow2 stream without saving it into a temporary file (or a memory buffer as large as the image file) simply isn't possible in the general case. Exporting to a stream is possible if we're allowed to make two passes over the source, but the existing QEMU code is useless for that because it inherently requires seeking. I think if I had to get something like this, I'd probably implement such an exporter as a script external to QEMU. Kevin ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7D7JQZMQXNSWOPEURXLT7BX3E3UUBER/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On 05/29/2018 04:18 PM, Nir Soffer wrote: You CAN get a logically collapsed view of storage (that is, what the guest would see), by using an NBD export of volume V. Reading from that volume will then pull sectors from whichever portion of the chain you need. You can use either qemu-nbd (if no guest is writing to the chain), or within a running qemu, you can use nbd-server-start and nbd-server-add (over QMP) to get such an NBD server running. NBD expose the guest data, but we want the qcow2 stream - without creating a new image. NBD can do both. You choose whether it exposes the guest data or the qcow2 data, by whether the client or the server is interpreting qcow2 data. Visually, if everything is local, qemu normally needs only two block layer entries: qcow2 format layer => file protocol layer But you can also make qemu use four block layer entries, since the raw layer is a normally passthrough layer (unless you are also using it for it's ability to support an offset within a larger file, such as reading from a tar file): raw format layer => qcow2 format layer => raw format layer => file protocol layer Then when you introduce NBD into the picture, you have the choice of WHERE in the four-layer system. The usual choice is: NBD server using -f qcow2, client using -f raw: raw format => NBD client protocol => (raw bytes) => NBD server => qcow2 format => raw format => file protocol (simplified to raw format => NBD client protocol => (raw bytes) => NBD server => qcow2 format => file protocol) But an alternative choice is: NBD server using -f raw, client using -f qcow2: raw format => qcow2 format => NBD client protocol => (qcow2 bytes) => NBD server => raw format => file protocol (simplified to qcow2 format => NBD client protocol => (qcow2 bytes) => NBD server => raw format => file protocol) -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7UGMMAKFC77DWBUAGZWRKOR2NEPGGNRO/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On 05/30/2018 07:35 AM, Nir Soffer wrote: This is not the flow we are looking for. We need a way to read qcow2 data from a pipe. Why? The qcow2 format inherently requires seeking (or a HUGE amount of free RAM) the moment you want to interpret the data as qcow2. It can't be piped when being produced or consumed for reading guest contents (it can be piped as a read-only format if you aren't inspecting guest contents, but then again so can raw). If you are asking how to connect a SEEKABLE network connection, so that you don't need temporary storage locally, then NBD is a great format (it is network friendly, where the local qemu-img process is using the remote server for all storage; no local storage required). But even then, it still requires a seekable input if you are not visiting the file in byte order, and qcow2 as an interpreted format cannot be written or read in byte order (for example, it inherently requires dereferencing through header, L1, L2, and refcount tables, which implies seeking). But in any case you can just use the nbdkit tar plugin which already does all of this. Can it work with a tar stream read from stdin, or it requires a tar file? nbdkit includes a plugin for creating a seekable layer on top of a pipe, at the expense of a huge memory cost (you have to have as much RAM available as you would ever have to seek backwards). It also makes it easy to have a plugin for a tar file (reading tar files is easy; writing is a bit harder, but should work as long as you don't need resize and don't have any compressed sparse regions that need to be rewritten to non-sparse data) -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/72CIHHKEZZJFCEKTZQAGU2524X27VBR6/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On Wed, May 30, 2018 at 12:35 AM Eric Blake wrote:
> On 05/29/2018 04:11 PM, Nir Soffer wrote:
>
> >> I think real streaming is unlikely to happen because most image formats
> >> that QEMU supports aren't made that way. If there is a compelling
> >> reason, we can consider it, but it would work only with very few target
> >> formats and as such would have to be separate from existing commands.
> >>
> >
> > Real streaming is exactly what we want, and we need it only for qcow2
> > format,
> > because it is our preferred way to pack images in ova.
> >
> > We have 2 possible use cases:
> >
> > Exporting images or ova files:
> >
> > image in any format -> qemu-img -> [qcow2 byte stream] -> imageio http
> > server -> http client
>
> image in any format -> qemu-img measure (to learn how large to size qcow2)
> then create destination qcow2 file that large
Isn't this a temporary qcow2 file we want to avoid?
> and serve it over NBD as
> raw (perhaps using an nbdkit plugin for this part)
> then qemu-img convert to destination format qcow2 as NBD client
>
> So, as long as your NBD server (via nbdkit plugin) can talk to imageio
> http server -> http client, and sized things properly according to
> qemu-img measure, then qemu-img can write qcow2 (rather than it's more
> usual raw) over the NBD connection, and when the process is complete,
> the http client will have a fully-populated qcow2 file with no temporary
> files created in the meantime.
>
Ok, it may work. What I need is:
qemu-img converting image to qcow2 format to a nbd server
nbd server writing qcow2 stream to stdout
imageio running both, sending data from nbd server stdout to client
But this means the nbd server cannot handle flow like:
- zero entire disk
- write data block 1 at offset x
- write data block 2 at offset y
We have seen this in virt-v2v nbdkit plugin on the server side using
raw output format, maybe this is not an issue with qcow2 output format?
qemu-img must know that the transport cannot seek.
When I tried in the past to do something like this it did not work, but
maybe
I was missing some options.
I think this should be implemented as a transport driver, like curl/ driver,
instead of creating these complex pipelines.
>
> > Importing images or ova files:
> >
> > http client -> imageio http server -> [qcow2 byte stream] -> qemu-img ->
> > image in any format
>
> Same sort of thing - as long as the NBD server is serving a qcow2 file
> as raw data, then the NBD client is interpreting that data as qcow2,
> then qemu-img convert should be able to convert that qcow2 stream into
> any format.
>
> Or, put another way, usually you do the conversion from qcow2 to raw at
> the server, then the client sees raw bytes:
>
> qemu-nbd -f qcow2 file.qcow2 # expose only the guest-visible bytes...
qemu-img convert -f raw nbd://host output # and write those bytes
> but in this case, you'd be serving raw bytes at the server, and letting
> the client do qcow2 conversion:
>
> qemu-nbd -f raw file.qcow2 # expose the full qcow2 file...
> qemu-img convert -f qcow2 nbd://host output # and extract the guest view
>
> where using nbdkit instead of qemu-nbd as your point of contact with the
> imageio http server may make more sense.
>
> >
> > If we will have this, we don't need to create temporary storage space
> and we
> > can avoid several image copies in the process.
> >
> > This will also improve the user experience, avoiding the wait until
> > a ova is created before the user can start downloading it.
> >
> > As for OVA files, I think it might be useful to have a tar block driver
> >> instead which would allow you to open a file inside a tar archive (you
> >> could then also directly run an OVA without extracting it first). We
> >> probably wouldn't be able to support resizing images there, but that
> >> should be okay.
> >>
> >> If you can create a tar file that reserves space for the image file
> >> without actually writing it, a possible workaround today would be using
> >> the offset/size runtime options of the raw driver to convert directly
> >> into a region inside the tar archive.
> >>
> >
> > What are the offset/size runtime options? I cannot find anything about
> > them in man qemu-img.
>
> ##
> # @BlockdevOptionsRaw:
> #
> # Driver specific block device options for the raw driver.
> #
> # @offset: position where the block device starts
> # @size:the assumed size of the device
> #
> # Since: 2.9
> ##
> { 'struct': 'BlockdevOptionsRaw',
>'base': 'BlockdevOptionsGenericFormat',
>'data': { '*offset': 'int', '*size': 'int' } }
>
>
> Yeah, it's a pity that "qemu-img create -o help -f raw" has forgotten to
> document them, the way "qemu-img create -o help -f qcow2" does for its
> options, so we should fix that.
>
Thanks for the hint, but I still don't understand for which command these
options can be used, and how.
Can you show an example of using the options?
Nir
___
Users mailing
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On 05/29/2018 04:11 PM, Nir Soffer wrote:
I think real streaming is unlikely to happen because most image formats
that QEMU supports aren't made that way. If there is a compelling
reason, we can consider it, but it would work only with very few target
formats and as such would have to be separate from existing commands.
Real streaming is exactly what we want, and we need it only for qcow2
format,
because it is our preferred way to pack images in ova.
We have 2 possible use cases:
Exporting images or ova files:
image in any format -> qemu-img -> [qcow2 byte stream] -> imageio http
server -> http client
image in any format -> qemu-img measure (to learn how large to size qcow2)
then create destination qcow2 file that large and serve it over NBD as
raw (perhaps using an nbdkit plugin for this part)
then qemu-img convert to destination format qcow2 as NBD client
So, as long as your NBD server (via nbdkit plugin) can talk to imageio
http server -> http client, and sized things properly according to
qemu-img measure, then qemu-img can write qcow2 (rather than it's more
usual raw) over the NBD connection, and when the process is complete,
the http client will have a fully-populated qcow2 file with no temporary
files created in the meantime.
Importing images or ova files:
http client -> imageio http server -> [qcow2 byte stream] -> qemu-img ->
image in any format
Same sort of thing - as long as the NBD server is serving a qcow2 file
as raw data, then the NBD client is interpreting that data as qcow2,
then qemu-img convert should be able to convert that qcow2 stream into
any format.
Or, put another way, usually you do the conversion from qcow2 to raw at
the server, then the client sees raw bytes:
qemu-nbd -f qcow2 file.qcow2 # expose only the guest-visible bytes...
qemu-img convert -f raw nbd://host output # and write those bytes
but in this case, you'd be serving raw bytes at the server, and letting
the client do qcow2 conversion:
qemu-nbd -f raw file.qcow2 # expose the full qcow2 file...
qemu-img convert -f qcow2 nbd://host output # and extract the guest view
where using nbdkit instead of qemu-nbd as your point of contact with the
imageio http server may make more sense.
If we will have this, we don't need to create temporary storage space and we
can avoid several image copies in the process.
This will also improve the user experience, avoiding the wait until
a ova is created before the user can start downloading it.
As for OVA files, I think it might be useful to have a tar block driver
instead which would allow you to open a file inside a tar archive (you
could then also directly run an OVA without extracting it first). We
probably wouldn't be able to support resizing images there, but that
should be okay.
If you can create a tar file that reserves space for the image file
without actually writing it, a possible workaround today would be using
the offset/size runtime options of the raw driver to convert directly
into a region inside the tar archive.
What are the offset/size runtime options? I cannot find anything about
them in man qemu-img.
##
# @BlockdevOptionsRaw:
#
# Driver specific block device options for the raw driver.
#
# @offset: position where the block device starts
# @size:the assumed size of the device
#
# Since: 2.9
##
{ 'struct': 'BlockdevOptionsRaw',
'base': 'BlockdevOptionsGenericFormat',
'data': { '*offset': 'int', '*size': 'int' } }
Yeah, it's a pity that "qemu-img create -o help -f raw" has forgotten to
document them, the way "qemu-img create -o help -f qcow2" does for its
options, so we should fix that.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KOTQHXWNJMRONEK7BEK7QXOWE3W4JMGB/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On Tue, May 29, 2018 at 10:43 PM Eric Blake wrote: > On 05/28/2018 05:27 AM, Arik Hadas wrote: > ... > > Few months ago people from the oVirt-storage team checked the qemu > toolset > > and replied that this capability is not yet provided, therefore we > > implemented the workaround described above. Apparently, the desired > ability > > can also be useful for the flow discussed in this thread so it worth > asking > > for it again :) > > You CAN get a logically collapsed view of storage (that is, what the > guest would see), by using an NBD export of volume V. Reading from that > volume will then pull sectors from whichever portion of the chain you > need. You can use either qemu-nbd (if no guest is writing to the > chain), or within a running qemu, you can use nbd-server-start and > nbd-server-add (over QMP) to get such an NBD server running. NBD expose the guest data, but we want the qcow2 stream - without creating a new image. Nir ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/YZYWGO2UCRB2VGSMARUTZOE5UKG5XM2C/
[ovirt-users] Re: [Qemu-block] Libvirt ERROR cannot access backing file after importing VM from OpenStack
On 05/28/2018 05:27 AM, Arik Hadas wrote: [Answering before reading the entire thread; apologies if I'm repeating things, or if I have to chime in again at other spots] Let me demonstrate briefly the flow for OVA: Let's say that we have a VM that is based on a template and has one disk and one snapshot, so its volume-chain would be: T -> S -> V (V is the volume the VM writes to, S is the backing file of V and T is the backing file of S). I tend to write backing relationships as a left arrow, as in: T <- S <- V (can be read as: S depends on T, and V depends on S) When exporting that VM to an OVA file we want the produced tar file to be comprised of: (1) OVF configuration (2) single disk volume (preferably qcow). So we need to collapse T, S, V into a single volume. Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2: (a) qemu-img convert produces a 'temporary' collapsed volume (b) make a tar file of the OVf configuration and that 'temporary' volume (c) delete the temporary volume But the fact that we produce that 'temporary' volume obviously slows down the entire operation. It would be much better if we could "open" a stream that we can read from the 'collapsed' form of that chain and stream it directly into the appropriate tar file entry, without extra writes to the storage device. Few months ago people from the oVirt-storage team checked the qemu toolset and replied that this capability is not yet provided, therefore we implemented the workaround described above. Apparently, the desired ability can also be useful for the flow discussed in this thread so it worth asking for it again :) You CAN get a logically collapsed view of storage (that is, what the guest would see), by using an NBD export of volume V. Reading from that volume will then pull sectors from whichever portion of the chain you need. You can use either qemu-nbd (if no guest is writing to the chain), or within a running qemu, you can use nbd-server-start and nbd-server-add (over QMP) to get such an NBD server running. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AZUCGA5Z54VCG3OQMH2YLHE4Y6UFXX4L/
