Better late than never - thank you all for the input, it was very useful!
With the ability to measure the collapsed form of a volume chain in
the qcow2 format, we managed to simplify and improve the process of
creating an OVA significantly. What we do now is:
1. Measure the collapsed qcow2 volumes that would be written to the OVA
2. Create the OVA file accordingly
3. Write the "metadata" of the OVA - headers and OVF, while skipping the
reserved places for disks
4. Mount each reserved place for a disk as a loopback device and convert
the volume-chain directly to it [1]

This saves us from allocating temporary disks and extra copies.
That change would hopefully land in one of the next updates of oVirt 4.2.

[1]
https://github.com/oVirt/ovirt-engine/commit/23230a131af33bfa55a3fe828660c32f7fff04d7

On Wed, May 30, 2018 at 7:39 PM Kevin Wolf <kw...@redhat.com> wrote:

> Am 30.05.2018 um 18:14 hat Arik Hadas geschrieben:
> > On Wed, May 30, 2018 at 6:33 PM, Kevin Wolf <kw...@redhat.com> wrote:
> > > I think the problem is that we're talking about two different things in
> > > one thread. If I understand correctly, what oVirt does today is:
> > >
> > > 1. qemu-img convert to create a temporary qcow2 image that merges the
> > >    whole backing chain in a single file
> > >
> > > 2. tar to create an temporary OVA archive that contains, amongst
> others,
> > >    the temporary qcow2 image. This is a second temporary file.
> > >
> > > 3. Stream this temporary OVA archive over HTTP
> > >
> >
> > Well, today we suggest users to mount a shared storage to multiple hosts
> > that reside in different oVirt/RHV deployments so they could export
> > VMs/templates as OVAs to that shared storage and import these OVAs from
> the
> > shared storage to a destination deployment. This process involves only #1
> > and #2.
> >
> > The technique you proposed earlier for writing disks directly into an
> OVA,
> > assuming that the target size can be retrieved with 'qemu-img measure',
> > sounds like a nice approach to accelerate this process. I think we should
> > really consider doing that if that's as easy as it sounds.
>
> Writing the image to a given offset in a file is the example that I gave
> further down in the mail:
>
> > > You added another host into the mix, which just receives the image
> > > content via NBD and then re-exports it as HTTP. Does this host actually
> > > exist or is it the same host where the original images are located?
> > >
> > > Because if you stay local for this step, there is no need to use NBD at
> > > all:
> > >
> > > $ ./qemu-img measure -O qcow2 ~/images/hd.img
> > > required size: 67436544
> > > fully allocated size: 67436544
> > > $ ./qemu-img create -f file /tmp/test.qcow2 67436544
> > > Formatting '/tmp/test.qcow2', fmt=file size=67436544
> > > $ ./qemu-img convert -n --target-image-opts ~/images/hd.img
> > > driver=raw,file.driver=file,file.filename=/tmp/test.qcow2,offset=65536
> > >
> > > hexdump verifies that this does the expected thing.
>
> > But #3 is definitely something we are interested in because we expect the
> > next step to be exporting the OVAs to a remote instance of Glance that
> > serves as a shared repository for the different deployments. Being able
> to
> > stream the collapsed form of a volume chain without writing anything to
> the
> > storage device would be fantastic. I think that even at the expense of
> > iterating the chain twice - once to map the structure of the jump tables
> > (right?) and once to stream the whole data.
>
> If the target is not a stupid web browser, but something actually
> virt-related like Glance, I'm sure it can offer a more suitable protocol
> than HTTP? If you could talk NBD to Glance, you'd get rid of the
> streaming requirement. I think it would make more sense to invest the
> effort there.
>
> Kevin
>

Reply via email to