Re: RFC: don't store backing filename in qcow2 image
11.12.2020 12:44, Peter Krempa wrote: On Thu, Dec 10, 2020 at 17:26:52 +0300, Vladimir Sementsov-Ogievskiy wrote: Hi all! Hi, I have an idea, that not storing backing filename in qcow2 image at all may be a good thing. I'll give some reasons and want to know what do you think about it. 1. Libvirt has to manage and keep in mind backing chains anyway. This means, that storing this information in qcow2 header is a source of bugs when we update it in one place but failed/forget to update in another. Of course, Libvirt is not the only user of qemu.. But we are moving to "blockdev" anyway, when management tool should control all node-names at least. It would be strange to not control the relations between images in the same time. At the same time many users depend on this. If you move in images from another host, you'd have to remember the dependencies/order. 2. backing file name specified in qcow2 metadata doesn't relate to any other thing, and nothing rely on it. 3. calculating and updating backing file name in Qemu is a headache: - with some options specified or with filters we risk to write json filenames into qcow2 metadata, which is almost never what user wants. Also, json may exceed the qcow2 limitation of backing_file_size to be <= 1023 As long as it works (libvirt and qemu have parsers for json:) I don't think the user cares. - updating it in transactional way for read-only image during reopen, when another transactional permission update is ongoing is difficult (who know, how to do it?) (remember recent d669ed6ab02849 "block: make bdrv_drop_intermediate() less wrong") 4. Moving qcow2 files to another directory is a problem: you should care to update backing file names in all dependent qcow2 images. Or alternatively use relative names. So, what about moving libvirt (at least) to not rely on backing file name stored in qcow2 image? Backing chain then should be in xml? Is it hard or not? Finally, will it make the code simpler, or more difficult? Then, if the idea is good in general, what to do on Qemu part? If we want to finally get rid of problem code (see [3.]) we should deprecate something.. Just deprecate support for qcow2 images with backing file specified, requiring user always specify backing chain by hand? I don't see anything that should be changed in qcow2 format itself: no reason to add some kind of restricted bits, etc.. I think this will create headaches for many users. Libvirt does support specification of the chain manually, but doesn't mandate it. It's also a fairly recent addition to libvirt so I doubt that any other project which uses libvirt only for a part of the functionality (such as oVirt or openstack) picked up the full specification of chain in the XML. The problem here is that libvirt isn't used for the whole knowledge state here. Rather projects like oVirt feed us a new XML every single time. This means that they'd need to start keeping the chain info internally too. Rather they currently rely on our detection code and the proper setting of paths in the image, and thus removing it would be a rather serious regression in behaviour, which would be visible beyond libvirt without any way for us to make it opaque to higher levels. Thanks for explanation. Hmm, yes, it sounds like we'll never drop support for filename-based backing chain. And if we can't drop the support, no reason to deprecate it. -- Best regards, Vladimir
Re: RFC: don't store backing filename in qcow2 image
On Thu, Dec 10, 2020 at 17:26:52 +0300, Vladimir Sementsov-Ogievskiy wrote: > Hi all! Hi, > > I have an idea, that not storing backing filename in qcow2 image at all may > be a good thing. I'll give some reasons and want to know what do you think > about it. > > 1. Libvirt has to manage and keep in mind backing chains anyway. > > This means, that storing this information in qcow2 header is a source of bugs > when we update it in one place but failed/forget to update in another. Of > course, Libvirt is not the only user of qemu.. But we are moving to > "blockdev" anyway, when management tool should control all node-names at > least. It would be strange to not control the relations between images in the > same time. At the same time many users depend on this. If you move in images from another host, you'd have to remember the dependencies/order. > 2. backing file name specified in qcow2 metadata doesn't relate to any other > thing, and nothing rely on it. > > 3. calculating and updating backing file name in Qemu is a headache: >- with some options specified or with filters we risk to write json > filenames into qcow2 metadata, which is almost never what user wants. Also, > json may exceed the qcow2 limitation of backing_file_size to be <= 1023 As long as it works (libvirt and qemu have parsers for json:) I don't think the user cares. >- updating it in transactional way for read-only image during reopen, when > another transactional permission update is ongoing is difficult (who know, > how to do it?) (remember recent d669ed6ab02849 "block: make > bdrv_drop_intermediate() less wrong") > > 4. Moving qcow2 files to another directory is a problem: you should care to > update backing file names in all dependent qcow2 images. Or alternatively use relative names. > So, what about moving libvirt (at least) to not rely on backing file name > stored in qcow2 image? Backing chain then should be in xml? Is it hard or > not? Finally, will it make the code simpler, or more difficult? > > > Then, if the idea is good in general, what to do on Qemu part? If we want to > finally get rid of problem code (see [3.]) we should deprecate something.. > Just deprecate support for qcow2 images with backing file specified, > requiring user always specify backing chain by hand? I don't see anything > that should be changed in qcow2 format itself: no reason to add some kind of > restricted bits, etc.. I think this will create headaches for many users. Libvirt does support specification of the chain manually, but doesn't mandate it. It's also a fairly recent addition to libvirt so I doubt that any other project which uses libvirt only for a part of the functionality (such as oVirt or openstack) picked up the full specification of chain in the XML. The problem here is that libvirt isn't used for the whole knowledge state here. Rather projects like oVirt feed us a new XML every single time. This means that they'd need to start keeping the chain info internally too. Rather they currently rely on our detection code and the proper setting of paths in the image, and thus removing it would be a rather serious regression in behaviour, which would be visible beyond libvirt without any way for us to make it opaque to higher levels.
RFC: don't store backing filename in qcow2 image
Hi all! I have an idea, that not storing backing filename in qcow2 image at all may be a good thing. I'll give some reasons and want to know what do you think about it. 1. Libvirt has to manage and keep in mind backing chains anyway. This means, that storing this information in qcow2 header is a source of bugs when we update it in one place but failed/forget to update in another. Of course, Libvirt is not the only user of qemu.. But we are moving to "blockdev" anyway, when management tool should control all node-names at least. It would be strange to not control the relations between images in the same time. 2. backing file name specified in qcow2 metadata doesn't relate to any other thing, and nothing rely on it. 3. calculating and updating backing file name in Qemu is a headache: - with some options specified or with filters we risk to write json filenames into qcow2 metadata, which is almost never what user wants. Also, json may exceed the qcow2 limitation of backing_file_size to be <= 1023 - updating it in transactional way for read-only image during reopen, when another transactional permission update is ongoing is difficult (who know, how to do it?) (remember recent d669ed6ab02849 "block: make bdrv_drop_intermediate() less wrong") 4. Moving qcow2 files to another directory is a problem: you should care to update backing file names in all dependent qcow2 images. So, what about moving libvirt (at least) to not rely on backing file name stored in qcow2 image? Backing chain then should be in xml? Is it hard or not? Finally, will it make the code simpler, or more difficult? Then, if the idea is good in general, what to do on Qemu part? If we want to finally get rid of problem code (see [3.]) we should deprecate something.. Just deprecate support for qcow2 images with backing file specified, requiring user always specify backing chain by hand? I don't see anything that should be changed in qcow2 format itself: no reason to add some kind of restricted bits, etc.. -- Best regards, Vladimir
