Re: [Qemu-devel] [RFC v2 0/8] qemu-img: add measure sub-command
On Fri, Mar 17, 2017 at 11:45:20PM +, Nir Soffer wrote: > On Wed, Mar 15, 2017 at 11:29 AM Stefan Hajnoczi> wrote: > > > RFCv2: > > * Publishing RFC again to discuss the new user-visible interfaces. Code > > has > >changed quite a bit, I have not kept any Reviewed-by tags. > > * Rename qemu-img sub-command "measure" and API bdrv_measure() [Nir] > > * Report both "required bytes" and "fully allocated bytes" to handle the > > empty > >image file and prealloc use cases [Nir and Dan] > > * Use bdrv_getlength() instead of bdrv_nb_sectors() [Berto] > > * Rename "err" label "out" in qemu-img-cmds.c [Nir] > > * Add basic qcow2 support, doesn't support qemu-img convert from existing > > files yet > > > > RFCv1: > > * Publishing patch series with just raw support, no qcow2 yet. Please > > review > >the command-line interface and let me know if you are happy with this > >approach. > > > > Users and management tools sometimes need to know the size required for a > > new > > disk image so that an LVM volume, SAN LUN, etc can be allocated ahead of > > time. > > Image formats like qcow2 have non-trivial metadata that makes it hard to > > estimate the exact size without knowledge of file format internals. > > > > This patch series introduces a new qemu-img sub-command that calculates the > > required size for both image creation and conversion scenarios. > > > > The conversion scenario is: > > > > $ qemu-img measure -f raw -O qcow2 input.img > > required bytes: 1327680 > > fully allocated bytes: 1074069504 > > > Other commands like qemu-img info are using "size" instead of "bytes", I > guess > we like to keep the term "size" for consistency. Will fix. > Here an existing image file is taken and the output includes the space > > required > > for data from the input image file. > > > > The creation scenario is: > > > > $ qemu-img measure -O qcow2 --size 5G > > required bytes: 327680 > > fully allocated bytes: 1074069504 > > > > There is a third scenario that you may want to describe > here, extending a block device when it becomes full. In this case > we want to limit the block device to the fully allocated size, including > additional size needed for bitmaps. Currently ovirt is using virtual_size * > 1.1 > for this limit, which seems to work for images without additional bitmaps. Please elaborate, I'm not sure which use case you mean: 1. The user wishes to grow the virtual disk size because the guest has filled up the disk. 2. The block device was thinly provisioned and needs to grow so that the guest can write more data. In this case the virtual disk size stays constant. Case #1 is the qemu-img convert case. We have an existing file with some data and the new image will have a larger virtual disk size. (Nevermind that no actual new image needs to be created.) Regarding case #2, the management tool doesn't need qemu-img measure. It just increments the block device size by an arbitrary amount (e.g. 1 GB) whenever the write threshold is reached. Stefan signature.asc Description: PGP signature
Re: [Qemu-devel] [RFC v2 0/8] qemu-img: add measure sub-command
On Wed, Mar 15, 2017 at 11:29 AM Stefan Hajnocziwrote: > RFCv2: > * Publishing RFC again to discuss the new user-visible interfaces. Code > has >changed quite a bit, I have not kept any Reviewed-by tags. > * Rename qemu-img sub-command "measure" and API bdrv_measure() [Nir] > * Report both "required bytes" and "fully allocated bytes" to handle the > empty >image file and prealloc use cases [Nir and Dan] > * Use bdrv_getlength() instead of bdrv_nb_sectors() [Berto] > * Rename "err" label "out" in qemu-img-cmds.c [Nir] > * Add basic qcow2 support, doesn't support qemu-img convert from existing > files yet > > RFCv1: > * Publishing patch series with just raw support, no qcow2 yet. Please > review >the command-line interface and let me know if you are happy with this >approach. > > Users and management tools sometimes need to know the size required for a > new > disk image so that an LVM volume, SAN LUN, etc can be allocated ahead of > time. > Image formats like qcow2 have non-trivial metadata that makes it hard to > estimate the exact size without knowledge of file format internals. > > This patch series introduces a new qemu-img sub-command that calculates the > required size for both image creation and conversion scenarios. > The conversion scenario is: > > $ qemu-img measure -f raw -O qcow2 input.img > required bytes: 1327680 > fully allocated bytes: 1074069504 Other commands like qemu-img info are using "size" instead of "bytes", I guess we like to keep the term "size" for consistency. Here an existing image file is taken and the output includes the space > required > for data from the input image file. > > The creation scenario is: > > $ qemu-img measure -O qcow2 --size 5G > required bytes: 327680 > fully allocated bytes: 1074069504 > There is a third scenario that you may want to describe here, extending a block device when it becomes full. In this case we want to limit the block device to the fully allocated size, including additional size needed for bitmaps. Currently ovirt is using virtual_size * 1.1 for this limit, which seems to work for images without additional bitmaps. > Stefan Hajnoczi (8): > block: add bdrv_measure() API > raw-format: add bdrv_measure() support > qcow2: extract preallocation calculation function > qcow2: extract image creation option parsing > qcow2: add bdrv_measure() support > qemu-img: add measure subcommand > qemu-iotests: support per-format golden output files > iotests: add test 178 for qemu-img measure > > qapi/block-core.json | 19 +++ > include/block/block.h| 4 + > include/block/block_int.h| 2 + > block.c | 33 + > block/qcow2.c| 296 > ++- > block/raw-format.c | 22 +++ > qemu-img.c | 213 > qemu-img-cmds.hx | 9 ++ > tests/qemu-iotests/178 | 75 ++ > tests/qemu-iotests/178.out.qcow2 | 33 + > tests/qemu-iotests/178.out.raw | 33 + > tests/qemu-iotests/check | 5 + > tests/qemu-iotests/group | 1 + > 13 files changed, 651 insertions(+), 94 deletions(-) > create mode 100755 tests/qemu-iotests/178 > create mode 100644 tests/qemu-iotests/178.out.qcow2 > create mode 100644 tests/qemu-iotests/178.out.raw > > -- > 2.9.3 > >
[Qemu-devel] [RFC v2 0/8] qemu-img: add measure sub-command
RFCv2: * Publishing RFC again to discuss the new user-visible interfaces. Code has changed quite a bit, I have not kept any Reviewed-by tags. * Rename qemu-img sub-command "measure" and API bdrv_measure() [Nir] * Report both "required bytes" and "fully allocated bytes" to handle the empty image file and prealloc use cases [Nir and Dan] * Use bdrv_getlength() instead of bdrv_nb_sectors() [Berto] * Rename "err" label "out" in qemu-img-cmds.c [Nir] * Add basic qcow2 support, doesn't support qemu-img convert from existing files yet RFCv1: * Publishing patch series with just raw support, no qcow2 yet. Please review the command-line interface and let me know if you are happy with this approach. Users and management tools sometimes need to know the size required for a new disk image so that an LVM volume, SAN LUN, etc can be allocated ahead of time. Image formats like qcow2 have non-trivial metadata that makes it hard to estimate the exact size without knowledge of file format internals. This patch series introduces a new qemu-img sub-command that calculates the required size for both image creation and conversion scenarios. The conversion scenario is: $ qemu-img measure -f raw -O qcow2 input.img required bytes: 1327680 fully allocated bytes: 1074069504 Here an existing image file is taken and the output includes the space required for data from the input image file. The creation scenario is: $ qemu-img measure -O qcow2 --size 5G required bytes: 327680 fully allocated bytes: 1074069504 Stefan Hajnoczi (8): block: add bdrv_measure() API raw-format: add bdrv_measure() support qcow2: extract preallocation calculation function qcow2: extract image creation option parsing qcow2: add bdrv_measure() support qemu-img: add measure subcommand qemu-iotests: support per-format golden output files iotests: add test 178 for qemu-img measure qapi/block-core.json | 19 +++ include/block/block.h| 4 + include/block/block_int.h| 2 + block.c | 33 + block/qcow2.c| 296 ++- block/raw-format.c | 22 +++ qemu-img.c | 213 qemu-img-cmds.hx | 9 ++ tests/qemu-iotests/178 | 75 ++ tests/qemu-iotests/178.out.qcow2 | 33 + tests/qemu-iotests/178.out.raw | 33 + tests/qemu-iotests/check | 5 + tests/qemu-iotests/group | 1 + 13 files changed, 651 insertions(+), 94 deletions(-) create mode 100755 tests/qemu-iotests/178 create mode 100644 tests/qemu-iotests/178.out.qcow2 create mode 100644 tests/qemu-iotests/178.out.raw -- 2.9.3