[PATCH 1/3] qcow2: update_refcount(): Reset old_table_index after qcow2_cache_put()

2020-02-11 Thread Kevin Wolf
In the case that update_refcount() frees a refcount block, it evicts it from the metadata cache. Before doing so, however, it returns the currently used refcount block to the cache because it might be the same. Returning the refcount block early means that we need to reset old_table_index so that w

[PATCH 0/3] qcow2: Fix write/copy error path with data file

2020-02-11 Thread Kevin Wolf
Kevin Wolf (3): qcow2: update_refcount(): Reset old_table_index after qcow2_cache_put() qcow2: Fix qcow2_alloc_cluster_abort() for external data file iotests: Test copy offloading with external data file block/qcow2-cluster.c | 7 +-- block/qcow2-refcount.c | 1 + tests/q

[PATCH 2/3] qcow2: Fix qcow2_alloc_cluster_abort() for external data file

2020-02-11 Thread Kevin Wolf
For external data file, cluster allocations return an offset in the data file and are not refcounted. In this case, there is nothing to do for qcow2_alloc_cluster_abort(). Freeing the same offset in the qcow2 file is wrong and causes crashes in the better case or image corruption in the worse case.

[PATCH 3/3] iotests: Test copy offloading with external data file

2020-02-11 Thread Kevin Wolf
This adds a test for 'qemu-img convert' with copy offloading where the target image has an external data file. If the test hosts supports it, it tests both the case where copy offloading is supported and the case where it isn't (otherwise we just test unsupported twice). More specifically, the cas

Re: [PATCH v1 2/4] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

2020-02-11 Thread Michael S. Tsirkin
On Mon, Feb 10, 2020 at 06:34:15PM +0300, Denis Plotnikov wrote: > > > On 09.02.2020 10:49, Michael S. Tsirkin wrote: > > On Fri, Feb 07, 2020 at 11:48:05AM +0300, Denis Plotnikov wrote: > > > > > > On 05.02.2020 14:19, Stefan Hajnoczi wrote: > > > > On Tue, Feb 04, 2020 at 12:59:04PM +0300, Den

[PATCH] block: make BlockConf.*_size properties 32-bit

2020-02-11 Thread Roman Kagan
Devices (virtio-blk, scsi, etc.) and the block layer are happy to use 32-bit for logical_block_size, physical_block_size, and min_io_size. However, the properties in BlockConf are defined as uint16_t limiting the values to 32768. This appears unnecessary tight, and we've seen bigger block sizes ha

[PATCH v2] virtio: increase virtuqueue size for virtio-scsi and virtio-blk

2020-02-11 Thread Denis Plotnikov
The goal is to reduce the amount of requests issued by a guest on 1M reads/writes. This rises the performance up to 4% on that kind of disk access pattern. The maximum chunk size to be used for the guest disk accessing is limited with seg_max parameter, which represents the max amount of pices in

Re: Cross-project NBD extension proposal: NBD_INFO_INIT_STATE

2020-02-11 Thread Eric Blake
On 2/10/20 4:52 PM, Richard W.M. Jones wrote: On Mon, Feb 10, 2020 at 04:29:53PM -0600, Eric Blake wrote: On 2/10/20 4:12 PM, Richard W.M. Jones wrote: On Mon, Feb 10, 2020 at 03:37:20PM -0600, Eric Blake wrote: For now, only 2 of those 16 bits are defined: NBD_INIT_SPARSE (the image has at le

Re: [PATCH 1/7] migration/block-dirty-bitmap: refactor incoming state to be one struct

2020-02-11 Thread Vladimir Sementsov-Ogievskiy
24.01.2020 13:56, Juan Quintela wrote: Vladimir Sementsov-Ogievskiy wrote: Move enabled_bitmaps and finish_lock, which are part of incoming state to DirtyBitmapLoadState, and make static global variable to store state instead of static local one. Signed-off-by: Vladimir Sementsov-Ogievskiy --

Re: [PATCH 4/7] migration/block-dirty-bitmap: keep bitmap state for all bitmaps

2020-02-11 Thread Vladimir Sementsov-Ogievskiy
24.01.2020 14:01, Juan Quintela wrote: Vladimir Sementsov-Ogievskiy wrote: Keep bitmap state for disabled bitmaps too. Keep the state until the end of the process. It's needed for the following commit to implement bitmap postcopy canceling. Signed-off-by: Vladimir Sementsov-Ogievskiy - -

Re: [PATCH 4/7] migration/block-dirty-bitmap: keep bitmap state for all bitmaps

2020-02-11 Thread Vladimir Sementsov-Ogievskiy
22.01.2020 16:23, Vladimir Sementsov-Ogievskiy wrote: Keep bitmap state for disabled bitmaps too. Keep the state until the end of the process. It's needed for the following commit to implement bitmap postcopy canceling. Signed-off-by: Vladimir Sementsov-Ogievskiy --- migration/block-dirty-bit

Re: [PATCH v2 01/33] block: Add BlockDriver.is_format

2020-02-11 Thread Alberto Garcia
On Tue 04 Feb 2020 06:08:16 PM CET, Max Reitz wrote: > We want to unify child_format and child_file at some point. One of the > important things that set format drivers apart from other drivers is > that they do not expect other format nodes under them (except in the > backing chain). That means

Re: [PATCH v2 02/33] block: Rename BdrvChildRole to BdrvChildClass

2020-02-11 Thread Alberto Garcia
On Tue 04 Feb 2020 06:08:17 PM CET, Max Reitz wrote: > This structure nearly only contains parent callbacks for child state > changes. It cannot really reflect a child's role, because different > roles may overlap (as we will see when real roles are introduced), and > because parents can have cust

Re: [PATCH v2 03/33] block: Add BdrvChildRole

2020-02-11 Thread Eric Blake
On 2/6/20 4:49 AM, Max Reitz wrote: + +    /* Filtered child */ +    BDRV_CHILD_FILTERED = (1 << 2), I'm not sure this comment does justice for what the flag represents, but am not sure of what longer comment to put in its place. You’re right. I thought I could just rely on our .is_filt

Re: [PATCH v2 04/33] block: Add BdrvChildRole to BdrvChild

2020-02-11 Thread Eric Blake
On 2/6/20 4:53 AM, Max Reitz wrote: +++ b/block.c @@ -2381,6 +2381,7 @@ static void bdrv_replace_child(BdrvChild *child, BlockDriverState *new_bs)   BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,     const char *child_name,   

Re: [PATCH v2 15/33] block: Pull out bdrv_default_perms_for_backing()

2020-02-11 Thread Eric Blake
On 2/6/20 5:19 AM, Max Reitz wrote: On 05.02.20 21:56, Eric Blake wrote: On 2/4/20 11:08 AM, Max Reitz wrote: Signed-off-by: Max Reitz Rather light on the commit message. But looks like straightforward refactoring (with the previous patch making it easier to follow). Would this work: Righ

Re: [PATCH v2 22/33] block: Make backing files child_of_bds children

2020-02-11 Thread Eric Blake
On 2/6/20 5:33 AM, Max Reitz wrote: On 05.02.20 23:45, Eric Blake wrote: On 2/4/20 11:08 AM, Max Reitz wrote: Signed-off-by: Max Reitz Another sparse commit message (a recurring theme of this series). The subject line says 'what', and the patch appears to be faithful to that, but if a future

Re: [PATCH v2 03/33] block: Add BdrvChildRole

2020-02-11 Thread Alberto Garcia
On Tue 04 Feb 2020 06:08:18 PM CET, Max Reitz wrote: > +/* Child to COW from (backing child) */ > +BDRV_CHILD_COW = (1 << 3), Without the comment in brackets I'm not sure that I would have understood that this is meant for backing files. This is the "child that contains the data

Re: [PATCH v2 27/33] tests: Use child_of_bds instead of child_file

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- Another sparse commit message. tests/test-bdrv-drain.c | 29 + tests/test-bdrv-graph-mod.c | 6 -- 2 files changed, 21 insertions(+), 14 deletions(-) Just touches the tests - so if t

Re: [PATCH v2 28/33] block: Use bdrv_default_perms()

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: bdrv_default_perms() can decide which permission profile to use based on the BdrvChildRole, so block drivers do not need to select it explicitly. The blkverify driver now no longer shares the WRITE permission for the image to verify. We thus have to adjust t

Re: [PATCH v2 29/33] block: Make bdrv_filter_default_perms() static

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- block.c | 12 ++-- include/block/block_int.h | 10 -- 2 files changed, 6 insertions(+), 16 deletions(-) diff --git a/block.c b/block.c index 747a2973f5..091e9cc16a 100644 --- a/block.c +++ b/

Re: [PATCH v2 30/33] block: Drop bdrv_format_default_perms()

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- block.c | 19 --- include/block/block_int.h | 11 --- 2 files changed, 30 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +

Re: [PATCH v2 31/33] block: Drop child_file

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- block.c | 30 +- include/block/block_int.h | 1 - tests/test-bdrv-drain.c | 8 +++- 3 files changed, 4 insertions(+), 35 deletions(-) Reviewed-by: Eric Blake -- Eric

Re: [PATCH v2 32/33] block: Pass BdrvChildRole in remaining cases

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: These calls have no real use for the child role yet, but it will not harm to give one. Notably, the bdrv_root_attach_child() call in blockjob.c is left unmodified because there is not much the generic BlockJob object wants from its children. Signed-off-by: M

Re: [PATCH v2 33/33] block: Drop @child_class from bdrv_child_perm()

2020-02-11 Thread Eric Blake
On 2/4/20 11:08 AM, Max Reitz wrote: Implementations should decide the necessary permissions based on @role. Signed-off-by: Max Reitz --- block.c | 45 - block/backup-top.c | 3 +-- block/blkdebug.c| 3 +-- bloc

[PATCH v3 0/4] luks: add qemu-img measure support

2020-02-11 Thread Stefan Hajnoczi
v3: * Move payload offset calculation function to crypto/block.c [Max] * Zero/unallocated blocks always require disk space on encrypted files [Max] * Update qemu-iotests 178 output when changing qemu-img measure command-line options v2: * Fix uint64_t <-> size_t type mismatch in block_crypt

[PATCH v3 1/4] luks: extract qcrypto_block_calculate_payload_offset()

2020-02-11 Thread Stefan Hajnoczi
The qcow2 .bdrv_measure() code calculates the crypto payload offset. This logic really belongs in crypto/block.c where it can be reused by other image formats. The "luks" block driver will need this same logic in order to implement .bdrv_measure(), so extract the qcrypto_block_calculate_payload_of

[PATCH v3 2/4] luks: implement .bdrv_measure()

2020-02-11 Thread Stefan Hajnoczi
Add qemu-img measure support in the "luks" block driver. Signed-off-by: Stefan Hajnoczi --- block/crypto.c | 62 ++ 1 file changed, 62 insertions(+) diff --git a/block/crypto.c b/block/crypto.c index 24823835c1..453119875e 100644 --- a/block/crypt

[PATCH v3 4/4] iotests: add 282 luks qemu-img measure test

2020-02-11 Thread Stefan Hajnoczi
This test exercises the block/crypto.c "luks" block driver .bdrv_measure() code. Signed-off-by: Stefan Hajnoczi --- tests/qemu-iotests/282 | 93 ++ tests/qemu-iotests/282.out | 30 tests/qemu-iotests/group | 1 + 3 files changed, 124 insert

[PATCH v3 3/4] qemu-img: allow qemu-img measure --object without a filename

2020-02-11 Thread Stefan Hajnoczi
In most qemu-img sub-commands the --object option only makes sense when there is a filename. qemu-img measure is an exception because objects may be referenced from the image creation options instead of an existing image file. Allow --object without a filename. Signed-off-by: Stefan Hajnoczi --

Re: [PATCH v2 2/4] virtio-scsi: default num_queues to -smp N

2020-02-11 Thread Stefan Hajnoczi
On Mon, Feb 03, 2020 at 12:39:49PM +0100, Sergio Lopez wrote: > On Mon, Feb 03, 2020 at 10:57:44AM +, Daniel P. Berrangé wrote: > > On Mon, Feb 03, 2020 at 11:25:29AM +0100, Sergio Lopez wrote: > > > On Thu, Jan 30, 2020 at 10:52:35AM +, Stefan Hajnoczi wrote: > > > > On Thu, Jan 30, 2020 a

Re: [PATCH v2 2/4] virtio-scsi: default num_queues to -smp N

2020-02-11 Thread Michael S. Tsirkin
On Tue, Feb 11, 2020 at 04:20:41PM +, Stefan Hajnoczi wrote: > On Mon, Feb 03, 2020 at 12:39:49PM +0100, Sergio Lopez wrote: > > On Mon, Feb 03, 2020 at 10:57:44AM +, Daniel P. Berrangé wrote: > > > On Mon, Feb 03, 2020 at 11:25:29AM +0100, Sergio Lopez wrote: > > > > On Thu, Jan 30, 2020 a

[PATCH] nbd-client: Support leading / in NBD URI

2020-02-11 Thread Eric Blake
The NBD URI specification [1] states that only one leading slash at the beginning of the URI path component is stripped, not all such slashes. This becomes important to a patch I just proposed to nbdkit [2], which would allow the exportname to select a file embedded within an ext2 image: ext2fs de

Re: Cross-project NBD extension proposal: NBD_INFO_INIT_STATE

2020-02-11 Thread Wouter Verhelst
Hi, On Mon, Feb 10, 2020 at 10:52:55PM +, Richard W.M. Jones wrote: > But anyway ... could a flag indicating that the whole image is sparse > be useful, either as well as NBD_INIT_SPARSE or instead of it? You > could use it to avoid an initial disk trim, which is something that > mke2fs does: