Re: [PATCH v2 12/20] iotests: 56: prepare for backup over block-copy

2020-07-23 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > After introducing parallel async copy requests instead of plain > cluster-by-cluster copying loop, we'll have to wait for paused status, > as we need to wait for several parallel request. So, let's gently wait > instead of just asserting

Re: [PATCH v2 11/20] qapi: backup: add x-max-chunk and x-max-workers parameters

2020-07-23 Thread Max Reitz
On 22.07.20 14:22, Max Reitz wrote: > On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: >> Add new parameters to configure future backup features. The patch >> doesn't introduce aio backup requests (so we actually have only one >> worker) neither requests larger th

Re: [PATCH v2 11/20] qapi: backup: add x-max-chunk and x-max-workers parameters

2020-07-22 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > Add new parameters to configure future backup features. The patch > doesn't introduce aio backup requests (so we actually have only one > worker) neither requests larger than one cluster. Still, formally we > satisfy these maximums anyway,

Re: [PATCH v2 10/20] job: call job_enter from job_user_pause

2020-07-22 Thread Max Reitz
ff-by: Vladimir Sementsov-Ogievskiy > --- > job.c | 1 + > 1 file changed, 1 insertion(+) Sounds reasonable to me, although I’d prefer an opinion from John. So, a middle-weak: Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 09/20] blockjob: add set_speed to BlockJobDriver

2020-07-22 Thread Max Reitz
2 ++ > blockjob.c | 6 ++ > 2 files changed, 8 insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 08/20] block/block-copy: add block_copy_cancel

2020-07-22 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > Add function to cancel running async block-copy call. It will be used > in backup. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > include/block/block-copy.h | 7 +++ > block/block-copy.c | 22 +++--- >

Re: [PATCH v2 07/20] block/block-copy: add ratelimit to block-copy

2020-07-22 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > We are going to directly use one async block-copy operation for backup > job, so we need rate limitator. %s/limitator/limiter/g, I think. > We want to maintain current backup behavior: only background copying is > limited and

Re: [PATCH v2 06/20] block/block-copy: add max_chunk and max_workers parameters

2020-07-22 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > They will be used for backup. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > include/block/block-copy.h | 5 + > block/block-copy.c | 10 -- > 2 files changed, 13 insertions(+), 2 deletions(-) > > diff --git

[PATCH for-5.2 v3 2/3] iotests.py: Add wait_for_runstate()

2020-07-22 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 5 + 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 3590ed78a0..20645a6e7d 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -28,6

[PATCH for-5.2 v3 3/3] iotests: Test node/bitmap aliases during migration

2020-07-22 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/300 | 515 + tests/qemu-iotests/300.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 521 insertions(+) create mode 100755 tests/qemu-iotests/300 create mode 100644 tests/qemu-iotests/300

[PATCH for-5.2 v3 1/3] migration: Add block-bitmap-mapping parameter

2020-07-22 Thread Max Reitz
(on the source, the destination, or even both with arbitrary aliases in the migration stream). Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- qapi/migration.json| 104 - migration/migration.h | 3 + migration/block-dirty-bitmap.c | 373

[PATCH for-5.2 v3 0/3] migration: Add block-bitmap-mapping parameter

2020-07-22 Thread Max Reitz
during migration' Max Reitz (3): migration: Add block-bitmap-mapping parameter iotests.py: Add wait_for_runstate() iotests: Test node/bitmap aliases during migration qapi/migration.json| 104 ++- migration/migration.h | 3 + migration/block-dirty-bitmap.c | 373

Re: [PATCH for-5.1 v2 2/2] iotests: Test sparseness for qemu-img convert -n

2020-07-21 Thread Max Reitz
On 21.07.20 15:55, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/122 | 30 ++ > tests/qemu-iotests/122.out | 17 + > 2 files changed, 47 insertions(+) Reviewed-by: Max Reitz signature.asc Description:

[PULL 2/3] qemu-iotests: add testcase for bz #1857490

2020-07-21 Thread Max Reitz
From: Maxim Levitsky Test that we can't write-share raw luks images by default, but we still can with share-rw=on Signed-off-by: Maxim Levitsky Message-Id: <20200719122059.59843-3-mlevi...@redhat.com> Signed-off-by: Max Reitz --- tests/qemu-iotests/296

[PULL 1/3] block/crypto: disallow write sharing by default

2020-07-21 Thread Max Reitz
ement") Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1857490 Signed-off-by: Maxim Levitsky Message-Id: <20200719122059.59843-2-mlevi...@redhat.com> Signed-off-by: Max Reitz --- block/crypto.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/crypto.c b/bloc

[PULL 3/3] block: fix bdrv_aio_cancel() for ENOMEDIUM requests

2020-07-21 Thread Max Reitz
: https://bugs.launchpad.net/qemu/+bug/1878255 Originally-by: Philippe Mathieu-Daudé Signed-off-by: Stefan Hajnoczi Message-Id: <20200720100141.129739-1-stefa...@redhat.com> Signed-off-by: Max Reitz --- block/block-backend.c | 8 1 file changed, 8 insertions(+) diff --git a/block/bloc

[PULL 0/3] Block patches for 5.1

2020-07-21 Thread Max Reitz
The following changes since commit af3d69058e09bede9900f266a618ed11f76f49f3: Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200720' into staging (2020-07-20 15:58:07 +0100) are available in the Git repository at: https://github.com/XanClic/qemu.git

Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n

2020-07-21 Thread Max Reitz
On 21.07.20 13:20, Kevin Wolf wrote: > Am 21.07.2020 um 12:19 hat Max Reitz geschrieben: >> On 20.07.20 15:18, Kevin Wolf wrote: >>> Signed-off-by: Kevin Wolf >>> --- >>> tests/qemu-iotests/122 | 34 ++

Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n

2020-07-21 Thread Max Reitz
On 20.07.20 15:18, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/122 | 34 ++ > tests/qemu-iotests/122.out | 17 + > 2 files changed, 51 insertions(+) > > diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122

Re: [PATCH for-5.1 1/2] qcow2: Implement v2 zero writes with discard if possible

2020-07-21 Thread Max Reitz
mpty. > > Reported-by: Nir Soffer > Signed-off-by: Kevin Wolf > --- > block/qcow2-cluster.c | 9 - > 1 file changed, 8 insertions(+), 1 deletion(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH for-5.1] block: fix bdrv_aio_cancel() for ENOMEDIUM requests

2020-07-21 Thread Max Reitz
On 20.07.20 12:01, Stefan Hajnoczi wrote: > bdrv_aio_cancel() calls aio_poll() on the AioContext for the given I/O > request until it has completed. ENOMEDIUM requests are special because > there is no BlockDriverState when the drive has no medium! > > Define a .get_aio_context() function for

Re: [PATCH v2 05/20] block/block-copy: implement block_copy_async

2020-07-21 Thread Max Reitz
On 17.07.20 17:24, Vladimir Sementsov-Ogievskiy wrote: > 17.07.2020 17:00, Max Reitz wrote: >> On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: >>> We'll need async block-copy invocation to use in backup directly. >>> >>> Signed-off-by: Vladimir Sement

Re: [PATCH v2 3/3] iotests: Test node/bitmap aliases during migration

2020-07-21 Thread Max Reitz
On 20.07.20 20:02, Vladimir Sementsov-Ogievskiy wrote: > 16.07.2020 16:53, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/qemu-iotests/300 | 511 + >>   tests/qemu-iotests/300.out |   5 + >>   tests/qemu-iot

Re: [PATCH v2 2/3] iotests.py: Add wait_for_runstate()

2020-07-21 Thread Max Reitz
On 20.07.20 18:46, Vladimir Sementsov-Ogievskiy wrote: > 16.07.2020 16:53, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/qemu-iotests/iotests.py | 4 >>   1 file changed, 4 insertions(+) >> >> diff --git a/tests/qemu-iotests/iotest

Re: [PATCH v2 1/3] migration: Add block-bitmap-mapping parameter

2020-07-21 Thread Max Reitz
On 20.07.20 18:31, Vladimir Sementsov-Ogievskiy wrote: > 16.07.2020 16:53, Max Reitz wrote: >> This migration parameter allows mapping block node names and bitmap >> names to aliases for the purpose of block dirty bitmap migration. >> >> This way, management tools can us

Re: [PATCH 0/2] Fix for write sharing on luks raw images

2020-07-20 Thread Max Reitz
On 19.07.20 14:20, Maxim Levitsky wrote: > A rebase gone wrong, and I ended up allowing a luks image > to be opened at the same time by two VMs without any warnings/overrides. > > Fix that and also add an iotest to prevent this from happening. > > Best regards, > Maxim Levisky > > Maxim

Re: [PATCH v2 05/20] block/block-copy: implement block_copy_async

2020-07-17 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > We'll need async block-copy invocation to use in backup directly. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > include/block/block-copy.h | 13 + > block/block-copy.c | 40

Re: [PATCH v2 04/20] block/block-copy: More explicit call_state

2020-07-17 Thread Max Reitz
> +ret = block_copy_dirty_clusters(call_state); It’s possible that much of this code will change in a future patch of this series, but as it is, it might be nice to make block_copy_dirty_clusters’s argument a const pointer so it’s clear that the call to block_copy_wait_one() bel

Re: [PATCH v2 03/20] qapi: backup: add x-use-copy-range parameter

2020-07-17 Thread Max Reitz
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote: > Add parameter to enable/disable copy_range. Keep current default for > now (enabled). Why x-, though? I can’t think of a reason why we would have to remove this. > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > qapi/block-core.json

Re: [PATCH v2 02/20] iotests: 129 don't check backup "busy"

2020-07-17 Thread Max Reitz
tests/129 | 1 - > 1 file changed, 1 deletion(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH for-5.1 3/3] file-posix: Fix leaked fd in raw_open_common() error path

2020-07-17 Thread Max Reitz
On 17.07.20 12:54, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > block/file-posix.c | 3 +++ > 1 file changed, 3 insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH for-5.1 2/3] file-posix: Fix check_hdev_writable() with auto-read-only

2020-07-17 Thread Max Reitz
ff-by: Kevin Wolf > --- > block/file-posix.c | 33 + > 1 file changed, 21 insertions(+), 12 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH for-5.1 1/3] file-posix: Move check_hdev_writable() up

2020-07-17 Thread Max Reitz
33 insertions(+), 33 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH for-5.1 v2 1/2] block: Require aligned image size to avoid assertion failure

2020-07-17 Thread Max Reitz
On 17.07.20 13:32, Kevin Wolf wrote: > Am 17.07.2020 um 13:02 hat Max Reitz geschrieben: >> On 16.07.20 16:26, Kevin Wolf wrote: >>> Unaligned requests will automatically be aligned to bl.request_alignment >>> and we can't extend write requests to access space beyo

Re: [PATCH for-5.1 v2 2/2] file-posix: Allow byte-aligned O_DIRECT with NFS

2020-07-17 Thread Max Reitz
ystem magic and disable > s->needs_alignment for NFS. This way, O_DIRECT can still be used on NFS > for images that are not aligned to 4k. > > Signed-off-by: Kevin Wolf > Reviewed-by: Eric Blake > --- > block/file-posix.c | 26 +- > 1 file changed, 2

Re: [PATCH for-5.1 v2 1/2] block: Require aligned image size to avoid assertion failure

2020-07-17 Thread Max Reitz
(i.e., qcow2 in practice) because WRITE_UNCHANGED is only used by COR and block jobs doing something with a backing chain, so it shouldn’t matter in practice, but, well. So, either way: Reviewed-by: Max Reitz > +!(cumulative_perms & BLK_PERM_RESIZE)) > +{ &

Re: [PATCH v7 21/47] block: Use CAFs in bdrv_refresh_filename()

2020-07-16 Thread Max Reitz
On 15.07.20 14:52, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> bdrv_refresh_filename() and the kind of related bdrv_dirname() should >> look to the primary child when they wish to copy the underlying file's >> filename. >> >> Signed-off-b

Re: [PATCH v7 20/47] block: Iterate over children in refresh_limits

2020-07-16 Thread Max Reitz
On 14.07.20 20:37, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Instead of looking at just bs->file and bs->backing, we should look at >> all children that could end up receiving forwarded requests. >> >> Signed-off-by: Max R

Re: [PATCH v7 19/47] vmdk: Drop vmdk_co_flush()

2020-07-16 Thread Max Reitz
On 14.07.20 16:52, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Before HEAD^, we needed this because bdrv_co_flush() by itself would >> only flush bs->file.  With HEAD^, bdrv_co_flush() will flush all >> children on which a WRITE or WRITE_UNCHANGED p

Re: [PATCH v7 17/47] block: Re-evaluate backing file handling in reopen

2020-07-16 Thread Max Reitz
On 10.07.20 21:42, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Reopening a node's backing child needs a bit of special handling because >> the "backing" child has different defaults than all other children >> (among other things).  Adding fi

Re: [PATCH v7 14/47] stream: Deal with filters

2020-07-16 Thread Max Reitz
On 10.07.20 19:41, Andrey Shinkevich wrote: > On 10.07.2020 18:24, Max Reitz wrote: >> On 09.07.20 16:52, Andrey Shinkevich wrote: >>> On 25.06.2020 18:21, Max Reitz wrote: >>>> Because of the (not so recent anymore) changes that make the stream job >>>>

Re: [PATCH v7 02/47] block: Add chain helper functions

2020-07-16 Thread Max Reitz
On 13.07.20 12:18, Vladimir Sementsov-Ogievskiy wrote: > 25.06.2020 18:21, Max Reitz wrote: >> Add some helper functions for skipping filters in a chain of block >> nodes. >> >> Signed-off-by: Max Reitz >> --- >>   include/block/block_int.h |  3

Re: [PATCH v7 01/47] block: Add child access functions

2020-07-16 Thread Max Reitz
On 13.07.20 11:06, Vladimir Sementsov-Ogievskiy wrote: > 25.06.2020 18:21, Max Reitz wrote: >> There are BDS children that the general block layer code can access, >> namely bs->file and bs->backing.  Since the introduction of filters and >> external data files, their

Re: [PATCH for-5.1] iotests/030: Reduce job speed to make race less likely

2020-07-16 Thread Max Reitz
Reported-by: Peter Maydell > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/030 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

[PATCH v2 2/3] iotests.py: Add wait_for_runstate()

2020-07-16 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 4 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 3590ed78a0..fb240a334c 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -803,6

[PATCH v2 3/3] iotests: Test node/bitmap aliases during migration

2020-07-16 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/300 | 511 + tests/qemu-iotests/300.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 517 insertions(+) create mode 100755 tests/qemu-iotests/300 create mode 100644 tests/qemu-iotests/300

[PATCH v2 1/3] migration: Add block-bitmap-mapping parameter

2020-07-16 Thread Max Reitz
(on the source, the destination, or even both with arbitrary aliases in the migration stream). Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- Vladimir noted in v1 that it would be better to ignore bitmaps whose names aren't mapped, or that are on nodes whose names aren't

[PATCH v2 0/3] migration: Add block-bitmap-mapping parameter

2020-07-16 Thread Max Reitz
own] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/3:[0117] [FC] 'migration: Add block-bitmap-mapping parameter' 002/3:[] [--] 'iotests.py: Add wait_for_runstate()' 003/3:[0202] [FC] 'iotests: Test node/bitmap aliases during migration'

Re: [PATCH v11 00/34] Add subcluster allocation to qcow2

2020-07-15 Thread Max Reitz
On 15.07.20 13:51, Alberto Garcia wrote: > On Tue, Jul 14, 2020 at 02:56:57PM +0200, Max Reitz wrote: >> Thanks, fixed up patch 21 (by initializing expected_type to just >> anything, I chose QCOW2_SUBCLUSTER_NORMAL, which I preferred over >> just using 0), resolved some iotes

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-14 Thread Max Reitz
On 14.07.20 13:08, Kevin Wolf wrote: > Am 14.07.2020 um 11:56 hat Max Reitz geschrieben: >> On 13.07.20 16:29, Kevin Wolf wrote: >>> Am 13.07.2020 um 13:19 hat Max Reitz geschrieben: >>>> On 10.07.20 16:21, Kevin Wolf wrote: >>>>> Una

Re: [PATCH v11 00/34] Add subcluster allocation to qcow2

2020-07-14 Thread Max Reitz
On 10.07.20 18:12, Alberto Garcia wrote: > Hi, > > here's the new version of the patches to add subcluster allocation > support to qcow2. > > Please refer to the cover letter of the first version for a full > description of the patches: > >

Re: [PATCH v11 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-07-14 Thread Max Reitz
d-by: Eric Blake > Reviewed-by: Max Reitz > --- > qapi/block-core.json | 7 +++ > block/qcow2.h| 8 ++- > include/block/block_int.h| 1 + > block/qcow2.c| 66 ++-- > tests/qemu-iotests/031.

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-14 Thread Max Reitz
On 13.07.20 16:29, Kevin Wolf wrote: > Am 13.07.2020 um 13:19 hat Max Reitz geschrieben: >> On 10.07.20 16:21, Kevin Wolf wrote: >>> Unaligned requests will automatically be aligned to bl.request_alignment >>> and we don't want to extend requests to access space beyond t

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-13 Thread Max Reitz
On 13.07.20 13:19, Max Reitz wrote: > On 10.07.20 16:21, Kevin Wolf wrote: >> Unaligned requests will automatically be aligned to bl.request_alignment >> and we don't want to extend requests to access space beyond the end of >> the image, so it's required that the

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-13 Thread Max Reitz
On 10.07.20 16:21, Kevin Wolf wrote: > Unaligned requests will automatically be aligned to bl.request_alignment > and we don't want to extend requests to access space beyond the end of > the image, so it's required that the image size is aligned. > > With write requests, this could cause

Re: [PATCH] block: Raise an error when backing file parameter is an empty string

2020-07-13 Thread Max Reitz
On 17.06.20 20:27, Sorry :/ Connor Kuehl wrote: > Providing an empty string for the backing file parameter like so: > > qemu-img create -f qcow2 -b '' /tmp/foo > > allows the flow of control to reach and subsequently fail an assert > statement because passing an empty string to > >

Re: [PATCH v6 01/10] qemu-img: Flush stdout before before potential stderr messages

2020-07-13 Thread Max Reitz
On 06.07.20 22:39, Eric Blake wrote: > During 'qemu-img create ... 2>&1', if --quiet is not in force, we can > end up with buffered I/O in stdout that was produced before failure, > but which appears in output after failure. This is confusing; the fix > is to flush stdout prior to attempting

Re: [PATCH for-5.1] file-posix: Mitigate file fragmentation with extent size hints

2020-07-13 Thread Max Reitz
On 10.07.20 18:12, Max Reitz wrote: > On 07.07.20 18:17, Kevin Wolf wrote: >> Am 07.07.2020 um 16:23 hat Kevin Wolf geschrieben: >>> Espeically when O_DIRECT is used with image files so that the page cache >>> indirection can't cause a merge of allocating requests,

Re: [PATCH 2/2] iotests: Set LC_ALL=C for sort

2020-07-13 Thread Max Reitz
On 11.07.20 10:57, Alex Bennée wrote: > > Max Reitz writes: > >> Otherwise the result is basically unpredictable. >> >> (Note that the precise environment variable to control sorting order is >> LC_COLLATE, but LC_ALL overrides LC_COLLATE, and we do not want the

Re: [PATCH 0/2] iotests: More _filter_img_create fixes

2020-07-13 Thread Max Reitz
On 10.07.20 19:18, John Snow wrote: > > > On 7/10/20 12:32 PM, Max Reitz wrote: >> Hi, >> >> I’m sorry. >> >> John, could I ask you to test whether this series fixes the problems >> you’re seeing? >> > > This is based on kwolf/block,

[PATCH 1/2] iotests: Drop readarray from _do_filter_img_create

2020-07-10 Thread Max Reitz
Some systems where we run tests on do not have a 4.x bash, so they do not have readarray. While it looked a bit nicer than messing with `head` and `tail`, we do not really need it, so we might as well not use it. Reported-by: Claudio Fontana Signed-off-by: Max Reitz --- tests/qemu-iotests

[PATCH 2/2] iotests: Set LC_ALL=C for sort

2020-07-10 Thread Max Reitz
-by: Max Reitz --- tests/qemu-iotests/common.filter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 345c3ca03e..4fd5c29b2a 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter

[PATCH 0/2] iotests: More _filter_img_create fixes

2020-07-10 Thread Max Reitz
Hi, I’m sorry. John, could I ask you to test whether this series fixes the problems you’re seeing? Max Reitz (2): iotests: Drop readarray from _do_filter_img_create iotests: Set LC_ALL=C for sort tests/qemu-iotests/common.filter | 12 ++-- 1 file changed, 6 insertions(+), 6

Re: [PATCH for-5.1] file-posix: Mitigate file fragmentation with extent size hints

2020-07-10 Thread Max Reitz
On 07.07.20 18:17, Kevin Wolf wrote: > Am 07.07.2020 um 16:23 hat Kevin Wolf geschrieben: >> Espeically when O_DIRECT is used with image files so that the page cache >> indirection can't cause a merge of allocating requests, the file will >> fragment on the file system layer, with a potentially

Re: [PATCH v7 14/47] stream: Deal with filters

2020-07-10 Thread Max Reitz
On 09.07.20 17:13, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Because of the (not so recent anymore) changes that make the stream job >> independent of the base node and instead track the node above it, we >> have to split that "bottom" node

Re: [PATCH v7 14/47] stream: Deal with filters

2020-07-10 Thread Max Reitz
On 09.07.20 16:52, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Because of the (not so recent anymore) changes that make the stream job >> independent of the base node and instead track the node above it, we >> have to split that "bottom" node

Re: [PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command

2020-07-10 Thread Max Reitz
On 09.07.20 17:09, Peter Maydell wrote: > On Mon, 6 Jul 2020 at 11:05, Max Reitz wrote: >> >> From: Maxim Levitsky >> >> blockdev-amend will be used similiar to blockdev-create >> to allow on the fly changes of the structure of the format based block >>

[PATCH] block/amend: Check whether the node exists

2020-07-10 Thread Max Reitz
: ced914d0ab9fb2c900f873f6349a0b8eecd1fdbe Signed-off-by: Max Reitz --- block/amend.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/amend.c b/block/amend.c index f4612dcf08..392df9ef83 100644 --- a/block/amend.c +++ b/block/amend.c @@ -69,8 +69,12 @@ void qmp_x_blockdev_amend(const

Re: [PATCH v10 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-07-09 Thread Max Reitz
d-by: Eric Blake > Reviewed-by: Max Reitz > --- After running basically all test combinations I usually run, I noticed that 191 should also be modified to drop the extended_l2 option from the qemu-img info output (down under “checking image base” and “checking image layer”, where there are

Re: [PATCH v10 34/34] iotests: Add tests for qcow2 images with extended L2 entries

2020-07-09 Thread Max Reitz
g extended L2 entries is not supported With Maxim’s patches, this code in patch 31 became unnecessary, so it should now read “qemu-img: Invalid parameter 'extended_l2'\nThis option is only supported for image creation”. With that fixed: Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v10 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2020-07-09 Thread Max Reitz
d-by: Eric Blake > Reviewed-by: Max Reitz > --- (This requires quite a bit of a rebase, but nothing spectacularly interesting, I think.) signature.asc Description: OpenPGP digital signature

Re: [PATCH v10 28/34] qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()

2020-07-09 Thread Max Reitz
ed. > > Signed-off-by: Alberto Garcia > --- > block/qcow2.h | 4 +-- > block/qcow2-cluster.c | 81 +++ > block/qcow2.c | 33 +++++- > 3 files changed, 94 insertions(+), 24 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v10 21/34] qcow2: Add subcluster support to qcow2_get_host_offset()

2020-07-09 Thread Max Reitz
- > 2 files changed, 92 insertions(+), 96 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

[PATCH] iotests: Simplify _filter_img_create() a bit

2020-07-09 Thread Max Reitz
(And fix 020 to work with that.) Reported-by: Kevin Wolf Signed-off-by: Max Reitz --- Kevin noted that the changes to _filter_img_create broke Eric's patch to flush the Formatting line out before a potential error message. This patch should fix it (and the diff stat is negative, so that's nice). ---

Re: [PATCH v10 02/34] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()

2020-07-09 Thread Max Reitz
- > 3 files changed, 32 insertions(+), 37 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [PATCH v7 12/47] block: Use bdrv_filter_(bs|child) where obvious

2020-07-09 Thread Max Reitz
On 08.07.20 20:24, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Places that use patterns like >> >> if (bs->drv->is_filter && bs->file) { >> ... something about bs->file->bs ... >> } >> >>

Re: [PATCH v7 02/47] block: Add chain helper functions

2020-07-09 Thread Max Reitz
On 08.07.20 19:20, Andrey Shinkevich wrote: > On 25.06.2020 18:21, Max Reitz wrote: >> Add some helper functions for skipping filters in a chain of block >> nodes. >> >> Signed-off-by: Max Reitz >> --- >>   include/block/block_int.h |  3

Re: [PATCH v7 00/47] block: Deal with filters

2020-07-09 Thread Max Reitz
On 08.07.20 22:47, Eric Blake wrote: > On 7/8/20 12:20 PM, Andrey Shinkevich wrote: >> On 25.06.2020 18:21, Max Reitz wrote: >>> v6: >>> https://lists.nongnu.org/archive/html/qemu-devel/2019-08/msg01715.html >>> >>> Branch: https://github.com/XanClic/

Re: [PATCH v7 00/47] block: Deal with filters

2020-07-09 Thread Max Reitz
On 08.07.20 22:37, Eric Blake wrote: > On 7/8/20 2:46 PM, Andrey Shinkevich wrote: >> >> On 08.07.2020 20:32, Eric Blake wrote: >>> On 7/8/20 12:20 PM, Andrey Shinkevich wrote: >>>> On 25.06.2020 18:21, Max Reitz wrote: >>>>> v6: >>>

[PULL 31/31] qed: Simplify backing reads

2020-07-06 Thread Max Reitz
Sementsov-Ogievskiy Message-Id: <20200528094405.145708-11-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- block/qed.h | 1 - block/qed.c | 64 + 2 files changed, 6 insertions(+), 59 deletions(-) diff --git a/block/qed.h b/block/qed.h

[PULL 30/31] block: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
this behavior for .supports_backing and drop .unallocated_blocks_are_zero Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200528094405.145708-10-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- include/block/block.h | 5 - include/block/block

[PULL 23/31] block: inline bdrv_unallocated_blocks_are_zero()

2020-07-06 Thread Max Reitz
: <20200528094405.145708-3-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- include/block/block.h | 1 - block.c | 15 --- block/io.c| 11 --- 3 files changed, 8 insertions(+), 19 deletions(-) diff --git a/include/block/block.h b/include

[PULL 27/31] block/iscsi: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
the field: bdrv_co_block_status()). Drop it now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200528094405.145708-7-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- block/iscsi.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/iscsi.c

[PULL 20/31] block/qcow2: implement blockdev-amend

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Currently the implementation only supports amending the encryption options, unlike the qemu-img version Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-14-mlevi...@redhat.com> Signed-off-b

[PULL 21/31] iotests: add tests for blockdev-amend

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This commit adds two tests that cover the new blockdev-amend functionality of luks and qcow2 driver Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé [mreitz: Let 295 verify that LUKS works; drop 295 and 296 from the auto group] Signed-off-by: Max

[PULL 28/31] block/file-posix: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
Message-Id: <20200528094405.145708-8-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- block/file-posix.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index 3ab8f5a0fa..d86ea57769 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -

[PULL 15/31] block/crypto: implement the encryption key management

2020-07-06 Thread Max Reitz
for the idea of unsharing read, rather that write permission which allows to avoid cases when the other user had opened the image read-only. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-8-mlevi...@redhat.com> Sign

[PULL 25/31] block/vpc: return ZERO block-status when appropriate

2020-07-06 Thread Max Reitz
> [mreitz: qemu-io -c map as used by iotest 146 now reports everything as allocated; in order to make the test do something useful, we use qemu-img map --output=json now] Signed-off-by: Max Reitz --- block/vpc.c| 3 +- tests/qemu-iotests/146 | 60

[PULL 29/31] block/vhdx: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200528094405.145708-9-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- block/vhdx.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/vhdx.c b/block/vhdx.c index fa9e544a5e..645dc4b4f4 100644 --- a/block/vhdx.c

[PULL 14/31] block/crypto: rename two functions

2020-07-06 Thread Max Reitz
: <20200608094030.670121-7-mlevi...@redhat.com> Signed-off-by: Max Reitz --- block/crypto.c | 25 - 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index dcf8b42bb2..1960b47ceb 100644 --- a/block/crypto.c +++ b/block/cr

[PULL 26/31] block/crypto: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
com> Signed-off-by: Max Reitz --- block/crypto.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/crypto.c b/block/crypto.c index 3fbd68832b..2636e959ae 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -758,7 +758,6 @@ static int block_crypto_get_info_luks(BlockDriverState *bs,

[PULL 22/31] qemu-img: convert: don't use unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200528094405.145708-2-vsement...@virtuozzo.com> Signed-off-by: Max Reitz --- qemu-img.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 7f4938a5ef..53bd

[PULL 12/31] block/amend: separate amend and create options for qemu-img

2020-07-06 Thread Max Reitz
macro and then include it in each action option list. In future it might be useful to remove some options which are not supported anyway from amend list, which currently cause an error message if amended. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz

[PULL 24/31] block/vdi: return ZERO block-status when appropriate

2020-07-06 Thread Max Reitz
d_blocks_are_zero doesn't make sense (as the only user of it is bdrv_co_block_status and it checks unallocated_blocks_are_zero only for unallocated areas). Drop it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200528094405.145708-4-vsement...@virtuozzo.com>

[PULL 19/31] block/crypto: implement blockdev-amend

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-13-mlevi...@redhat.com> Signed-off-by: Max Reitz --- qapi/block-core.json | 14 - block/crypto.c

[PULL 16/31] block/qcow2: extend qemu-img amend interface with crypto options

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Now that we have all the infrastructure in place, wire it in the qcow2 driver and expose this to the user. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-9-mlevi...@redhat.com> Signed-off-b

[PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command

2020-07-06 Thread Max Reitz
Levitsky Reviewed-by: Daniel P. Berrangé Message-Id: <20200608094030.670121-12-mlevi...@redhat.com> Signed-off-by: Max Reitz --- qapi/block-core.json | 42 ++ qapi/job.json | 4 +- include/block/block_int.h | 21 +-- block/amend.c

[PULL 13/31] block/amend: refactor qcow2 amend options

2020-07-06 Thread Max Reitz
that became unnecessary thanks to "iotests: Make _filter_img_create more active"] Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-12-mre...@redhat.com> --- block/qcow2.c | 138 +--- qemu-img.c | 18 +

[PULL 08/31] iotests: Check whether luks works

2020-07-06 Thread Max Reitz
Whenever running an iotest for the luks format, we should check whether luks actually really works. Tests that try to create luks-encrypted qcow2 images should do the same. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-7-mre...@redhat.com> Reviewed-by: Maxim Levitsky ---

[PULL 07/31] iotests.py: Add (verify|has)_working_luks()

2020-07-06 Thread Max Reitz
Similar to _require_working_luks for bash tests, these functions can be used to check whether our luks driver can actually create images. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-6-mre...@redhat.com> Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/iotests.p

<    8   9   10   11   12   13   14   15   16   17   >