Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11] qcow2: Fix overly broad madvise()

2017-11-14 Thread Eric Blake
On 11/14/2017 12:41 PM, Max Reitz wrote: > @mem_size and @offset are both size_t, thus subtracting them from one > another will just return a big size_t if mem_size < offset -- even more > obvious here because the result is stored in another size_t. > > Checking that result to be positive is

Re: [Qemu-block] [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

2017-11-14 Thread Max Reitz
On 2017-11-14 21:38, John Snow wrote: > > > On 11/14/2017 03:35 PM, Max Reitz wrote: >> On 2017-11-14 21:30, John Snow wrote: >>> >>> >>> On 11/14/2017 01:46 PM, Max Reitz wrote: On 2017-11-14 19:45, Thomas Huth wrote: > On 14.11.2017 14:32, Max Reitz wrote: > [...] >> Well, do

Re: [Qemu-block] [Qemu-devel] [PATCH] qapi: block-core: Clarify events emitted by 'block-job-cancel'

2017-11-14 Thread no-reply
Hi, This series failed build test on s390x host. Please find the details below. Type: series Subject: [Qemu-devel] [PATCH] qapi: block-core: Clarify events emitted by 'block-job-cancel' Message-id: 20171114191605.22349-1-kcham...@redhat.com === TEST SCRIPT BEGIN === #!/bin/bash # Testing

Re: [Qemu-block] [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

2017-11-14 Thread Max Reitz
On 2017-11-14 21:30, John Snow wrote: > > > On 11/14/2017 01:46 PM, Max Reitz wrote: >> On 2017-11-14 19:45, Thomas Huth wrote: >>> On 14.11.2017 14:32, Max Reitz wrote: >>> [...] Well, do you want to document it? I'd rather deprecate it altogether. >>> >>> Maybe a first step could be to

Re: [Qemu-block] [Qemu-devel] [PATCH] qapi: block-core: Clarify events emitted by 'block-job-cancel'

2017-11-14 Thread no-reply
Hi, This series failed build test on ppc host. Please find the details below. Type: series Subject: [Qemu-devel] [PATCH] qapi: block-core: Clarify events emitted by 'block-job-cancel' Message-id: 20171114191605.22349-1-kcham...@redhat.com === TEST SCRIPT BEGIN === #!/bin/bash # Testing script

Re: [Qemu-block] [PATCH for-2.11] util/stats64: Fix min/max comparisons

2017-11-14 Thread Paolo Bonzini
- Max Reitz ha scritto: > stat64_min_slow() and stat64_max_slow() compare the wrong way. This > makes iotest 136 fail with clang and -m32. Queued, thanks. Cc: qemu-sta...@nongnu.org Paolo > Signed-off-by: Max Reitz > --- > util/stats64.c | 4 ++--

Re: [Qemu-block] [Qemu-devel] [PATCH v8 10/14] migration: add postcopy migration of dirty bitmaps

2017-11-14 Thread John Snow
On 10/30/2017 12:33 PM, Vladimir Sementsov-Ogievskiy wrote: > Postcopy migration of dirty bitmaps. Only named dirty bitmaps, > associated with root nodes and non-root named nodes are migrated. > > If destination qemu is already containing a dirty bitmap with the same name > as a migrated bitmap

[Qemu-block] [PATCH for-2.11] util/stats64: Fix min/max comparisons

2017-11-14 Thread Max Reitz
stat64_min_slow() and stat64_max_slow() compare the wrong way. This makes iotest 136 fail with clang and -m32. Signed-off-by: Max Reitz --- util/stats64.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/util/stats64.c b/util/stats64.c index

Re: [Qemu-block] [PATCH for-2.11? v7 0/6] block: Don't compare strings in bdrv_reopen_prepare()

2017-11-14 Thread Max Reitz
On 2017-11-14 19:01, Max Reitz wrote: > bdrv_reopen_prepare() assumes that all BDS options are strings, which is > not necessarily correct. This series introduces a new qobject_is_equal() > function which can be used to test whether any options have changed, > independently of their type. Aaand

Re: [Qemu-block] [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

2017-11-14 Thread John Snow
On 11/14/2017 03:35 PM, Max Reitz wrote: > On 2017-11-14 21:30, John Snow wrote: >> >> >> On 11/14/2017 01:46 PM, Max Reitz wrote: >>> On 2017-11-14 19:45, Thomas Huth wrote: On 14.11.2017 14:32, Max Reitz wrote: [...] > Well, do you want to document it? I'd rather deprecate it

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 1/3] qapi: Add qdict_is_null()

2017-11-14 Thread Markus Armbruster
Max Reitz writes: > On 2017-11-14 15:57, Markus Armbruster wrote: >> Max Reitz writes: >> >>> Signed-off-by: Max Reitz >>> --- >>> include/qapi/qmp/qdict.h | 1 + >>> qobject/qdict.c | 10 ++ >>> 2 files changed, 11

Re: [Qemu-block] [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

2017-11-14 Thread Max Reitz
On 2017-11-13 19:08, Eric Blake wrote: > On 11/13/2017 11:58 AM, Eric Blake wrote: > qemu-system-aarch64: -drive if=none,file=hda.qcow2,format=qcow,id=hd: Unsupported qcow version 3 >>> >>> ah, this means it wants "format=qcow2". >> >> Oh, I should have read this followup before writing

[Qemu-block] [PATCH 0/5] compressed block-stream

2017-11-14 Thread Anton Nefedov
It might be useful to compress images during block-stream; this way the user can merge compressed images of a backing chain and the result will remain compressed. Anton Nefedov (4): qcow2: reject unaligned offsets in write compressed block: support compressed write for copy-on-read

[Qemu-block] [PATCH 1/5] qcow2: reject unaligned offsets in write compressed

2017-11-14 Thread Anton Nefedov
Misaligned compressed write is not supported. Signed-off-by: Anton Nefedov --- block/qcow2.c | 4 1 file changed, 4 insertions(+) diff --git a/block/qcow2.c b/block/qcow2.c index 92cb9f9..45c5651 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3349,6

[Qemu-block] [PATCH 5/5] iotest 030: add compressed block-stream test

2017-11-14 Thread Anton Nefedov
Signed-off-by: Anton Nefedov --- tests/qemu-iotests/030 | 69 +- tests/qemu-iotests/030.out | 4 +-- 2 files changed, 70 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index

Re: [Qemu-block] [PATCH v5 02/29] Replace all occurances of __FUNCTION__ with __func__

2017-11-14 Thread Gerd Hoffmann
On Mon, Nov 13, 2017 at 02:34:42PM -0800, Alistair Francis wrote: > Replace all occurs of __FUNCTION__ except for the check in checkpatch > with the non GCC specific __func__. > > One line in hcd-musb.c was manually tweaked to pass checkpatch. > > Signed-off-by: Alistair Francis

[Qemu-block] [PATCH 3/5] block: support compressed write for copy-on-read

2017-11-14 Thread Anton Nefedov
Signed-off-by: Anton Nefedov --- block/io.c | 30 -- block/trace-events | 2 +- 2 files changed, 25 insertions(+), 7 deletions(-) diff --git a/block/io.c b/block/io.c index 3d5ef2c..93c6b24 100644 --- a/block/io.c +++

[Qemu-block] [PATCH 4/5] block-stream: add compress option

2017-11-14 Thread Anton Nefedov
Signed-off-by: Anton Nefedov --- qapi/block-core.json | 4 include/block/block_int.h | 4 +++- block/stream.c| 16 blockdev.c| 13 - hmp.c | 2 ++ hmp-commands.hx | 4

[Qemu-block] [PATCH 2/5] qcow2: multiple clusters write compressed

2017-11-14 Thread Anton Nefedov
From: Pavel Butsykin At the moment, qcow2_co_pwritev_compressed can process the requests size less than or equal to one cluster. This patch added possibility to write compressed data in the QCOW2 more than one cluster. The implementation is simple, we just split large

Re: [Qemu-block] [PATCH for-2.11 2/5] qcow2: Unaligned zero cluster in handle_alloc()

2017-11-14 Thread Alberto Garcia
On Fri 10 Nov 2017 09:31:08 PM CET, Max Reitz wrote: > We should check whether the cluster offset we are about to use is > actually valid; that is, whether it is aligned to cluster boundaries. > > Reported-by: R. Nageswara Sastry > Buglink:

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 1/3] qapi: Add qdict_is_null()

2017-11-14 Thread Max Reitz
On 2017-11-14 15:57, Markus Armbruster wrote: > Max Reitz writes: > >> Signed-off-by: Max Reitz >> --- >> include/qapi/qmp/qdict.h | 1 + >> qobject/qdict.c | 10 ++ >> 2 files changed, 11 insertions(+) >> >> diff --git

Re: [Qemu-block] [PATCH for-2.11 5/5] qcow2: Refuse to get unaligned offsets from cache

2017-11-14 Thread Max Reitz
On 2017-11-14 16:06, Alberto Garcia wrote: > On Fri 10 Nov 2017 09:31:11 PM CET, Max Reitz wrote: >> +static inline const char *qcow2_cache_get_name(BDRVQcow2State *s, >> Qcow2Cache *c) >> +{ >> +if (c == s->refcount_block_cache) { >> +return "refcount block"; >> +} else if (c ==

Re: [Qemu-block] [PATCH v6 0/6] block: Don't compare strings in bdrv_reopen_prepare()

2017-11-14 Thread Max Reitz
On 2017-10-04 17:25, Max Reitz wrote: > bdrv_reopen_prepare() assumes that all BDS options are strings, which is > not necessarily correct. This series introduces a new qobject_is_equal() > function which can be used to test whether any options have changed, > independently of their type. > > >

Re: [Qemu-block] [Nbd] [Qemu-devel] How to online resize qemu disk with nbd protocol?

2017-11-14 Thread Eric Blake
[reviving an old thread] On 01/23/2017 08:54 AM, Eric Blake wrote: > I'm still thinking that allowing the client to query the current size is > useful. Over the weekend, I was thinking of SEEK_SET/SEEK_END semantics > (SEEK_CUR doesn't really make sense, since we don't maintain a current >

Re: [Qemu-block] [PATCH] iotests: Add test for failing qemu-img commit

2017-11-14 Thread Max Reitz
On 2017-11-10 22:21, Max Reitz wrote: > On 2017-06-16 15:58, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >> In order to pass, this depends on "fix: avoid an infinite loop or a >> dangling pointer problem in img_commit" >>

Re: [Qemu-block] [PATCH for-2.12 2/3] block: Handle null backing link

2017-11-14 Thread Alberto Garcia
On Fri 10 Nov 2017 11:13:28 PM CET, Max Reitz wrote: > Instead of converting all "backing": null instances into "backing": "", > handle a null value directly in bdrv_open_inherit(). > > This enables explicitly null backing links for json:{} filenames. > > Signed-off-by: Max Reitz

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-14 Thread Max Reitz
On 2017-10-23 11:29, Vladimir Sementsov-Ogievskiy wrote: > Snapshot-switch actually changes active state of disk so it should > reflect on dirty bitmaps. Otherwise next incremental backup using > these bitmaps will be invalid. > > Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 1/3] qapi: Add qdict_is_null()

2017-11-14 Thread Markus Armbruster
Max Reitz writes: > Signed-off-by: Max Reitz > --- > include/qapi/qmp/qdict.h | 1 + > qobject/qdict.c | 10 ++ > 2 files changed, 11 insertions(+) > > diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h > index

Re: [Qemu-block] [Qemu-devel] [PATCH] qcow2: fix image corruption after committing qcow2 image into base

2017-11-14 Thread Max Reitz
On 2017-11-10 18:22, Daniel P. Berrange wrote: > On Fri, Nov 10, 2017 at 10:34:59AM -0600, Eric Blake wrote: >> On 11/03/2017 09:41 AM, Daniel P. Berrange wrote: >>> After committing the qcow2 image contents into the base image, qemu-img >>> will call bdrv_make_empty to drop the payload in the

Re: [Qemu-block] [PATCH for-2.11 4/5] qcow2: Add bounds check to get_refblock_offset()

2017-11-14 Thread Alberto Garcia
On Fri 10 Nov 2017 09:31:10 PM CET, Max Reitz wrote: > +static int64_t get_refblock_offset(BlockDriverState *bs, uint64_t offset) > +{ > +BDRVQcow2State *s = bs->opaque; > +uint32_t index = offset_to_reftable_index(s, offset); > +int64_t covering_refblock_offset = 0; > + > +if

Re: [Qemu-block] [PATCH] iotests: 077: Filter out 'resume' lines

2017-11-14 Thread Max Reitz
On 2017-11-13 16:00, Fam Zheng wrote: > In the "Overlapping multiple requests" cases, the 3rd reqs (the break > point B) doesn't wait for the 2nd, and once resumed the I/O will just > continue. This is because the 2nd is already waiting for the 1st, and > in wait_serialising_requests() there is:

Re: [Qemu-block] [PATCH for-2.12 1/3] qapi: Add qdict_is_null()

2017-11-14 Thread Max Reitz
On 2017-11-14 15:07, Alberto Garcia wrote: > On Fri 10 Nov 2017 11:13:27 PM CET, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >> include/qapi/qmp/qdict.h | 1 + >> qobject/qdict.c | 10 ++ >> 2 files changed, 11 insertions(+) >> >> diff --git

Re: [Qemu-block] [PATCH for-2.11 5/5] qcow2: Refuse to get unaligned offsets from cache

2017-11-14 Thread Alberto Garcia
On Fri 10 Nov 2017 09:31:11 PM CET, Max Reitz wrote: > +static inline const char *qcow2_cache_get_name(BDRVQcow2State *s, Qcow2Cache > *c) > +{ > +if (c == s->refcount_block_cache) { > +return "refcount block"; > +} else if (c == s->l2_table_cache) { > +return "L2 table";

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 2/3] block: Handle null backing link

2017-11-14 Thread Markus Armbruster
Max Reitz writes: > Instead of converting all "backing": null instances into "backing": "", > handle a null value directly in bdrv_open_inherit(). > > This enables explicitly null backing links for json:{} filenames. > > Signed-off-by: Max Reitz > --- >

Re: [Qemu-block] [PATCH for-2.11 4/5] qcow2: Add bounds check to get_refblock_offset()

2017-11-14 Thread Max Reitz
On 2017-11-14 16:02, Alberto Garcia wrote: > On Fri 10 Nov 2017 09:31:10 PM CET, Max Reitz wrote: >> +static int64_t get_refblock_offset(BlockDriverState *bs, uint64_t offset) >> +{ >> +BDRVQcow2State *s = bs->opaque; >> +uint32_t index = offset_to_reftable_index(s, offset); >> +

Re: [Qemu-block] [PATCH for-2.11 5/5] qcow2: Refuse to get unaligned offsets from cache

2017-11-14 Thread Alberto Garcia
On Tue 14 Nov 2017 04:09:16 PM CET, Max Reitz wrote: >>> +static inline const char *qcow2_cache_get_name(BDRVQcow2State *s, >>> Qcow2Cache *c) >>> +{ >>> +if (c == s->refcount_block_cache) { >>> +return "refcount block"; >>> +} else if (c == s->l2_table_cache) { >>> +

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11 3/5] block: Guard against NULL bs->drv

2017-11-14 Thread Max Reitz
On 2017-11-10 22:46, Eric Blake wrote: > On 11/10/2017 02:31 PM, Max Reitz wrote: >> We currently do not guard everywhere against a NULL bs->drv where we >> should be doing so. Most of the places fixed here just do not care >> about that case at all. >> >> Some care implicitly, e.g. through a

Re: [Qemu-block] [PATCH for-2.11 4/5] qcow2: Add bounds check to get_refblock_offset()

2017-11-14 Thread Max Reitz
On 2017-11-14 16:38, Alberto Garcia wrote: > On Tue 14 Nov 2017 04:27:56 PM CET, Max Reitz wrote: +static int64_t get_refblock_offset(BlockDriverState *bs, uint64_t offset) +{ +BDRVQcow2State *s = bs->opaque; +uint32_t index = offset_to_reftable_index(s, offset); +

Re: [Qemu-block] [PATCH for-2.12 1/3] qapi: Add qdict_is_null()

2017-11-14 Thread Alberto Garcia
On Fri 10 Nov 2017 11:13:27 PM CET, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > include/qapi/qmp/qdict.h | 1 + > qobject/qdict.c | 10 ++ > 2 files changed, 11 insertions(+) > > diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h > index

Re: [Qemu-block] [PATCH for 2.9] qcow2: avoid nb_sectors int overflow at zero/discard

2017-11-14 Thread Max Reitz
On 2017-10-24 10:56, Anton Nefedov wrote: > during discard, signed integer overflow leads to end_offset possibly > be less than offset, > that in turn makes nb_clusters (to discard) much bigger than expected, > leading to a possible data loss, operation taking very long and > image growing large

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 3/3] block: Deprecate "backing": ""

2017-11-14 Thread Markus Armbruster
Max Reitz writes: > We have a clear replacement, so let's deprecate it. > > Signed-off-by: Max Reitz > --- > qapi/block-core.json | 4 ++-- > block.c | 4 > qemu-doc.texi| 7 +++ > qemu-options.hx | 4 ++-- > 4 files

Re: [Qemu-block] [PATCH v4 1/4] block/vhdx.c: Don't blindly update the header

2017-11-14 Thread Max Reitz
On 2017-11-07 14:10, Jeff Cody wrote: > The VHDX specification requires that before user data modification of > the vhdx image, the VHDX header file and data GUIDs need to be updated. > In vhdx_open(), if the image is set to RDWR, we go ahead and update the > header. > > However, just because the

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 2/3] block: Handle null backing link

2017-11-14 Thread Max Reitz
On 2017-11-14 16:17, Markus Armbruster wrote: > Max Reitz writes: > >> Instead of converting all "backing": null instances into "backing": "", >> handle a null value directly in bdrv_open_inherit(). >> >> This enables explicitly null backing links for json:{} filenames. >> >>

Re: [Qemu-block] [PATCH for-2.12 0/4] blockdev: Mark BD-{remove, insert}-medium stable

2017-11-14 Thread Alberto Garcia
On Fri 10 Nov 2017 11:42:58 PM CET, Max Reitz wrote: > Berto's "Test I/O limits with removable media" patch proves that > throttling survives a blockdev-remove-medium/blockdev-insert-medium pair > now, so let's mark them stable (because that was the reason they were > considered experimental, see

Re: [Qemu-block] [PATCH v4 0/4] Don't write headers if BDS is INACTIVE

2017-11-14 Thread Max Reitz
On 2017-11-07 14:10, Jeff Cody wrote: > Changes from v3->v4: > > Patch 3: Add migrate_del_blocker and error_free (Thanks Stefan) Thanks, applied to my block branch: https://github.com/XanClic/qemu/commits/block Max signature.asc Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH for-2.11? v7 0/6] block: Don't compare strings in bdrv_reopen_prepare()

2017-11-14 Thread Eric Blake
On 11/14/2017 12:01 PM, Max Reitz wrote: > bdrv_reopen_prepare() assumes that all BDS options are strings, which is > not necessarily correct. This series introduces a new qobject_is_equal() > function which can be used to test whether any options have changed, > independently of their type. > >

Re: [Qemu-block] [PATCH for-2.11? v7 6/6] tests: Add check-qobject for equality tests

2017-11-14 Thread Eric Blake
On 11/14/2017 12:01 PM, Max Reitz wrote: > Add a new test file (check-qobject.c) for unit tests that concern > QObjects as a whole. > > Its only purpose for now is to test the qobject_is_equal() function. > > + * Note that qobject_is_equal() is not really an equivalence relation, > + * so this

Re: [Qemu-block] [PATCH 1/5 for-2.11?] qcow2: reject unaligned offsets in write compressed

2017-11-14 Thread Eric Blake
On 11/14/2017 12:30 PM, Anton Nefedov wrote: > On 14/11/2017 7:50 PM, Eric Blake wrote: >> On 11/14/2017 04:16 AM, Anton Nefedov wrote: >>> Misaligned compressed write is not supported. >>> >>> Signed-off-by: Anton Nefedov >>> --- >>>   block/qcow2.c | 4 >>>   1

Re: [Qemu-block] [Qemu-devel] [PATCH] qapi: block-core: Clarify events emitted by 'block-job-cancel'

2017-11-14 Thread no-reply
Hi, This series failed automatic build test. Please find the testing commands and their output below. If you have docker installed, you can probably reproduce it locally. Subject: [Qemu-devel] [PATCH] qapi: block-core: Clarify events emitted by 'block-job-cancel' Type: series Message-id:

[Qemu-block] [PULL 09/20] iotests: Add missing 'blkdebug::' in 040

2017-11-14 Thread Max Reitz
040 tries to invoke pause_drive() on a drive that does not use blkdebug. Good idea, but let's use blkdebug to make it actually work. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-id:

Re: [Qemu-block] [Nbd] [Qemu-devel] How to online resize qemu disk with nbd protocol?

2017-11-14 Thread Wouter Verhelst
On Tue, Nov 14, 2017 at 10:41:39AM -0600, Eric Blake wrote: > Another thought - with structured replies, we finally have a way to let > the client ask for the server to send resize information whenever the > server wants, rather than having to be polled by a new client request > all the time.

[Qemu-block] [PULL 06/20] qcow2: Add iotest for an empty refcount table

2017-11-14 Thread Max Reitz
From: Alberto Garcia This patch adds a simple iotest in which we try to write to an image with an empty refcount table (i.e. with all entries set to 0). This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness.

[Qemu-block] [PULL 08/20] iotests: Make 030 less flaky

2017-11-14 Thread Max Reitz
This patch fixes two race conditions in 030: 1. The first is in TestENOSPC.test_enospc(). After resuming the job, querying it to confirm it is no longer paused may fail because in the meantime it might have completed already. The same was fixed in TestEIO.test_ignore() already (in

[Qemu-block] [PULL 16/20] iotests: 077: Filter out 'resume' lines

2017-11-14 Thread Max Reitz
From: Fam Zheng In the "Overlapping multiple requests" cases, the 3rd reqs (the break point B) doesn't wait for the 2nd, and once resumed the I/O will just continue. This is because the 2nd is already waiting for the 1st, and in wait_serialising_requests() there is: /* If

[Qemu-block] [PULL 10/20] iotests: Make 055 less flaky

2017-11-14 Thread Max Reitz
First of all, test 055 does a valiant job of invoking pause_drive() sometimes, but that is worth nothing without blkdebug. So the first thing to do is to sprinkle a couple of "blkdebug::" in there -- with the exception of the transaction tests, because the blkdebug break points make the

[Qemu-block] [PULL 15/20] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-14 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy Snapshot-switch actually changes active state of disk so it should reflect on dirty bitmaps. Otherwise next incremental backup using these bitmaps will be invalid. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PULL 18/20] block/parallels: Do not update header or truncate image when INMIGRATE

2017-11-14 Thread Max Reitz
From: Jeff Cody If we write or modify the image file while the QEMU run state is INMIGRATE, then the BDRV_O_INACTIVE BDS flag is set. This will cause an assert, since the image is marked inactive. Make sure we obey this flag. Tested-by: Alexey Kardashevskiy

[Qemu-block] [PULL 13/20] iotests: Use new-style NBD connections

2017-11-14 Thread Max Reitz
From: Eric Blake Old-style NBD is deprecated upstream (it is documented, but no longer implemented in the reference implementation), and it is severely limited (it cannot support structured replies, which means it cannot support efficient handling of zeroes), when compared to

[Qemu-block] [PULL 17/20] block/vhdx.c: Don't blindly update the header

2017-11-14 Thread Max Reitz
From: Jeff Cody The VHDX specification requires that before user data modification of the vhdx image, the VHDX header file and data GUIDs need to be updated. In vhdx_open(), if the image is set to RDWR, we go ahead and update the header. However, just because the image is set

[Qemu-block] [PULL 20/20] qemu-iotests: update unsupported image formats in 194

2017-11-14 Thread Max Reitz
From: Jeff Cody Test 194 checks for 'luks' to exclude as an unsupported format, However, most formats are unsupported, due to migration blockers. Rather than specifying a blacklist of unsupported formats, whitelist supported formats (specifically, qcow2, qed, raw, dmg).

[Qemu-block] [PULL 11/20] iotests: Make 083 less flaky

2017-11-14 Thread Max Reitz
083 has (at least) two issues: 1. By launching the nbd-fault-injector in background, it may not be scheduled until the first grep on its output file is executed. However, until then, that file may not have been created yet -- so it either does not exist yet (thus making the grep emit an

[Qemu-block] [PULL 12/20] iotests: Make 136 less flaky

2017-11-14 Thread Max Reitz
136 executes some AIO requests without a final aio_flush; then it advances the virtual clock and thus expects the last access time of the device to be less than the current time when queried (i.e. idle_time_ns to be greater than 0). However, without the aio_flush, some requests may be settled

[Qemu-block] [PULL 19/20] block/parallels: add migration blocker

2017-11-14 Thread Max Reitz
From: Jeff Cody Migration does not work for parallels, and has been broken for a while (see patch 'block/parallels: Do not update header or truncate image when INMIGRATE'). The bdrv_invalidate_cache() method needs to be added for migration to be supported. Until this is

Re: [Qemu-block] [PATCH 1/5 for-2.11?] qcow2: reject unaligned offsets in write compressed

2017-11-14 Thread Eric Blake
On 11/14/2017 04:16 AM, Anton Nefedov wrote: > Misaligned compressed write is not supported. > > Signed-off-by: Anton Nefedov > --- > block/qcow2.c | 4 > 1 file changed, 4 insertions(+) Should this one be applied in 2.11? Reviewed-by: Eric Blake

[Qemu-block] [PULL 02/20] qcow2: Prevent allocating L2 tables at offset 0

2017-11-14 Thread Max Reitz
From: Alberto Garcia If the refcount data is corrupted then we can end up trying to allocate a new L2 table at offset 0 in the image, triggering an assertion in the qcow2 cache that would crash QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset != 0' failed

[Qemu-block] [PULL 05/20] qcow2: Add iotest for an image with header.refcount_table_offset == 0

2017-11-14 Thread Max Reitz
From: Alberto Garcia This patch adds a simple iotest in which we try to write to an image with the refcount table offset set to 0. This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto

[Qemu-block] [PULL 03/20] qcow2: Prevent allocating compressed clusters at offset 0

2017-11-14 Thread Max Reitz
From: Alberto Garcia If the refcount data is corrupted then we can end up trying to allocate a new compressed cluster at offset 0 in the image, triggering an assertion in qcow2_alloc_bytes() that would crash QEMU: qcow2_alloc_bytes: Assertion `offset' failed. This patch

[Qemu-block] [PULL 01/20] qcow2: Prevent allocating refcount blocks at offset 0

2017-11-14 Thread Max Reitz
From: Alberto Garcia Each entry in the qcow2 cache contains an offset field indicating the location of the data in the qcow2 image. If the offset is 0 then it means that the entry contains no data and is available to be used when needed. Because of that it is not possible to

[Qemu-block] [PULL 00/20] Block patches for 2.11.0-rc1

2017-11-14 Thread Max Reitz
The following changes since commit 191b5fbfa66e5b23e2150f3c6981d30eb84418a9: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2017-11-14 16:11:19 +) are available in the git repository at: git://github.com/XanClic/qemu.git tags/pull-block-2017-11-14

[Qemu-block] [PULL 04/20] qcow2: Don't open images with header.refcount_table_clusters == 0

2017-11-14 Thread Max Reitz
From: Alberto Garcia qcow2_do_open() is checking that header.refcount_table_clusters is not too large, but it doesn't check that it's greater than zero. Apart from the fact that an image like that is obviously corrupted, trying to use it crashes QEMU since we end up with a null

[Qemu-block] [PULL 07/20] qcow2: Assert that the crypto header does not overlap other metadata

2017-11-14 Thread Max Reitz
From: Alberto Garcia The crypto header is initialized only when QEMU is creating a new image, so there's no chance of this happening on a corrupted image. If QEMU is really trying to allocate the header overlapping other existing metadata sections then this is a serious bug in

Re: [Qemu-block] [PULL 00/20] Block patches for 2.11.0-rc1

2017-11-14 Thread Peter Maydell
On 14 November 2017 at 17:23, Max Reitz wrote: > The following changes since commit 191b5fbfa66e5b23e2150f3c6981d30eb84418a9: > > Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' > into staging (2017-11-14 16:11:19 +) > > are available in the git

Re: [Qemu-block] [PULL 00/20] Block patches for 2.11.0-rc1

2017-11-14 Thread Max Reitz
On 2017-11-14 18:28, Peter Maydell wrote: > On 14 November 2017 at 17:23, Max Reitz wrote: >> The following changes since commit 191b5fbfa66e5b23e2150f3c6981d30eb84418a9: >> >> Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' >> into staging

[Qemu-block] [PATCH for-2.11? v7 0/6] block: Don't compare strings in bdrv_reopen_prepare()

2017-11-14 Thread Max Reitz
bdrv_reopen_prepare() assumes that all BDS options are strings, which is not necessarily correct. This series introduces a new qobject_is_equal() function which can be used to test whether any options have changed, independently of their type. v7: - Patch 6: Fix a clang warning:

[Qemu-block] [PATCH for-2.11? v7 1/6] qapi/qnull: Add own header

2017-11-14 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Reviewed-by: Markus Armbruster --- include/qapi/qmp/qdict.h| 1 + include/qapi/qmp/qnull.h| 30 ++

[Qemu-block] [PATCH for-2.11? v7 2/6] qapi/qlist: Add qlist_append_null() macro

2017-11-14 Thread Max Reitz
Besides the macro itself, this patch also adds a corresponding Coccinelle rule. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- include/qapi/qmp/qlist.h | 3 +++ scripts/coccinelle/qobject.cocci |

[Qemu-block] [PATCH for-2.11? v7 5/6] iotests: Add test for non-string option reopening

2017-11-14 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/qemu-iotests/133 | 9 + tests/qemu-iotests/133.out | 5 + 2 files changed, 14 insertions(+) diff --git a/tests/qemu-iotests/133

[Qemu-block] [PATCH for-2.11? v7 3/6] qapi: Add qobject_is_equal()

2017-11-14 Thread Max Reitz
This generic function (along with its implementations for different types) determines whether two QObjects are equal. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Reviewed-by: Markus Armbruster

[Qemu-block] [PATCH for-2.11? v7 4/6] block: qobject_is_equal() in bdrv_reopen_prepare()

2017-11-14 Thread Max Reitz
Currently, bdrv_reopen_prepare() assumes that all BDS options are strings. However, this is not the case if the BDS has been created through the json: pseudo-protocol or blockdev-add. Note that the user-invokable reopen command is an HMP command, so you can only specify strings there. Therefore,

[Qemu-block] [PATCH for-2.11? v7 6/6] tests: Add check-qobject for equality tests

2017-11-14 Thread Max Reitz
Add a new test file (check-qobject.c) for unit tests that concern QObjects as a whole. Its only purpose for now is to test the qobject_is_equal() function. Signed-off-by: Max Reitz --- tests/Makefile.include | 4 +- tests/check-qobject.c | 328

Re: [Qemu-block] [PULL 00/20] Block patches for 2.11.0-rc1

2017-11-14 Thread Peter Maydell
On 14 November 2017 at 17:23, Max Reitz wrote: > The following changes since commit 191b5fbfa66e5b23e2150f3c6981d30eb84418a9: > > Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' > into staging (2017-11-14 16:11:19 +) > > are available in the git

Re: [Qemu-block] [PATCH 1/5 for-2.11?] qcow2: reject unaligned offsets in write compressed

2017-11-14 Thread Anton Nefedov
On 14/11/2017 7:50 PM, Eric Blake wrote: On 11/14/2017 04:16 AM, Anton Nefedov wrote: Misaligned compressed write is not supported. Signed-off-by: Anton Nefedov --- block/qcow2.c | 4 1 file changed, 4 insertions(+) Should this one be applied in 2.11?

Re: [Qemu-block] [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

2017-11-14 Thread Thomas Huth
On 14.11.2017 14:32, Max Reitz wrote: [...] > Well, do you want to document it? I'd rather deprecate it altogether. Maybe a first step could be to change qemu-img so that it refuses to create new qcow1 images (but still can convert them into other formats). So basically make qcow1 read-only?

Re: [Qemu-block] [Nbd] [Qemu-devel] How to online resize qemu disk with nbd protocol?

2017-11-14 Thread Eric Blake
On 11/14/2017 11:37 AM, Wouter Verhelst wrote: > On Tue, Nov 14, 2017 at 10:41:39AM -0600, Eric Blake wrote: >> Another thought - with structured replies, we finally have a way to let >> the client ask for the server to send resize information whenever the >> server wants, rather than having to be

[Qemu-block] [PATCH for-2.11] qcow2: Fix overly broad madvise()

2017-11-14 Thread Max Reitz
@mem_size and @offset are both size_t, thus subtracting them from one another will just return a big size_t if mem_size < offset -- even more obvious here because the result is stored in another size_t. Checking that result to be positive is therefore not sufficient to excluse the case that

Re: [Qemu-block] [Qemu-devel] using "qemu-img convert -O qcow2" to convert qcow v1 to v2 creates a qcow v3 file?

2017-11-14 Thread Max Reitz
On 2017-11-14 19:45, Thomas Huth wrote: > On 14.11.2017 14:32, Max Reitz wrote: > [...] >> Well, do you want to document it? I'd rather deprecate it altogether. > > Maybe a first step could be to change qemu-img so that it refuses to > create new qcow1 images (but still can convert them into