[Qemu-block] [PATCH] iotests: Check for enabled drivers before testing them

2019-08-19 Thread Thomas Huth
It is possible to enable only a subset of the block drivers with the "--block-drv-rw-whitelist" option of the "configure" script. All other drivers are marked as unusable (or only included as read-only with the "--block-drv-ro-whitelist" option). If an iotest is now using such a disabled block driv

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: Check for enabled drivers before testing them

2019-08-19 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190819075348.4078-1-th...@redhat.com/ Hi, This series failed build test on s390x host. Please find the details below. === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked under the git checkout with # HEAD pointing to a commit that has

[Qemu-block] [PATCH] iotests: Add more "skip_if_unsupported" statements to the python tests

2019-08-19 Thread Thomas Huth
The python code already contains a possibility to skip tests if the corresponding driver is not available in the qemu binary - use it in more spots to avoid that the tests are failing if the driver has been disabled. Signed-off-by: Thomas Huth --- tests/qemu-iotests/030 | 3 +++ tests/qemu-iote

Re: [Qemu-block] [PULL 00/36] Bitmaps patches

2019-08-19 Thread Peter Maydell
On Sat, 17 Aug 2019 at 00:13, John Snow wrote: > > The following changes since commit afd760539308a5524accf964107cdb1d54a059e3: > > Merge remote-tracking branch > 'remotes/pmaydell/tags/pull-target-arm-20190816' into staging (2019-08-16 > 17:21:40 +0100) > > are available in the Git repository

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-19 Thread Maxim Levitsky
On Thu, 2019-08-15 at 10:00 -0500, Eric Blake wrote: > On 8/15/19 9:44 AM, Maxim Levitsky wrote: > > > > > > Does the idea of a union type with a default value for the > > > > > discriminator > > > > > help? Maybe we have a discriminator which defaults to 'auto', and > > > > > add a > > > > > u

Re: [Qemu-block] [PULL 0/3] Run iotests during "make check"

2019-08-19 Thread Peter Maydell
On Sat, 17 Aug 2019 at 09:54, Thomas Huth wrote: > > Hi Peter, > > the following changes since commit afd760539308a5524accf964107cdb1d54a059e3: > > Merge remote-tracking branch > 'remotes/pmaydell/tags/pull-target-arm-20190816' into staging (2019-08-16 > 17:21:40 +0100) > > are available in t

Re: [Qemu-block] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-19 Thread Maxim Levitsky
On Thu, 2019-08-15 at 17:40 -0400, John Snow wrote: > > On 8/14/19 4:22 PM, Maxim Levitsky wrote: > > This is also a preparation for key read/write/erase functions > > > > This is a matter of taste and I am not usually reviewing LUKS patches > (So don't take me too seriously), but I would prefer

[Qemu-block] [PATCH v3 1/3] qcow2: introduce compression type feature

2019-08-19 Thread Denis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to QCOW2 header that indicates that *all* compressed clusters must be (de)compressed using a certain compression type. It is implied that the compression type is set on the image creation and can be changed only later

[Qemu-block] [PATCH v3 0/3] qcow2: add zstd cluster compression

2019-08-19 Thread Denis Plotnikov
v3: * relax the compression type setting requirement when the compression type is not zlib [Eric, Kevin] * add compression type values to the spec [Eric] * fix wording in the spec and descriptions [Eric] * fix functions descriptions [Max] * fix zstd (de)compression functions flaws [Max] * fix zst

[Qemu-block] [PATCH v3 2/3] qcow2: rework the cluster compression routine

2019-08-19 Thread Denis Plotnikov
The patch allow to process image compression type defined in the image header and choose an appropriate method for image clusters (de)compression. Signed-off-by: Denis Plotnikov --- block/qcow2-threads.c | 78 +++ 1 file changed, 64 insertions(+), 14 delet

[Qemu-block] [PATCH v3 3/3] qcow2: add zstd cluster compression

2019-08-19 Thread Denis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of compression ratio in comparison with zlib, which, at the moment, has been the only compression method available. The performance test results: Test compresses and decompres

[Qemu-block] [PULL 01/17] LUKS: support preallocation

2019-08-19 Thread Max Reitz
From: Maxim Levitsky preallocation=off and preallocation=metadata both allocate luks header only, and preallocation=falloc/full is passed to underlying file. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951 Signed-off-by: Maxim Levitsky Message-id: 20190716161901.1430-1-mlevi...@redh

[Qemu-block] [PULL 00/17] Block patches

2019-08-19 Thread Max Reitz
The following changes since commit 3fbd3405d2b0604ea530fc7a1828f19da1e95ff9: Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-08-17' into staging (2019-08-19 14:14:09 +0100) are available in the Git repository at: https://github.com/XanClic/qemu.git tags/pull-block-

[Qemu-block] [PULL 04/17] block: Add bdrv_has_zero_init_truncate()

2019-08-19 Thread Max Reitz
No .bdrv_has_zero_init() implementation returns 1 if growing the file would add non-zero areas (at least with PREALLOC_MODE_OFF), so using it in lieu of this new function was always safe. But on the other hand, it is possible that growing an image that is not zero-initialized would still add a zer

[Qemu-block] [PULL 02/17] qemu-img: Fix bdrv_has_zero_init() use in convert

2019-08-19 Thread Max Reitz
bdrv_has_zero_init() only has meaning for newly created images or image areas. If qemu-img convert did not create the image itself, it cannot rely on bdrv_has_zero_init()'s result to carry any meaning. Signed-off-by: Max Reitz Message-id: 20190724171239.8764-2-mre...@redhat.com Reviewed-by: Maxi

[Qemu-block] [PULL 05/17] block: Implement .bdrv_has_zero_init_truncate()

2019-08-19 Thread Max Reitz
We need to implement .bdrv_has_zero_init_truncate() for every block driver that supports truncation and has a .bdrv_has_zero_init() implementation. Implement it the same way each driver implements .bdrv_has_zero_init(). This is at least not any more unsafe than what we had before. Signed-off-by:

[Qemu-block] [PULL 06/17] block: Use bdrv_has_zero_init_truncate()

2019-08-19 Thread Max Reitz
vhdx and parallels call bdrv_has_zero_init() when they do not really care about an image's post-create state but only about what happens when you grow an image. That is a bit ugly, and also overly safe when growing preallocated images without preallocating the new areas. Let them use bdrv_has_zer

[Qemu-block] [PULL 07/17] qcow2: Fix .bdrv_has_zero_init()

2019-08-19 Thread Max Reitz
If a qcow2 file is preallocated, it can no longer guarantee that it initially appears as filled with zeroes. So implement .bdrv_has_zero_init() by checking whether the file is preallocated; if so, forward the call to the underlying storage node, except for when it is encrypted: Encrypted prealloca

[Qemu-block] [PULL 03/17] mirror: Fix bdrv_has_zero_init() use

2019-08-19 Thread Max Reitz
bdrv_has_zero_init() only has meaning for newly created images or image areas. If the mirror job itself did not create the image, it cannot rely on bdrv_has_zero_init()'s result to carry any meaning. This is the case for drive-mirror with mode=existing and always for blockdev-mirror. Note that w

[Qemu-block] [PULL 12/17] iotests: Full mirror to existing non-zero image

2019-08-19 Thread Max Reitz
The result of a sync=full mirror should always be the equal to the input. Therefore, existing images should be treated as potentially non-zero and thus should be explicitly initialized to be zero beforehand. Signed-off-by: Max Reitz Message-id: 20190724171239.8764-12-mre...@redhat.com Signed-off

[Qemu-block] [PULL 16/17] iotests: Fix 141 when run with qed

2019-08-19 Thread Max Reitz
69f47505ee has changed qcow2 in such a way that the commit job run in test 141 (and 144[1]) returns before it emits the READY event. However, 141 also runs with qed, where the order is still the other way around. Just filter out the {"return": {}} so the test passes for qed again. [1] 144 only ru

[Qemu-block] [PULL 08/17] vdi: Fix .bdrv_has_zero_init()

2019-08-19 Thread Max Reitz
Static VDI images cannot guarantee to be zero-initialized. If the image has been statically allocated, forward the call to the underlying storage node. Reported-by: Stefano Garzarella Signed-off-by: Max Reitz Reviewed-by: Stefan Weil Acked-by: Stefano Garzarella Tested-by: Stefano Garzarella

[Qemu-block] [PULL 09/17] vhdx: Fix .bdrv_has_zero_init()

2019-08-19 Thread Max Reitz
Fixed VHDX images cannot guarantee to be zero-initialized. If the image has the "fixed" subformat, forward the call to the underlying storage node. Reported-by: Stefano Garzarella Signed-off-by: Max Reitz Message-id: 20190724171239.8764-9-mre...@redhat.com Reviewed-by: Maxim Levitsky Signed-of

[Qemu-block] [PULL 10/17] iotests: Convert to preallocated encrypted qcow2

2019-08-19 Thread Max Reitz
Add a test case for converting an empty image (which only returns zeroes when read) to a preallocated encrypted qcow2 image. qcow2_has_zero_init() should return 0 then, thus forcing qemu-img convert to create zero clusters. Signed-off-by: Max Reitz Acked-by: Stefano Garzarella Tested-by: Stefano

[Qemu-block] [PULL 14/17] vmdk: Make block_status recurse for flat extents

2019-08-19 Thread Max Reitz
Fixes: 69f47505ee66afaa513305de0c1895a224e52c45 Signed-off-by: Max Reitz Message-id: 20190725155512.9827-3-mre...@redhat.com Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Signed-off-by: Max Reitz --- block/vmdk.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/v

[Qemu-block] [PULL 15/17] vpc: Do not return RAW from block_status

2019-08-19 Thread Max Reitz
vpc is not really a passthrough driver, even when using the fixed subformat (where host and guest offsets are equal). It should handle preallocation like all other drivers do, namely by returning DATA | RECURSE instead of RAW. There is no tangible difference but the fact that bdrv_is_allocated()

[Qemu-block] [PULL 11/17] iotests: Test convert -n to pre-filled image

2019-08-19 Thread Max Reitz
Signed-off-by: Max Reitz Message-id: 20190724171239.8764-11-mre...@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- tests/qemu-iotests/122 | 17 + tests/qemu-iotests/122.out | 8 2 files changed, 25 insertions(+) diff --git a/tests/qemu-iotests/

[Qemu-block] [PATCH v9] qemu-io: add pattern file for write command

2019-08-19 Thread Denis Plotnikov
The patch allows to provide a pattern file for write command. There was no similar ability before. Signed-off-by: Denis Plotnikov --- v9: * replace flag cast to int with bool [Eric] * fix the error message [Eric] * use qemu_io_free instead of qemu_vfree [Eric] * add function description [

[Qemu-block] [PULL 13/17] vdi: Make block_status recurse for fixed images

2019-08-19 Thread Max Reitz
Suggested-by: Vladimir Sementsov-Ogievskiy Fixes: 69f47505ee66afaa513305de0c1895a224e52c45 Signed-off-by: Max Reitz Message-id: 20190725155512.9827-2-mre...@redhat.com Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Signed-off-by: Max Reitz --- block/vdi.c | 3 ++- 1 file cha

[Qemu-block] [PULL 17/17] doc: Preallocation does not require writing zeroes

2019-08-19 Thread Max Reitz
When preallocating an encrypted qcow2 image, it just lets the protocol driver write data and then does not mark the clusters as zero. Therefore, reading this image will yield effectively random data. As such, we have not fulfilled the promise of always writing zeroes when preallocating an image in

Re: [Qemu-block] [PATCH v4 00/14] block: Try to create well-typed json:{} filenames

2019-08-19 Thread Max Reitz
Ping On 24.06.19 19:39, Max Reitz wrote: > Hi, > > There are two explanations of this cover letter, a relative one (to v3) > and an absolute one. > > > *** Important note *** > > The final patch in this series is an example that converts most of > block-core.json to use default values where po

[Qemu-block] [PATCH] nbd: Tolerate more errors to structured reply request

2019-08-19 Thread Eric Blake
A server may have a reason to reject a request for structured replies, beyond just not recognizing them as a valid request. It doesn't hurt us to continue talking to such a server; otherwise 'qemu-nbd --list' of such a server fails to display all possible details about the export. Encountered whe

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-19 Thread Eric Blake
On 8/17/19 8:31 PM, Nir Soffer wrote: >>> Also, for qemu-nbd, shouldn't we allow -e only together with -r ? >> >> I'm reluctant to; it might break whatever existing user is okay exposing >> it (although such users are questionable, so maybe we can argue they >> were already broken). Maybe it's tim

Re: [Qemu-block] [Qemu-devel] [PATCH v7 25/42] hw/misc: Declare device little or big endian

2019-08-19 Thread Paolo Bonzini
On 16/08/19 12:04, Philippe Mathieu-Daudé wrote: >> diff --git a/hw/misc/a9scu.c b/hw/misc/a9scu.c >> index 4307f00..3de8cd3 100644 >> --- a/hw/misc/a9scu.c >> +++ b/hw/misc/a9scu.c >> @@ -94,7 +94,7 @@ static void a9_scu_write(void *opaque, hwaddr offset, >>  static const MemoryRegionOps a9_scu_op

Re: [Qemu-block] [qemu-s390x] [Qemu-devel] [PATCH v7 33/42] exec: Replace device_endian with MemOp

2019-08-19 Thread Paolo Bonzini
On 16/08/19 12:12, Thomas Huth wrote: > This patch is *huge*, more than 800kB. It keeps being stuck in the the > filter of the qemu-s390x list each time you send it. Please: > > 1) Try to break it up in more digestible pieces, e.g. change only one > subsystem at a time (this is also better reviewa

Re: [Qemu-block] [qemu-s390x] [Qemu-devel] [PATCH v7 33/42] exec: Replace device_endian with MemOp

2019-08-19 Thread Paolo Bonzini
On 19/08/19 20:28, Paolo Bonzini wrote: > On 16/08/19 12:12, Thomas Huth wrote: >> This patch is *huge*, more than 800kB. It keeps being stuck in the the >> filter of the qemu-s390x list each time you send it. Please: >> >> 1) Try to break it up in more digestible pieces, e.g. change only one >> su

[Qemu-block] [PATCH v2 01/16] include: Move endof() up from hw/virtio/virtio.h

2019-08-19 Thread Max Reitz
endof() is a useful macro, we can make use of it outside of virtio. Signed-off-by: Max Reitz --- include/hw/virtio/virtio.h | 7 --- include/qemu/compiler.h| 7 +++ hw/block/virtio-blk.c | 4 ++-- hw/net/virtio-net.c| 10 +- 4 files changed, 14 insertions(+),

[Qemu-block] [PATCH v2 00/16] qcow2: Let check -r all repair some snapshot bits

2019-08-19 Thread Max Reitz
Hi, See the v1 cover letter here if you haven’t already: https://lists.nongnu.org/archive/html/qemu-block/2019-07/msg01290.html I kept all patches from the previous version because Eric deemed the patches I might have dropped useful. I kept the way I implemented fixing overly long snapshot table

[Qemu-block] [PATCH v2 04/16] qcow2: Keep unknown extra snapshot data

2019-08-19 Thread Max Reitz
The qcow2 specification says to ignore unknown extra data fields in snapshot table entries. Currently, we discard it whenever we update the image, which is a bit different from "ignore". This patch makes the qcow2 driver keep all unknown extra data fields when updating an image's snapshot table.

[Qemu-block] [PATCH v2 02/16] qcow2: Use endof()

2019-08-19 Thread Max Reitz
Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index d0e7fa9311..752883e5c3 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -92,11 +92,12 @@ int qcow

[Qemu-block] [PATCH v2 06/16] qcow2: Put qcow2_upgrade() into its own function

2019-08-19 Thread Max Reitz
This does not make sense right now, but it will make sense once we need to do more than to just update s->qcow_version. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.c | 43 ++- 1 file changed, 38 insertions(+), 5 deletions(-) diff --g

[Qemu-block] [PATCH v2 05/16] qcow2: Make qcow2_write_snapshots() public

2019-08-19 Thread Max Reitz
Updating the snapshot list will be useful when upgrading a v2 image to v3, so we will need to call this function in qcow2.c. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 1 + block/qcow2-snapshot.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --

[Qemu-block] [PATCH v2 03/16] qcow2: Add Error ** to qcow2_read_snapshots()

2019-08-19 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 2 +- block/qcow2-snapshot.c | 7 ++- block/qcow2.c | 3 +-- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index fc1b0d3c1e..175708cee0 100644 --- a/block/qco

[Qemu-block] [PATCH v2 11/16] qcow2: Keep track of the snapshot table length

2019-08-19 Thread Max Reitz
When repairing the snapshot table, we truncate entries that have too much extra data. This frees up space that we do not have to count towards the snapshot table size. Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff

[Qemu-block] [PATCH v2 08/16] qcow2: Separate qcow2_check_read_snapshot_table()

2019-08-19 Thread Max Reitz
Reading the snapshot table can fail. That is a problem when we want to repair the image. Therefore, stop reading the snapshot table in qcow2_do_open() in check mode. Instead, add a new function qcow2_check_read_snapshot_table() that reads the snapshot table at a later point. In the future, we w

[Qemu-block] [PATCH v2 12/16] qcow2: Fix overly long snapshot tables

2019-08-19 Thread Max Reitz
We currently refuse to open qcow2 images with overly long snapshot tables. This patch makes qemu-img check -r all drop all offending entries past what we deem acceptable. Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 88 +- 1 file changed, 78 inse

[Qemu-block] [PATCH v2 09/16] qcow2: Add qcow2_check_fix_snapshot_table()

2019-08-19 Thread Max Reitz
qcow2_check_read_snapshot_table() can perform consistency checks, but it cannot fix everything. Specifically, it cannot allocate new clusters, because that should wait until the refcount structures are known to be consistent (i.e., after qcow2_check_refcounts()). Thus, it cannot call qcow2_write_

[Qemu-block] [PATCH v2 14/16] qcow2: Fix v3 snapshot table entry compliancy

2019-08-19 Thread Max Reitz
qcow2 v3 images require every snapshot table entry to have at least 16 bytes of extra data. If they do not, let qemu-img check -r all fix it. Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 18 ++ 1 file changed, 18 insertions(+) diff --git a/block/qcow2-snapshot.c b/bloc

[Qemu-block] [PATCH v2 16/16] iotests: Test qcow2's snapshot table handling

2019-08-19 Thread Max Reitz
Add a test how our qcow2 driver handles extra data in snapshot table entries, and how it repairs overly long snapshot tables. Signed-off-by: Max Reitz --- tests/qemu-iotests/261 | 523 + tests/qemu-iotests/261.out | 346 tests/qemu

[Qemu-block] [PATCH v2 07/16] qcow2: Write v3-compliant snapshot list on upgrade

2019-08-19 Thread Max Reitz
qcow2 v3 requires every snapshot table entry to have two extra data fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present. qcow2_upgrade() therefore should update the snapshot table to ensure all entries have these extra data fiel

[Qemu-block] [PATCH v2 10/16] qcow2: Fix broken snapshot table entries

2019-08-19 Thread Max Reitz
The only case where we currently reject snapshot table entries is when they have too much extra data. Fix them with qemu-img check -r all by counting it as a corruption, reducing their extra_data_size, and then letting qcow2_check_fix_snapshot_table() do the rest. Signed-off-by: Max Reitz --- b

[Qemu-block] [PATCH v2 13/16] qcow2: Repair snapshot table with too many entries

2019-08-19 Thread Max Reitz
Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 366d9f574c..dac8a778e4 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -444,6 +444,14 @@ int coroutine_

Re: [Qemu-block] [PATCH v2 01/16] include: Move endof() up from hw/virtio/virtio.h

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > endof() is a useful macro, we can make use of it outside of virtio. > > Signed-off-by: Max Reitz > --- > include/hw/virtio/virtio.h | 7 --- > include/qemu/compiler.h| 7 +++ > hw/block/virtio-blk.c | 4 ++-- > hw/net/virtio-net.c

Re: [Qemu-block] [PATCH v2 02/16] qcow2: Use endof()

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > block/qcow2-snapshot.c | 7 --- > 1 file changed, 4 insertions(+), 3 deletions(-) What, the file got larger in spite of using a helper macro for ease of readability? Good thing that 'lines of code' is not the only metri

[Qemu-block] [PATCH v2 15/16] iotests: Add peek_file* functions

2019-08-19 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/common.rc | 20 1 file changed, 20 insertions(+) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 5502c3da2f..78decfd5d5 100644 --- a/tests/qemu-iotests/common.rc +++ b/tes

Re: [Qemu-block] [PATCH] iotests: Add more "skip_if_unsupported" statements to the python tests

2019-08-19 Thread Max Reitz
On 19.08.19 11:21, Thomas Huth wrote: > The python code already contains a possibility to skip tests if the > corresponding driver is not available in the qemu binary - use it > in more spots to avoid that the tests are failing if the driver has > been disabled. > > Signed-off-by: Thomas Huth > -

Re: [Qemu-block] [PATCH v2 04/16] qcow2: Keep unknown extra snapshot data

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > The qcow2 specification says to ignore unknown extra data fields in > snapshot table entries. Currently, we discard it whenever we update the > image, which is a bit different from "ignore". > > This patch makes the qcow2 driver keep all unknown extra data f

Re: [Qemu-block] [PATCH v2 07/16] qcow2: Write v3-compliant snapshot list on upgrade

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > qcow2 v3 requires every snapshot table entry to have two extra data > fields: The 64-bit VM state size, and the virtual disk size. Both are > optional for v2 images, so they may not be present. > > qcow2_upgrade() therefore should update the snapshot table t

Re: [Qemu-block] [PATCH v2 04/16] qcow2: Keep unknown extra snapshot data

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > The qcow2 specification says to ignore unknown extra data fields in > snapshot table entries. Currently, we discard it whenever we update the > image, which is a bit different from "ignore". > > This patch makes the qcow2 driver keep all unknown extra data f

Re: [Qemu-block] [PATCH v2 10/16] qcow2: Fix broken snapshot table entries

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > The only case where we currently reject snapshot table entries is when > they have too much extra data. Fix them with qemu-img check -r all by > counting it as a corruption, reducing their extra_data_size, and then > letting qcow2_check_fix_snapshot_table() d

Re: [Qemu-block] [PATCH v2 11/16] qcow2: Keep track of the snapshot table length

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > When repairing the snapshot table, we truncate entries that have too > much extra data. This frees up space that we do not have to count > towards the snapshot table size. > > Signed-off-by: Max Reitz > --- > block/qcow2-snapshot.c | 14 +- > 1

Re: [Qemu-block] [PATCH v2 12/16] qcow2: Fix overly long snapshot tables

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > We currently refuse to open qcow2 images with overly long snapshot > tables. This patch makes qemu-img check -r all drop all offending > entries past what we deem acceptable. > > Signed-off-by: Max Reitz > --- > block/qcow2-snapshot.c | 88

Re: [Qemu-block] [PATCH v2 13/16] qcow2: Repair snapshot table with too many entries

2019-08-19 Thread Eric Blake
On 8/19/19 1:55 PM, Max Reitz wrote: > Signed-off-by: Max Reitz > --- Short on the reasoning why this isn't a problem in practice. (Again, because we only do it via opt-in qemu-img -r; you can already learn if qemu-img will have problem with your file created externally without destroying the im

Re: [Qemu-block] [Qemu-devel] [PATCH] block/io.c: fix for the allocation failure

2019-08-19 Thread Denis V. Lunev
On 8/17/19 5:56 PM, Eric Blake wrote: > On 8/17/19 9:49 AM, Eric Blake wrote: > >>> This change is a regression of sorts. Now, you are unconditionally >>> attempting the fallback for ALL failures (such as EIO) and for all >>> drivers, even when that was not previously attempted and increases the >

Re: [Qemu-block] [PATCH v2 14/16] qcow2: Fix v3 snapshot table entry compliancy

2019-08-19 Thread Eric Blake
On 8/19/19 1:56 PM, Max Reitz wrote: > qcow2 v3 images require every snapshot table entry to have at least 16 > bytes of extra data. If they do not, let qemu-img check -r all fix it. > > Signed-off-by: Max Reitz > --- > block/qcow2-snapshot.c | 18 ++ > 1 file changed, 18 insert

[Qemu-block] [PATCH v3 2/8] iotests: Prefer null-co over null-aio

2019-08-19 Thread Max Reitz
We use null-co basically everywhere in the iotests. Unless we want to test null-aio specifically, we should use it instead (for consistency). Signed-off-by: Max Reitz Reviewed-by: John Snow --- tests/qemu-iotests/093 | 7 +++ tests/qemu-iotests/245 | 2 +- 2 files changed, 4 insertions(+),

[Qemu-block] [PATCH v3 6/8] iotests: Test driver whitelisting in 093

2019-08-19 Thread Max Reitz
null-aio may not be whitelisted. Skip all test cases that require it. Signed-off-by: Max Reitz --- tests/qemu-iotests/093 | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 index 50c1e7f2ec..f03fa24a07 100755 --- a/te

Re: [Qemu-block] [PATCH v3 2/8] iotests: Prefer null-co over null-aio

2019-08-19 Thread Max Reitz
On 19.08.19 22:18, Max Reitz wrote: > We use null-co basically everywhere in the iotests. Unless we want to > test null-aio specifically, we should use it instead (for consistency). > > Signed-off-by: Max Reitz > Reviewed-by: John Snow Hm, sorry, I just noticed that I probably should have drop

[Qemu-block] [PATCH v3 1/8] iotests: Add -display none to the qemu options

2019-08-19 Thread Max Reitz
Without this argument, qemu will print an angry message about not being able to connect to a display server if $DISPLAY is not set. For me, that breaks iotests.supported_formats() because it thus only sees ["Could", "not", "connect"] as the supported formats. Signed-off-by: Max Reitz Reviewed-by

[Qemu-block] [PATCH v3 4/8] iotests: Use case_skip() in skip_if_unsupported()

2019-08-19 Thread Max Reitz
skip_if_unsupported() should use the stronger variant case_skip(), because this allows it to be used even with setUp() (in a meaningful way). Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.

[Qemu-block] [PATCH v3 5/8] iotests: Let skip_if_unsupported() accept a method

2019-08-19 Thread Max Reitz
This lets tests use skip_if_unsupported() with a potentially variable list of required formats. Suggested-by: Kevin Wolf Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/q

[Qemu-block] [PATCH v3 0/8] iotests: Selfish patches

2019-08-19 Thread Max Reitz
Hi, Nothing has changed too much since the previous version, thus I’ll start with a link to its cover letter: https://lists.nongnu.org/archive/html/qemu-block/2019-06/msg01102.html What has changed is this: v3: - Patches 2 and 3: Resolved rebase conflicts - Added patch 4. It seems useful to

[Qemu-block] [PATCH v3 7/8] iotests: Test driver whitelisting in 136

2019-08-19 Thread Max Reitz
null-aio may not be whitelisted. Skip all test cases that require it. Signed-off-by: Max Reitz --- tests/qemu-iotests/136 | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/136 b/tests/qemu-iotests/136 index a46a7b7630..012ea111ac 100755 --- a

[Qemu-block] [PATCH v3 3/8] iotests: Allow skipping test cases

2019-08-19 Thread Max Reitz
case_notrun() does not actually skip the current test case. It just adds a "notrun" note and then returns to the caller, who manually has to skip the test. Generally, skipping a test case is as simple as returning from the current function, but not always: For example, this model does not allow s

Re: [Qemu-block] [Qemu-devel] [PATCH] block/io.c: fix for the allocation failure

2019-08-19 Thread Eric Blake
On 8/19/19 2:46 PM, Denis V. Lunev wrote: > On 8/17/19 5:56 PM, Eric Blake wrote: >> On 8/17/19 9:49 AM, Eric Blake wrote: >> This change is a regression of sorts. Now, you are unconditionally attempting the fallback for ALL failures (such as EIO) and for all drivers, even when that

Re: [Qemu-block] [PATCH v2 16/16] iotests: Test qcow2's snapshot table handling

2019-08-19 Thread Eric Blake
On 8/19/19 1:56 PM, Max Reitz wrote: > Add a test how our qcow2 driver handles extra data in snapshot table > entries, and how it repairs overly long snapshot tables. > > Signed-off-by: Max Reitz > --- > +++ b/tests/qemu-iotests/261.out > @@ -0,0 +1,346 @@ > +QA output created by 261 > + > +===

[Qemu-block] [PATCH v3 8/8] iotests: Cache supported_formats()

2019-08-19 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 8f315538e9..f6492b355f 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iot

Re: [Qemu-block] [PATCH] iotests: Add more "skip_if_unsupported" statements to the python tests

2019-08-19 Thread Max Reitz
On 19.08.19 21:13, Max Reitz wrote: > On 19.08.19 11:21, Thomas Huth wrote: >> The python code already contains a possibility to skip tests if the >> corresponding driver is not available in the qemu binary - use it >> in more spots to avoid that the tests are failing if the driver has >> been disa

Re: [Qemu-block] [Qemu-devel] [PATCH] block/io.c: fix for the allocation failure

2019-08-19 Thread Denis V. Lunev
On 8/19/19 11:30 PM, Eric Blake wrote: > On 8/19/19 2:46 PM, Denis V. Lunev wrote: >> On 8/17/19 5:56 PM, Eric Blake wrote: >>> On 8/17/19 9:49 AM, Eric Blake wrote: >>> > This change is a regression of sorts. Now, you are unconditionally > attempting the fallback for ALL failures (such as

Re: [Qemu-block] [Qemu-devel] [qemu-s390x] [PATCH v7 33/42] exec: Replace device_endian with MemOp

2019-08-19 Thread Richard Henderson
On 8/19/19 11:29 AM, Paolo Bonzini wrote: > On 19/08/19 20:28, Paolo Bonzini wrote: >> On 16/08/19 12:12, Thomas Huth wrote: >>> This patch is *huge*, more than 800kB. It keeps being stuck in the the >>> filter of the qemu-s390x list each time you send it. Please: >>> >>> 1) Try to break it up in m

Re: [Qemu-block] [Qemu-devel] [PATCH] block/io.c: fix for the allocation failure

2019-08-19 Thread Eric Blake
On 8/19/19 3:53 PM, Denis V. Lunev wrote: Or even better, fix the call site of fallocate() to skip attempting an unaligned fallocate(), and just directly return ENOTSUP, rather than trying to diagnose EINVAL after the fact. >>> No way. Single ENOTSUP will turn off fallocate() s

Re: [Qemu-block] [Qemu-devel] [qemu-s390x] [PATCH v7 33/42] exec: Replace device_endian with MemOp

2019-08-19 Thread Edgar E. Iglesias
On Mon, 19 Aug. 2019, 23:01 Richard Henderson, wrote: > On 8/19/19 11:29 AM, Paolo Bonzini wrote: > > On 19/08/19 20:28, Paolo Bonzini wrote: > >> On 16/08/19 12:12, Thomas Huth wrote: > >>> This patch is *huge*, more than 800kB. It keeps being stuck in the the > >>> filter of the qemu-s390x list

Re: [Qemu-block] [PATCH v3 1/8] iotests: Add -display none to the qemu options

2019-08-19 Thread Thomas Huth
On 8/19/19 10:18 PM, Max Reitz wrote: > Without this argument, qemu will print an angry message about not being > able to connect to a display server if $DISPLAY is not set. For me, > that breaks iotests.supported_formats() because it thus only sees > ["Could", "not", "connect"] as the supported f

Re: [Qemu-block] [PATCH v3 6/8] iotests: Test driver whitelisting in 093

2019-08-19 Thread Thomas Huth
On 8/19/19 10:18 PM, Max Reitz wrote: > null-aio may not be whitelisted. Skip all test cases that require it. > > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/093 | 12 +--- > 1 file changed, 9 insertions(+), 3 deletions(-) > > diff --git a/tests/qemu-iotests/093 b/tests/qemu-io

Re: [Qemu-block] [PATCH v3 2/8] iotests: Prefer null-co over null-aio

2019-08-19 Thread Thomas Huth
On 8/19/19 10:18 PM, Max Reitz wrote: > We use null-co basically everywhere in the iotests. Unless we want to > test null-aio specifically, we should use it instead (for consistency). > > Signed-off-by: Max Reitz > Reviewed-by: John Snow > --- > tests/qemu-iotests/093 | 7 +++ > tests/qemu