Re: [Qemu-block] [Qemu-devel] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result

2019-07-24 Thread Markus Armbruster
John Snow writes: > On 7/24/19 12:47 AM, Markus Armbruster wrote: >> John Snow writes: >> >>> From: Vladimir Sementsov-Ogievskiy >>> >>> Let's add a possibility to query dirty-bitmaps not only on root nodes. >>> It is useful when dealing both with snapshots and incremental backups. >>> >>> Sig

Re: [Qemu-block] [QEMU] [PATCH v5 0/8] Add Qemu to SeaBIOS LCHS interface

2019-07-24 Thread John Snow
On 7/24/19 8:47 PM, John Snow wrote: > > > On 7/19/19 6:10 AM, Sam Eiderman wrote: >> Well, this patch introduces 3 command line parameters (“lcyls”, “lheads”, >> “lsecs”) >> to “scsi-hd” “ide-hd” and “virtio-pci-blk” so this somehow has something to >> do with >> block. >> >> This patch als

Re: [Qemu-block] [QEMU] [PATCH v5 0/8] Add Qemu to SeaBIOS LCHS interface

2019-07-24 Thread John Snow
On 7/19/19 6:10 AM, Sam Eiderman wrote: > Well, this patch introduces 3 command line parameters (“lcyls”, “lheads”, > “lsecs”) > to “scsi-hd” “ide-hd” and “virtio-pci-blk” so this somehow has something to > do with > block. > > This patch also adds fw_cfg interface to send these parameters to

Re: [Qemu-block] [PATCH-for-4.1? 6/7] vl: Rewrite a fall through comment

2019-07-24 Thread John Snow
On 7/19/19 9:14 AM, Philippe Mathieu-Daudé wrote: > GCC9 is confused by this comment when building with CFLAG > -Wimplicit-fallthrough=2: > > vl.c: In function ‘qemu_ref_timedate’: > vl.c:773:15: error: this statement may fall through > [-Werror=implicit-fallthrough=] > 773 | v

Re: [Qemu-block] [PATCH-for-4.1 3/7] hw/block/pflash_cfi02: Rewrite a fall through comment

2019-07-24 Thread John Snow
On 7/22/19 7:43 AM, Philippe Mathieu-Daudé wrote: > On 7/19/19 3:14 PM, Philippe Mathieu-Daudé wrote: >> GCC9 is confused by this comment when building with CFLAG >> -Wimplicit-fallthrough=2: >> >> hw/block/pflash_cfi02.c: In function ‘pflash_write’: >> hw/block/pflash_cfi02.c:574:16: error:

Re: [Qemu-block] [PATCH] Fixes: a6862418fec4072 iotests change in 051.out

2019-07-24 Thread John Snow
On 7/24/19 4:47 AM, Christian Borntraeger wrote: > > > On 24.07.19 10:25, Andrey Shinkevich wrote: >> The patch "iotests: Set read-zeroes on in null block driver for Valgrind" >> needs the change in 051.out when compared against on the s390 system. >> >> Reported-by: Christian Borntraeger > T

Re: [Qemu-block] [Qemu-devel] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result

2019-07-24 Thread John Snow
On 7/24/19 12:47 AM, Markus Armbruster wrote: > John Snow writes: > >> From: Vladimir Sementsov-Ogievskiy >> >> Let's add a possibility to query dirty-bitmaps not only on root nodes. >> It is useful when dealing both with snapshots and incremental backups. >> >> Signed-off-by: Vladimir Sement

Re: [Qemu-block] [PATCH v3] block/rbd: add preallocation support

2019-07-24 Thread Jason Dillaman
On Tue, Jul 23, 2019 at 3:13 AM Stefano Garzarella wrote: > > This patch adds the support of preallocation (off/full) for the RBD > block driver. > If rbd_writesame() is available and supports zeroed buffers, we use > it to quickly fill the image when full preallocation is required. > > Signed-off

[Qemu-block] [PATCH v2 08/11] vhdx: Fix .bdrv_has_zero_init()

2019-07-24 Thread Max Reitz
Fixed VHDX images cannot guarantee to be zero-initialized. If the image has the "fixed" subformat, forward the call to the underlying storage node. Reported-by: Stefano Garzarella Signed-off-by: Max Reitz --- block/vhdx.c | 26 +- 1 file changed, 25 insertions(+), 1 del

[Qemu-block] [PATCH v2 09/11] iotests: Convert to preallocated encrypted qcow2

2019-07-24 Thread Max Reitz
Add a test case for converting an empty image (which only returns zeroes when read) to a preallocated encrypted qcow2 image. qcow2_has_zero_init() should return 0 then, thus forcing qemu-img convert to create zero clusters. Signed-off-by: Max Reitz Acked-by: Stefano Garzarella Tested-by: Stefano

Re: [Qemu-block] [PATCH v6 14/14] qemu-iotest: enable testing with qemu-io aio options

2019-07-24 Thread Julia Suvorova
On 7/19/19 3:35 PM, Aarushi Mehta wrote: @@ -225,6 +227,10 @@ s/ .*//p CACHEMODE_IS_DEFAULT=false cachemode=false continue +elif $aiomode +then +AIOMODE="$r" +aiomode=false 'continue' is missed here. Best regards, Julia Suvorova.

[Qemu-block] [PATCH v2 07/11] vdi: Fix .bdrv_has_zero_init()

2019-07-24 Thread Max Reitz
Static VDI images cannot guarantee to be zero-initialized. If the image has been statically allocated, forward the call to the underlying storage node. Reported-by: Stefano Garzarella Signed-off-by: Max Reitz Reviewed-by: Stefan Weil Acked-by: Stefano Garzarella Tested-by: Stefano Garzarella

[Qemu-block] [PATCH v2 10/11] iotests: Test convert -n to pre-filled image

2019-07-24 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/122 | 17 + tests/qemu-iotests/122.out | 8 2 files changed, 25 insertions(+) diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122 index 85c3a8d047..059011ebb1 100755 --- a/tests/qemu-iotests/122 +++ b/tests/qe

[Qemu-block] [PATCH v2 11/11] iotests: Full mirror to existing non-zero image

2019-07-24 Thread Max Reitz
The result of a sync=full mirror should always be the equal to the input. Therefore, existing images should be treated as potentially non-zero and thus should be explicitly initialized to be zero beforehand. Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 62 ++

[Qemu-block] [PATCH v2 05/11] block: Use bdrv_has_zero_init_truncate()

2019-07-24 Thread Max Reitz
Signed-off-by: Max Reitz --- block/parallels.c | 2 +- block/vhdx.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 00fae125d1..7cd2714b69 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -835,7 +835,7 @@ static int

[Qemu-block] [PATCH v2 06/11] qcow2: Fix .bdrv_has_zero_init()

2019-07-24 Thread Max Reitz
If a qcow2 file is preallocated, it can no longer guarantee that it initially appears as filled with zeroes. So implement .bdrv_has_zero_init() by checking whether the file is preallocated; if so, forward the call to the underlying storage node, except for when it is encrypted: Encrypted prealloca

Re: [Qemu-block] [PATCH v6 11/14] qemu-io: adds option to use aio engine

2019-07-24 Thread Julia Suvorova
On 7/19/19 3:35 PM, Aarushi Mehta wrote: @@ -489,7 +493,7 @@ static QemuOptsList file_opts = { int main(int argc, char **argv) { int readonly = 0; -const char *sopt = "hVc:d:f:rsnCmkt:T:U"; +const char *sopt = "hVc:d:f:rsnCmit:T:U"; Add ':' after 'i' to pass an argument after

[Qemu-block] [PATCH v2 04/11] block: Implement .bdrv_has_zero_init_truncate()

2019-07-24 Thread Max Reitz
We need to implement .bdrv_has_zero_init_truncate() for every block driver that supports truncation and has a .bdrv_has_zero_init() implementation. Implement it the same way each driver implements .bdrv_has_zero_init(). This is at least not any more unsafe than what we had before. Signed-off-by:

[Qemu-block] [PATCH v2 03/11] block: Add bdrv_has_zero_init_truncate()

2019-07-24 Thread Max Reitz
No .bdrv_has_zero_init() implementation returns 1 if growing the file would add non-zero areas (at least with PREALLOC_MODE_OFF), so using it in lieu of this new function was always safe. But on the other hand, it is possible that growing an image that is not zero-initialized would still add a zer

[Qemu-block] [PATCH v2 00/11] block: Fix some things about bdrv_has_zero_init()

2019-07-24 Thread Max Reitz
Hi, See the previous cover letter for the reason for patches 6 through 9: https://lists.nongnu.org/archive/html/qemu-block/2019-07/msg00563.html But no only some bdrv_has_zero_init() implementations are wrong, some callers also use it the wrong way. First, qemu-img and mirror use it for pre-exis

[Qemu-block] [PATCH v2 01/11] qemu-img: Fix bdrv_has_zero_init() use in convert

2019-07-24 Thread Max Reitz
bdrv_has_zero_init() only has meaning for newly created images or image areas. If qemu-img convert did not create the image itself, it cannot rely on bdrv_has_zero_init()'s result to carry any meaning. Signed-off-by: Max Reitz --- qemu-img.c | 11 --- 1 file changed, 8 insertions(+), 3

[Qemu-block] [PATCH v2 02/11] mirror: Fix bdrv_has_zero_init() use

2019-07-24 Thread Max Reitz
bdrv_has_zero_init() only has meaning for newly created images or image areas. If the mirror job itself did not create the image, it cannot rely on bdrv_has_zero_init()'s result to carry any meaning. This is the case for drive-mirror with mode=existing and always for blockdev-mirror. Note that w

Re: [Qemu-block] [Qemu-devel] qemu-iotests 069 and 111 are failing on NetBSD

2019-07-24 Thread Paolo Bonzini
On 24/07/19 11:34, Thomas Huth wrote: > In case somebody is interested, two of the "auto" iotests are failing > on NetBSD due to non-matching output: > > TESTiotest-qcow2: 069 [fail] > --- /var/tmp/qemu-test.1BMupF/tests/qemu-iotests/069.out2019-07-24 > 09:19:22.0 + > ++

Re: [Qemu-block] [PATCH v3 2/3] qapi: implement block-dirty-bitmap-remove transaction action

2019-07-24 Thread John Snow
On 7/24/19 9:58 AM, Vladimir Sementsov-Ogievskiy wrote: > 09.07.2019 1:05, John Snow wrote: >> It is used to do transactional movement of the bitmap (which is >> possible in conjunction with merge command). Transactional bitmap >> movement is needed in scenarios with external snapshot, when we d

Re: [Qemu-block] [PATCH v3 2/3] qapi: implement block-dirty-bitmap-remove transaction action

2019-07-24 Thread Vladimir Sementsov-Ogievskiy
09.07.2019 1:05, John Snow wrote: > It is used to do transactional movement of the bitmap (which is > possible in conjunction with merge command). Transactional bitmap > movement is needed in scenarios with external snapshot, when we don't > want to leave copy of the bitmap in the base image. > >

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/3] qapi: block-dirty-bitmap-remove transaction action

2019-07-24 Thread John Snow
On 7/24/19 7:12 AM, Vladimir Sementsov-Ogievskiy wrote: > 15.07.2019 22:48, John Snow wrote: >> >> >> On 7/8/19 6:04 PM, John Snow wrote: >>> Hi, this is a proposal based off of Vladimir's patchset: >>> [Qemu-devel] [PATCH 0/4] qapi: block-dirty-bitmap-remove transaction action >>> >>> === >>> V

Re: [Qemu-block] [PATCH] util/async: hold AioContext ref to prevent use-after-free

2019-07-24 Thread Stefan Hajnoczi
On Tue, Jul 23, 2019 at 08:06:23PM +0100, Stefan Hajnoczi wrote: > The tests/test-bdrv-drain /bdrv-drain/iothread/drain test case does the > following: > > 1. The preadv coroutine calls aio_bh_schedule_oneshot() and then yields. > 2. The one-shot BH executes in another AioContext. All it does is

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/3] qapi: block-dirty-bitmap-remove transaction action

2019-07-24 Thread Vladimir Sementsov-Ogievskiy
15.07.2019 22:48, John Snow wrote: > > > On 7/8/19 6:04 PM, John Snow wrote: >> Hi, this is a proposal based off of Vladimir's patchset: >> [Qemu-devel] [PATCH 0/4] qapi: block-dirty-bitmap-remove transaction action >> >> === >> V3: >> === >> >> 001/3:[] [--] 'blockdev: reduce aio_context loc

Re: [Qemu-block] [PATCH] util/async: hold AioContext ref to prevent use-after-free

2019-07-24 Thread Philippe Mathieu-Daudé
On 7/23/19 9:09 PM, Stefan Hajnoczi wrote: > On Tue, Jul 23, 2019 at 8:06 PM Stefan Hajnoczi wrote: >> So if co causes ctx to be freed then we're in trouble. Fix this problem >> by holding a reference to ctx. > > For QEMU 4.2. I'm not aware of a way to trigger this bug in QEMU > proper. This f

Re: [Qemu-block] [PATCH v5 34/42] block: Inline bdrv_co_block_status_from_*()

2019-07-24 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 16:39, Vladimir Sementsov-Ogievskiy wrote: > 13.06.2019 1:09, Max Reitz wrote: >> With bdrv_filtered_rw_bs(), we can easily handle this default filter >> behavior in bdrv_co_block_status(). >> >> blkdebug wants to have an additional assertion, so it keeps its own >> implementation, excep

Re: [Qemu-block] [PATCH v5 33/42] blockdev: Fix active commit choice

2019-07-24 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 16:26, Vladimir Sementsov-Ogievskiy wrote: > 19.06.2019 18:59, Max Reitz wrote: >> On 19.06.19 11:31, Vladimir Sementsov-Ogievskiy wrote: >>> 13.06.2019 1:09, Max Reitz wrote: We have to perform an active commit whenever the top node has a parent that has taken the WRITE permis

Re: [Qemu-block] [PATCH v5 30/42] qemu-img: Use child access functions

2019-07-24 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 16:15, Vladimir Sementsov-Ogievskiy wrote: > 19.06.2019 18:49, Max Reitz wrote: >> On 19.06.19 11:18, Vladimir Sementsov-Ogievskiy wrote: >>> 13.06.2019 1:09, Max Reitz wrote: This changes iotest 204's output, because blkdebug on top of a COW node used to make qemu-img map disr

[Qemu-block] [PATCH v3] blockjob: drain all job nodes in block_job_drain

2019-07-24 Thread Vladimir Sementsov-Ogievskiy
Instead of draining additional nodes in each job code, let's do it in common block_job_drain, draining just all job's children. BlockJobDriver.drain becomes unused, so, drop it at all. It's also a first step to finally get rid of blockjob->blk. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v3

[Qemu-block] qemu-iotests 069 and 111 are failing on NetBSD

2019-07-24 Thread Thomas Huth
In case somebody is interested, two of the "auto" iotests are failing on NetBSD due to non-matching output: TESTiotest-qcow2: 069 [fail] --- /var/tmp/qemu-test.1BMupF/tests/qemu-iotests/069.out2019-07-24 09:19:22.0 + +++ /var/tmp/qemu-test.1BMupF/tests/qemu-iotests/069.o

Re: [Qemu-block] [PATCH] Fixes: a6862418fec4072 iotests change in 051.out

2019-07-24 Thread Christian Borntraeger
On 24.07.19 10:25, Andrey Shinkevich wrote: > The patch "iotests: Set read-zeroes on in null block driver for Valgrind" > needs the change in 051.out when compared against on the s390 system. > > Reported-by: Christian Borntraeger Tested-by: Christian Borntraeger FWIW, the Fixes tag should b

Re: [Qemu-block] [PATCH] util/async: hold AioContext ref to prevent use-after-free

2019-07-24 Thread Paolo Bonzini
On 23/07/19 21:06, Stefan Hajnoczi wrote: > The tests/test-bdrv-drain /bdrv-drain/iothread/drain test case does the > following: > > 1. The preadv coroutine calls aio_bh_schedule_oneshot() and then yields. > 2. The one-shot BH executes in another AioContext. All it does is call >aio_co_wakeup

Re: [Qemu-block] [Qemu-devel] [PATCH v4 00/18] bitmaps: introduce 'bitmap' sync mode

2019-07-24 Thread Fabian Grünbichler
On Tue, Jul 23, 2019 at 12:58:10PM -0400, John Snow wrote: > > > On 7/23/19 5:47 AM, Fabian Grünbichler wrote: > > On Mon, Jul 22, 2019 at 01:21:02PM -0400, John Snow wrote: > >> > >> > >> On 7/22/19 8:17 AM, Fabian Grünbichler wrote: > >>> On Tue, Jul 09, 2019 at 07:25:32PM -0400, John Snow wrot

[Qemu-block] [PATCH] Fixes: a6862418fec4072 iotests change in 051.out

2019-07-24 Thread Andrey Shinkevich
The patch "iotests: Set read-zeroes on in null block driver for Valgrind" needs the change in 051.out when compared against on the s390 system. Reported-by: Christian Borntraeger Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/051.out | 10 +- 1 file changed, 5 insertions(+), 5

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Andrey Shinkevich
On 24/07/2019 11:05, Kevin Wolf wrote: > Am 24.07.2019 um 09:57 hat Andrey Shinkevich geschrieben: >> >> >> On 24/07/2019 10:38, Kevin Wolf wrote: >>> Am 24.07.2019 um 09:30 hat Andrey Shinkevich geschrieben: On 24/07/2019 10:18, Christian Borntraeger wrote: > > On 19.07.19

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Kevin Wolf
Am 24.07.2019 um 09:57 hat Andrey Shinkevich geschrieben: > > > On 24/07/2019 10:38, Kevin Wolf wrote: > > Am 24.07.2019 um 09:30 hat Andrey Shinkevich geschrieben: > >> > >> > >> On 24/07/2019 10:18, Christian Borntraeger wrote: > >>> > >>> On 19.07.19 15:43, Kevin Wolf wrote: > From: Andre

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Andrey Shinkevich
On 24/07/2019 10:38, Kevin Wolf wrote: > Am 24.07.2019 um 09:30 hat Andrey Shinkevich geschrieben: >> >> >> On 24/07/2019 10:18, Christian Borntraeger wrote: >>> >>> On 19.07.19 15:43, Kevin Wolf wrote: From: Andrey Shinkevich The Valgrind tool reports about the uninitialised buff

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Kevin Wolf
Am 24.07.2019 um 09:30 hat Andrey Shinkevich geschrieben: > > > On 24/07/2019 10:18, Christian Borntraeger wrote: > > > > On 19.07.19 15:43, Kevin Wolf wrote: > >> From: Andrey Shinkevich > >> > >> The Valgrind tool reports about the uninitialised buffer 'buf' > >> instantiated on the stack of

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Andrey Shinkevich
On 24/07/2019 10:33, Christian Borntraeger wrote: > > > On 24.07.19 09:30, Andrey Shinkevich wrote: >> >> >> On 24/07/2019 10:18, Christian Borntraeger wrote: >>> >>> On 19.07.19 15:43, Kevin Wolf wrote: From: Andrey Shinkevich The Valgrind tool reports about the uninitialised b

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Christian Borntraeger
On 24.07.19 09:30, Andrey Shinkevich wrote: > > > On 24/07/2019 10:18, Christian Borntraeger wrote: >> >> On 19.07.19 15:43, Kevin Wolf wrote: >>> From: Andrey Shinkevich >>> >>> The Valgrind tool reports about the uninitialised buffer 'buf' >>> instantiated on the stack of the function guess

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Andrey Shinkevich
On 24/07/2019 10:18, Christian Borntraeger wrote: > > On 19.07.19 15:43, Kevin Wolf wrote: >> From: Andrey Shinkevich >> >> The Valgrind tool reports about the uninitialised buffer 'buf' >> instantiated on the stack of the function guess_disk_lchs(). >> Pass 'read-zeroes=on' to the null block d

Re: [Qemu-block] [Qemu-devel] [PULL 01/13] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-24 Thread Christian Borntraeger
On 19.07.19 15:43, Kevin Wolf wrote: > From: Andrey Shinkevich > > The Valgrind tool reports about the uninitialised buffer 'buf' > instantiated on the stack of the function guess_disk_lchs(). > Pass 'read-zeroes=on' to the null block driver to make it deterministic. > The output of the tests 0