Re: [PATCH 02/23] iotests.py: Store socket files in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > iotests.py itself does not store socket files, but it machine.py and > qtest.py do. iotests.py needs to pass the respective path to them, and > they need to adhere to it. > > Signed-off-by: Max Reitz > --- > python/qemu/machine.py| 15

Re: [PATCH 00/23] iotests: Add and use $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Hi, > > Perhaps the main reason we cannot run important tests such as 041 in CI > is that when they care Unix sockets in $TEST_DIR, the path may become > too long to connect to them. > > To get by this problem, this series lets the check script create a

Re: [PATCH v2 1/2] nbd: Don't send oversize strings

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
11.10.2019 0:00, Eric Blake wrote: > Qemu as server currently won't accept export names larger than 256 > bytes, nor create dirty bitmap names longer than 1023 bytes, so most > uses of qemu as client or server have no reason to get anywhere near > the NBD spec maximum of a 4k limit per string. >

Re: [PATCH 06/23] iotests/083: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/083 | 6 +++--- > tests/qemu-iotests/083.out | 34 +- > 2 files changed, 20 insertions(+), 20 deletions(-) Reviewed-by: Thomas Huth

Re: [PATCH 10/23] iotests/181: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/181 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Thomas Huth

Re: [PATCH 09/23] iotests/147: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/147 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Thomas Huth

Re: [PATCH 11/23] iotests/182: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/182 | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Reviewed-by: Thomas Huth

Re: Problems with c8bb23cbdbe3 on ppc64le

2019-10-11 Thread Max Reitz
On 10.10.19 18:15, Anton Nefedov wrote: > On 10/10/2019 6:17 PM, Max Reitz wrote: >> Hi everyone, >> >> (CCs just based on tags in the commit in question) >> >> I have two bug reports which claim problems of qcow2 on XFS on ppc64le >> machines since qemu 4.1.0. One of those is about bad

Re: [PATCH 14/23] iotests/194: Create sockets in $SOCK_DIR

2019-10-11 Thread Max Reitz
On 10.10.19 21:43, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/qemu-iotests/194 | 4 ++-- >>   1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 >> index

Re: [PATCH 04/23] iotests: Filter $SOCK_DIR

2019-10-11 Thread Max Reitz
On 10.10.19 21:50, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/qemu-iotests/common.filter | 8 ++-- >>   1 file changed, 6 insertions(+), 2 deletions(-) >> > >> @@ -218,7 +221,8 @@ _filter_nbd() >>   # Filter out the TCP port

Re: [PATCH 04/23] iotests: Filter $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 11/10/2019 09.54, Max Reitz wrote: > On 10.10.19 20:42, Eric Blake wrote: >> On 10/10/19 10:24 AM, Max Reitz wrote: >>> Signed-off-by: Max Reitz >>> --- >>>   tests/qemu-iotests/common.filter | 8 ++-- >>>   1 file changed, 6 insertions(+), 2 deletions(-) >>> >>> diff --git

Re: [PATCH 00/23] iotests: Add and use $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 11/10/2019 10.03, Max Reitz wrote: > On 11.10.19 09:27, Thomas Huth wrote: >> On 10/10/2019 17.24, Max Reitz wrote: >>> Hi, >>> >>> Perhaps the main reason we cannot run important tests such as 041 in CI >>> is that when they care Unix sockets in $TEST_DIR, the path may become >>> too long to

CfP: Software Defined Storage devroom at FOSDEM 2020

2019-10-11 Thread Niels de Vos
FOSDEM is a free software event that offers open source communities a place to meet, share ideas and collaborate. It is renown for being highly developer- oriented and brings together 8000+ participants from all over the world. It is held in the city of Brussels (Belgium). FOSDEM 2020 will take

Re: [PATCH 3/4] block/mirror: support unaligned write in active mirror

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
04.10.2019 19:31, Max Reitz wrote: > On 12.09.19 17:13, Vladimir Sementsov-Ogievskiy wrote: >> Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up >> region in the dirty bitmap, which means that we may not copy some bytes >> and assume them copied, which actually leads to

Re: [Qemu-devel] [PULL 5/5] multifd: Use number of channels as listen backlog

2019-10-11 Thread Wei Yang
On Wed, Sep 04, 2019 at 08:29:15AM +0200, Juan Quintela wrote: >Reviewed-by: Daniel P. Berrangé >Signed-off-by: Juan Quintela >--- > migration/socket.c | 7 ++- > 1 file changed, 6 insertions(+), 1 deletion(-) > >diff --git a/migration/socket.c b/migration/socket.c >index

Re: [PATCH 07/23] iotests/140: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/140 | 8 > tests/qemu-iotests/140.out | 2 +- > 2 files changed, 5 insertions(+), 5 deletions(-) Reviewed-by: Thomas Huth

Re: [PATCH 08/23] iotests/143: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/143 | 6 +++--- > tests/qemu-iotests/143.out | 2 +- > 2 files changed, 4 insertions(+), 4 deletions(-) Reviewed-by: Thomas Huth

Re: [PATCH v2 2/2] nbd: Allow description when creating NBD blockdev

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
11.10.2019 0:00, Eric Blake wrote: > Allow blockdevs to match the feature already present in qemu-nbd -D. > Enhance iotest 223 to cover it. > > Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy > --- > qapi/block.json| 8 +--- > blockdev-nbd.c

Re: [PATCH 12/23] iotests/183: Create socket in $SOCK_DIR

2019-10-11 Thread Thomas Huth
On 10/10/2019 17.24, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/183 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Thomas Huth

Re: [PATCH 01/23] iotests: Introduce $SOCK_DIR

2019-10-11 Thread Max Reitz
On 10.10.19 20:18, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Unix sockets generally have a maximum path length.  Depending on your >> $TEST_DIR, it may be exceeded and then all tests that create and use >> Unix sockets there may fail. >> >> Circumvent this by adding a new

Re: [PATCH 04/23] iotests: Filter $SOCK_DIR

2019-10-11 Thread Max Reitz
On 10.10.19 20:42, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/qemu-iotests/common.filter | 8 ++-- >>   1 file changed, 6 insertions(+), 2 deletions(-) >> >> diff --git a/tests/qemu-iotests/common.filter >>

Re: [PATCH 00/23] iotests: Add and use $SOCK_DIR

2019-10-11 Thread Max Reitz
On 11.10.19 09:27, Thomas Huth wrote: > On 10/10/2019 17.24, Max Reitz wrote: >> Hi, >> >> Perhaps the main reason we cannot run important tests such as 041 in CI >> is that when they care Unix sockets in $TEST_DIR, the path may become >> too long to connect to them. >> >> To get by this problem,

Re: [PATCH 3/4] block/mirror: support unaligned write in active mirror

2019-10-11 Thread Max Reitz
On 11.10.19 10:33, Vladimir Sementsov-Ogievskiy wrote: > 04.10.2019 19:31, Max Reitz wrote: >> On 12.09.19 17:13, Vladimir Sementsov-Ogievskiy wrote: >>> Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up >>> region in the dirty bitmap, which means that we may not copy some

Re: [PATCH 00/23] iotests: Add and use $SOCK_DIR

2019-10-11 Thread Max Reitz
On 11.10.19 09:27, Thomas Huth wrote: > On 10/10/2019 17.24, Max Reitz wrote: >> Hi, >> >> Perhaps the main reason we cannot run important tests such as 041 in CI >> is that when they care Unix sockets in $TEST_DIR, the path may become >> too long to connect to them. >> >> To get by this problem,

[PATCH v2 5/5] Revert "mirror: Only mirror granularity-aligned chunks"

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
This reverts commit 9adc1cb49af8d4e54f57980b1eed5c0a4b2dafa6. "mirror: Only mirror granularity-aligned chunks" Since previous commit unaligned chunks are supported by do_sync_target_write. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/mirror.c | 29

[PATCH v2 1/5] hbitmap: handle set/reset with zero length

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Passing zero length to these functions leads to unpredicted results. Zero-length set/reset may occur in active-mirror, on zero-length write (which is unlikely, but not guaranteed to never happen). Let's just do nothing on zero-length request. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[PATCH v2 0/5] active-mirror: support unaligned guest operations

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Commit 9adc1cb49af8d fixed a bug about unaligned (to dirty bitmap granularity) guest writes (and discards) by simply requesting corresponding alignment on mirror-top filter. However forcing large alignment obviously decreases performance of unaligned requests. So it's time for a new solution

[PATCH] iotests/028: Fix for long $TEST_DIRs

2019-10-11 Thread Max Reitz
For long test image paths, the order of the "Formatting" line and the "(qemu)" prompt after a drive_backup HMP command may be reversed. It is not well-defined anyway, so just silence everything and wait until we get a prompt. Reported-by: Thomas Huth Signed-off-by: Max Reitz ---

[PATCH v2 2/5] block/mirror: simplify do_sync_target_write

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
do_sync_target_write is called from bdrv_mirror_top_do_write after write/discard operation, all inside active_write/active_write_settle protecting us from mirror iteration. So the whole area is dirty for sure, no reason to examine dirty bitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy

[PATCH v2 4/5] block/mirror: support unaligned write in active mirror

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up region in the dirty bitmap, which means that we may not copy some bytes and assume them copied, which actually leads to producing corrupted target. So 9adc1cb49af8d forced dirty bitmap granularity to be request_alignment for

[PATCH v2 3/5] block/block-backend: add blk_co_pwritev_part

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Add blk write function with qiov_offset parameter. It's needed for the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/sysemu/block-backend.h | 4 block/block-backend.c | 17 + 2 files changed, 17 insertions(+), 4

Re: [PATCH 3/4] block/mirror: support unaligned write in active mirror

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
11.10.2019 11:58, Max Reitz wrote: > On 11.10.19 10:33, Vladimir Sementsov-Ogievskiy wrote: >> 04.10.2019 19:31, Max Reitz wrote: >>> On 12.09.19 17:13, Vladimir Sementsov-Ogievskiy wrote: Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up region in the dirty bitmap,

Re: [PATCH 3/4] block/mirror: support unaligned write in active mirror

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
11.10.2019 11:58, Max Reitz wrote: > On 11.10.19 10:33, Vladimir Sementsov-Ogievskiy wrote: >> 04.10.2019 19:31, Max Reitz wrote: >>> On 12.09.19 17:13, Vladimir Sementsov-Ogievskiy wrote: Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up region in the dirty bitmap,

Re: [PATCH v2 1/1] IDE: deprecate ide-drive

2019-10-11 Thread Peter Krempa
On Thu, Oct 10, 2019 at 14:08:12 -0400, John Snow wrote: > On 10/10/19 7:54 AM, Peter Krempa wrote: > > On Thu, Oct 10, 2019 at 13:42:26 +0200, Philippe Mathieu-Daudé wrote: > >> On 10/10/19 1:26 PM, Peter Krempa wrote: [...] > > To be honest, I didn't really review the code nor the

Re: [PATCH] iotests/028: Fix for long $TEST_DIRs

2019-10-11 Thread Thomas Huth
On 11/10/2019 11.07, Max Reitz wrote: > For long test image paths, the order of the "Formatting" line and the > "(qemu)" prompt after a drive_backup HMP command may be reversed. It is > not well-defined anyway, so just silence everything and wait until we > get a prompt. > > Reported-by: Thomas

[PATCH 5/5] iotests: Enable more tests in the 'auto' group to improve test coverage

2019-10-11 Thread Thomas Huth
According to Kevin, tests 030, 040 and 041 are among the most valuable tests that we have, so we should always run them if possible, even if they take a little bit longer. According to Max, it would be good to have a test for iothreads and migration. 127 and 256 seem to be good candidates for

[PATCH v3 15/16] iotests: Add peek_file* functions

2019-10-11 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/common.rc | 20 1 file changed, 20 insertions(+) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 12b4751848..fa7bae2422 100644 --- a/tests/qemu-iotests/common.rc +++

[PATCH v3 13/16] qcow2: Repair snapshot table with too many entries

2019-10-11 Thread Max Reitz
The user cannot choose which snapshots are removed. This is fine because we have chosen the maximum snapshot table size to be so large (65536 entries) that it cannot be reasonably reached. If the snapshot table exceeds this size, the image has probably been corrupted in some way; in this case,

[RFC v5 011/126] block/snapshot: rename Error ** parameter to more common errp

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/snapshot.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/snapshot.c b/block/snapshot.c index 8081616ae9..bd9fb01817 100644 --- a/block/snapshot.c +++ b/block/snapshot.c @@ -426,7 +426,7 @@ fail: } int

[RFC v5 055/126] virtio-blk: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

Re: [PATCH v3 07/16] qcow2: Write v3-compliant snapshot list on upgrade

2019-10-11 Thread Eric Blake
On 10/11/19 10:28 AM, Max Reitz wrote: qcow2 v3 requires every snapshot table entry to have two extra data fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present. qcow2_upgrade() therefore should update the snapshot table to

[RFC v5 094/126] RBD: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 025/126] scripts: add coccinelle script to use auto propagated errp

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- CC: Gerd Hoffmann CC: "Gonglei (Arei)" CC: Eduardo Habkost CC: Igor Mammedov CC: Laurent Vivier CC: Amit Shah CC: Kevin Wolf CC: Max Reitz CC: John Snow CC: Ari Sundholm CC: Pavel Dovgalyuk CC: Paolo Bonzini CC: Stefan Hajnoczi CC: Fam

[RFC v5 026/126] python: add commit-per-subsystem.py

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Add script to automatically commit tree-wide changes per-subsystem. Signed-off-by: Vladimir Sementsov-Ogievskiy --- CC: Gerd Hoffmann CC: "Gonglei (Arei)" CC: Eduardo Habkost CC: Igor Mammedov CC: Laurent Vivier CC: Amit Shah CC: Kevin Wolf CC: Max Reitz CC: John Snow CC: Ari Sundholm

Re: [RFC v5 099/126] nbd: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Eric Blake
On 10/11/19 11:05 AM, Vladimir Sementsov-Ogievskiy wrote: If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint()

Re: [PATCH] qcow2-bitmaps: fix qcow2_can_store_new_dirty_bitmap

2019-10-11 Thread John Snow
On 10/11/19 6:02 AM, Vladimir Sementsov-Ogievskiy wrote: > 10.10.2019 21:46, John Snow wrote: >> >> >> On 10/10/19 11:39 AM, Eric Blake wrote: >>> On 6/7/19 1:53 PM, Vladimir Sementsov-Ogievskiy wrote: 07.06.2019 21:48, Vladimir Sementsov-Ogievskiy wrote: >

Re: [RFC v5 007/126] nbd: well form nbd_iter_channel_error errp handler

2019-10-11 Thread Eric Blake
On 10/11/19 11:03 AM, Vladimir Sementsov-Ogievskiy wrote: Make nbd_iter_channel_error errp handler well formed: rename errp to errp_in, as it is IN-parameter here (which is unusual Actually, rename local_err to errp_in. for errp). Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [RFC v5 000/126] error: auto propagated local_err

2019-10-11 Thread Eric Blake
On 10/11/19 11:03 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all! At the request of Markus: full version of errp propagation. Let's look at it. Cover as much as possible, except inserting macro invocation where it's not necessary. It's huge, and so it's an RFC. Is there a repo containing

Re: [RFC v5 067/126] block: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Eric Blake
On 10/11/19 11:04 AM, Vladimir Sementsov-Ogievskiy wrote: If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint()

[PULL 11/19] iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Reopening bitmaps to RW was broken prior to previous commit. Check that it works now. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190927122355.7344-4-vsement...@virtuozzo.com Signed-off-by: John Snow --- tests/qemu-iotests/165 | 57

[PULL 19/19] dirty-bitmaps: remove deprecated autoload parameter

2019-10-11 Thread John Snow
This parameter has been deprecated since 2.12.0 and is eligible for removal. Remove this parameter as it is actually completely ignored; let's not give false hope. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id:

Re: [PATCH 4/4] qemu-nbd: Support help options for --object

2019-10-11 Thread Eric Blake
On 10/11/19 3:55 PM, Kevin Wolf wrote: Instead of parsing help options as normal object properties and returning an error, provide the same help functionality as the system emulator in qemu-nbd, too. Signed-off-by: Kevin Wolf --- qemu-nbd.c | 9 - 1 file changed, 8 insertions(+), 1

[RFC v5 067/126] block: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

Re: [RFC v5 019/126] include/block/snapshot.h: rename Error ** parameter to more common errp

2019-10-11 Thread Eric Blake
On 10/11/19 11:04 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/snapshot.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Eric Blake diff --git a/include/block/snapshot.h b/include/block/snapshot.h index

Re: [PATCH 1/4] vl: Split off user_creatable_print_help()

2019-10-11 Thread Eric Blake
On 10/11/19 3:55 PM, Kevin Wolf wrote: Printing help for --object is something that we don't only want in the s/don't/not/ system emulator, but also in tools that support --object. Move it into a separate function in qom/object_interfaces.c to make the code accessible for tools.

Re: [PATCH 3/4] qemu-img: Support help options for --object

2019-10-11 Thread Eric Blake
On 10/11/19 3:55 PM, Kevin Wolf wrote: Instead of parsing help options as normal object properties and returning an error, provide the same help functionality as the system emulator in qemu-img, too. Signed-off-by: Kevin Wolf --- qemu-img.c | 34 +- 1 file

[RFC v5 102/126] CURL: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 116/126] vvfat: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

Re: [RFC v5 025/126] scripts: add coccinelle script to use auto propagated errp

2019-10-11 Thread Eric Blake
On 10/11/19 12:12 PM, Eric Blake wrote: On 10/11/19 11:04 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- +@@ +identifier rule1.fn; +identifier rule1.local_err; +@@ + + fn(...) + { + <... +( +-    error_free(local_err); +-    local_err = NULL; ++ 

[RFC v5 117/126] Replication: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

Re: Occasional VM soft lockup when a remote cdrom is attached

2019-10-11 Thread John Snow
On 10/11/19 9:22 AM, Guoheyi wrote: > Hi folks, > > We observed Linux on VM occasionally (at very low rate) got soft lockup > when a remote cdrom is attached. The guest hangs up at below call trace: > That's certainly a new one to me :) > [Tue Oct8 23:02:53 2019]ata_scsi_queuecmd+0xe0/0x2a0

Re: [PATCH v2 1/4] qemu-iotests: remove bash shebang from library files

2019-10-11 Thread Cleber Rosa
On Fri, Oct 11, 2019 at 02:27:25PM +0300, Nir Soffer wrote: > On Fri, Oct 11, 2019, 12:36 Kevin Wolf wrote: > > > Am 09.10.2019 um 21:47 hat Cleber Rosa geschrieben: > > > Due to not being able to find a reason to have shebangs on files that > > > are not executable. > > > > > > While at it, add

Re: [PATCH v2 1/1] IDE: deprecate ide-drive

2019-10-11 Thread John Snow
On 10/11/19 5:12 AM, Peter Krempa wrote: > On Thu, Oct 10, 2019 at 14:08:12 -0400, John Snow wrote: >> On 10/10/19 7:54 AM, Peter Krempa wrote: >>> On Thu, Oct 10, 2019 at 13:42:26 +0200, Philippe Mathieu-Daudé wrote: On 10/10/19 1:26 PM, Peter Krempa wrote: > > [...] > >>> To be honest,

[PULL 01/19] util/hbitmap: strict hbitmap_reset

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy hbitmap_reset has an unobvious property: it rounds requested region up. It may provoke bugs, like in recently fixed write-blocking mode of mirror: user calls reset on unaligned region, not keeping in mind that there are possible unrelated dirty bytes, covered

[PULL 02/19] block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy block/dirty-bitmap.c seems to be more appropriate for it and bdrv_remove_persistent_dirty_bitmap already in it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190920082543.23444-2-vsement...@virtuozzo.com Signed-off-by: John

[PULL 04/19] block/qcow2: proper locking on bitmap add/remove paths

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy qmp_block_dirty_bitmap_add and do_block_dirty_bitmap_remove do acquire aio context since 0a6c86d024c52b. But this is not enough: we also must lock qcow2 mutex when access in-image metadata. Especially it concerns freeing qcow2 clusters. To achieve this, move

[PULL 00/19] Bitmaps patches

2019-10-11 Thread John Snow
The following changes since commit 98b2e3c9ab3abfe476a2b02f8f51813edb90e72d: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2019-10-08 16:08:35 +0100) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request

[PULL 17/19] qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy The only reason I can imagine for this strange code at the very-end of bdrv_reopen_commit is the fact that bs->read_only updated after calling drv->bdrv_reopen_commit in bdrv_reopen_commit. And in the same time, prior to previous commit, qcow2_reopen_bitmaps_rw

[PULL 14/19] block/qcow2-bitmap: do not remove bitmaps on reopen-ro

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy qcow2_reopen_bitmaps_ro wants to store bitmaps and then mark them all readonly. But the latter don't work, as qcow2_store_persistent_dirty_bitmaps removes bitmaps after storing. It's OK for inactivation but bad idea for reopen-ro. And this leads to the

[PULL 16/19] block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy - Correct check for write access to file child, and in correct place (only if we want to write). - Support reopen rw -> rw (which will be used in following commit), for example, !bdrv_dirty_bitmap_readonly() is not a corruption if bitmap is marked IN_USE

[PULL 15/19] iotests: add test 260 to check bitmap life after snapshot + commit

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190927122355.7344-8-vsement...@virtuozzo.com Signed-off-by: John Snow --- tests/qemu-iotests/260 | 89 ++ tests/qemu-iotests/260.out | 52 ++

[PULL 10/19] block: reverse order for reopen commits

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy It's needed to fix reopening qcow2 with bitmaps to RW. Currently it can't work, as qcow2 needs write access to file child, to mark bitmaps in-image with IN_USE flag. But usually children goes after parents in reopen queue and file child is still RO on qcow2

[PULL 18/19] MAINTAINERS: Add Vladimir as a reviewer for bitmaps

2019-10-11 Thread John Snow
I already try to make sure all bitmaps patches have been reviewed by both Red Hat and Virtuozzo anyway, so this formalizes the arrangement. Fam meanwhile is no longer as active, so I am removing him as a co-maintainer simply to reflect the current practice. Signed-off-by: John Snow Reviewed-by:

[PULL 08/19] block/dirty-bitmap: refactor bdrv_dirty_bitmap_next

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_next is always used in same pattern. So, split it into _next and _first, instead of combining two functions into one and add FOR_EACH_DIRTY_BITMAP macro. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id:

Re: [PULL 01/19] util/hbitmap: strict hbitmap_reset

2019-10-11 Thread Eric Blake
On 10/11/19 4:25 PM, John Snow wrote: From: Vladimir Sementsov-Ogievskiy hbitmap_reset has an unobvious property: it rounds requested region up. It may provoke bugs, like in recently fixed write-blocking mode of mirror: user calls reset on unaligned region, not keeping in mind that there are

[RFC v5 110/126] qed: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 112/126] qcow2: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

Re: [RFC v5 025/126] scripts: add coccinelle script to use auto propagated errp

2019-10-11 Thread Eric Blake
On 10/11/19 11:04 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/coccinelle/auto-propagated-errp.cocci | 118 ++ 1 file changed, 118 insertions(+) create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci diff

[PATCH 4/4] qemu-nbd: Support help options for --object

2019-10-11 Thread Kevin Wolf
Instead of parsing help options as normal object properties and returning an error, provide the same help functionality as the system emulator in qemu-nbd, too. Signed-off-by: Kevin Wolf --- qemu-nbd.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/qemu-nbd.c

[PATCH 1/4] vl: Split off user_creatable_print_help()

2019-10-11 Thread Kevin Wolf
Printing help for --object is something that we don't only want in the system emulator, but also in tools that support --object. Move it into a separate function in qom/object_interfaces.c to make the code accessible for tools. Signed-off-by: Kevin Wolf --- include/qom/object_interfaces.h | 12

[PATCH 0/4] qemu-img/io/nbd: Support help options for --object

2019-10-11 Thread Kevin Wolf
This series fixes that the tools don't consider help options for --object and just treat them as regular object properties. For example: $ ./qemu-io --object secret,help qemu-io: Parameter 'id' is missing With these patches, we get the expected behaviour like in the system emulator.

[PATCH 2/4] qemu-io: Support help options for --object

2019-10-11 Thread Kevin Wolf
Instead of parsing help options as normal object properties and returning an error, provide the same help functionality as the system emulator in qemu-io, too. Signed-off-by: Kevin Wolf --- qemu-io.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/qemu-io.c

[PATCH 3/4] qemu-img: Support help options for --object

2019-10-11 Thread Kevin Wolf
Instead of parsing help options as normal object properties and returning an error, provide the same help functionality as the system emulator in qemu-img, too. Signed-off-by: Kevin Wolf --- qemu-img.c | 34 +- 1 file changed, 21 insertions(+), 13 deletions(-)

[PULL 03/19] block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy It's more comfortable to not deal with local_err. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190920082543.23444-3-vsement...@virtuozzo.com Signed-off-by: John Snow --- block/qcow2.h| 5 ++---

[PULL 09/19] block: switch reopen queue from QSIMPLEQ to QTAILQ

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy We'll need reverse-foreach in the following commit, QTAILQ support it, so move to QTAILQ. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20190927122355.7344-2-vsement...@virtuozzo.com Signed-off-by: John Snow ---

Re: [PATCH 2/4] qemu-io: Support help options for --object

2019-10-11 Thread Eric Blake
On 10/11/19 3:55 PM, Kevin Wolf wrote: Instead of parsing help options as normal object properties and returning an error, provide the same help functionality as the system emulator in qemu-io, too. Signed-off-by: Kevin Wolf --- qemu-io.c | 9 - 1 file changed, 8 insertions(+), 1

[RFC v5 092/126] Record/replay: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 115/126] vpc: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 096/126] VHDX: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 093/126] VMDK: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

[RFC v5 104/126] NVMe Block Driver: introduce ERRP_AUTO_PROPAGATE

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == _err (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want to

RE: [PATCH v6 4/4] colo: Update Documentation for continuous replication

2019-10-11 Thread Zhang, Chen
> -Original Message- > From: Lukas Straub > Sent: Saturday, October 12, 2019 12:01 AM > To: Zhang, Chen > Cc: qemu-devel ; Jason Wang > ; Wen Congyang ; > Xie Changlong ; Kevin Wolf > ; Max Reitz ; qemu-block > > Subject: Re: [PATCH v6 4/4] colo: Update Documentation for continuous >

Re: [RFC v5 011/126] block/snapshot: rename Error ** parameter to more common errp

2019-10-11 Thread Eric Blake
On 10/11/19 11:03 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/snapshot.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) Reviewed-by: Eric Blake diff --git a/block/snapshot.c b/block/snapshot.c index 8081616ae9..bd9fb01817

Re: [PATCH v2 1/4] qemu-iotests: remove bash shebang from library files

2019-10-11 Thread Kevin Wolf
Am 11.10.2019 um 22:05 hat Cleber Rosa geschrieben: > On Fri, Oct 11, 2019 at 02:27:25PM +0300, Nir Soffer wrote: > > On Fri, Oct 11, 2019, 12:36 Kevin Wolf wrote: > > > > > Am 09.10.2019 um 21:47 hat Cleber Rosa geschrieben: > > > > Due to not being able to find a reason to have shebangs on

[PULL 07/19] block/dirty-bitmap: drop BdrvDirtyBitmap.mutex

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy mutex field is just a pointer to bs->dirty_bitmap_mutex, so no needs to store it in BdrvDirtyBitmap when we have bs pointer in it (since previous patch). Drop mutex field. Constantly use bdrv_dirty_bitmaps_lock/unlock in block/dirty-bitmap.c to make it more

[PULL 06/19] block/dirty-bitmap: add bs link

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Add bs field to BdrvDirtyBitmap structure. Drop BlockDriverState parameter from bitmap APIs where possible. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190916141911.5255-3-vsement...@virtuozzo.com Signed-off-by: John Snow

[PULL 13/19] block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint()

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy The function is unused, drop it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190927122355.7344-6-vsement...@virtuozzo.com Signed-off-by: John Snow --- block/qcow2.h| 2 -- block/qcow2-bitmap.c | 15

[PULL 12/19] block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Firstly, no reason to optimize failure path. Then, function name is ambiguous: it checks for readonly and similar things, but someone may think that it will ignore normal bitmaps which was just unchanged, and this is in bad relation with the fact that we should

[PULL 05/19] block/dirty-bitmap: drop meta

2019-10-11 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Drop meta bitmaps, as they are unused. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190916141911.5255-2-vsement...@virtuozzo.com Signed-off-by: John Snow --- include/block/dirty-bitmap.h | 5 block/dirty-bitmap.c

[RFC v5 007/126] nbd: well form nbd_iter_channel_error errp handler

2019-10-11 Thread Vladimir Sementsov-Ogievskiy
Make nbd_iter_channel_error errp handler well formed: rename errp to errp_in, as it is IN-parameter here (which is unusual for errp). Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block/nbd.c

  1   2   >