Re: [PATCH] iotests/118: Drop 'change' test

2021-01-26 Thread Max Reitz
On 26.01.21 12:18, Kevin Wolf wrote: Am 26.01.2021 um 11:48 hat Max Reitz geschrieben: Commit 0afec75734331 removed the 'change' QMP command, so we can no longer test it in 118. Fixes: 0afec75734331a0b52fa3aa4235220eda8c7846f ('qmp: remove deprecated "change" command')

[PATCH] iotests/118: Drop 'change' test

2021-01-26 Thread Max Reitz
Commit 0afec75734331 removed the 'change' QMP command, so we can no longer test it in 118. Fixes: 0afec75734331a0b52fa3aa4235220eda8c7846f ('qmp: remove deprecated "change" command') Signed-off-by: Max Reitz --- tests/qemu-iotests/118 | 17 - tests/qemu-i

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 20.01.21 17:04, Daniel P. Berrangé wrote: On Wed, Jan 20, 2021 at 04:53:26PM +0100, Max Reitz wrote: On 20.01.21 15:44, Max Reitz wrote: On 20.01.21 15:34, Max Reitz wrote: [...]  From a glance, it looks to me like two coroutines are created simultaneously in two threads, and so one

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 20.01.21 16:53, Max Reitz wrote: On 20.01.21 15:44, Max Reitz wrote: On 20.01.21 15:34, Max Reitz wrote: [...]  From a glance, it looks to me like two coroutines are created simultaneously in two threads, and so one thread sets up a special SIGUSR2 action, then another reverts SIGUSR2

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 20.01.21 15:44, Max Reitz wrote: On 20.01.21 15:34, Max Reitz wrote: [...]  From a glance, it looks to me like two coroutines are created simultaneously in two threads, and so one thread sets up a special SIGUSR2 action, then another reverts SIGUSR2 to the default, and then the first

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 20.01.21 15:34, Max Reitz wrote: On 20.01.21 14:50, Max Reitz wrote: On 20.01.21 11:39, Max Reitz wrote: On 19.01.21 20:22, Vladimir Sementsov-Ogievskiy wrote: 19.01.2021 21:40, Max Reitz wrote: On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: Hi Max! I applied my series onto yours

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 20.01.21 14:50, Max Reitz wrote: On 20.01.21 11:39, Max Reitz wrote: On 19.01.21 20:22, Vladimir Sementsov-Ogievskiy wrote: 19.01.2021 21:40, Max Reitz wrote: On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: Hi Max! I applied my series onto yours 129-fixing and found, that 129 fails

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 20.01.21 11:39, Max Reitz wrote: On 19.01.21 20:22, Vladimir Sementsov-Ogievskiy wrote: 19.01.2021 21:40, Max Reitz wrote: On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: Hi Max! I applied my series onto yours 129-fixing and found, that 129 fails for backup. And setting small max

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-20 Thread Max Reitz
On 19.01.21 20:22, Vladimir Sementsov-Ogievskiy wrote: 19.01.2021 21:40, Max Reitz wrote: On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: Hi Max! I applied my series onto yours 129-fixing and found, that 129 fails for backup. And setting small max-chunk and even max-workers to 1

[PATCH v4 11.5/23] iotests/129: Limit backup's max-chunk/max-workers

2021-01-20 Thread Max Reitz
Right now, this does not change anything, because backup ignores max-chunk and max-workers. However, as soon as backup is switched over to block-copy for the background copying process, we will need it to keep 129 passing. Signed-off-by: Max Reitz --- Hi Vladimir, would you be OK

Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter

2021-01-20 Thread Max Reitz
On 20.01.21 07:09, Vladimir Sementsov-Ogievskiy wrote: 19.01.2021 23:21, Eric Blake wrote: On 10/26/20 12:17 PM, Vladimir Sementsov-Ogievskiy wrote: Experiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-19 Thread Max Reitz
On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: Hi Max! I applied my series onto yours 129-fixing and found, that 129 fails for backup. And setting small max-chunk and even max-workers to 1 doesn't help! (setting speed like in v3 still helps). And I found, that the problem is that

Re: [PATCH 2/2] virtio-scsi-test: Test writing to scsi-cd device

2021-01-19 Thread Max Reitz
/qtest/virtio-scsi-test.c | 39 ++ 1 file changed, 39 insertions(+) Reviewed-by: Max Reitz

Re: [PATCH 1/2] block: Separate blk_is_writable() and blk_supports_write_perm()

2021-01-19 Thread Max Reitz
| 2 +- hw/scsi/scsi-disk.c| 10 +- hw/scsi/scsi-generic.c | 4 ++-- hw/sd/sd.c | 6 +++--- hw/usb/dev-storage.c | 4 ++-- 20 files changed, 57 insertions(+), 41 deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v2] iotests: Add test for the regression fixed in c8bf9a9169

2021-01-19 Thread Max Reitz
On 12.01.21 18:05, Alberto Garcia wrote: Signed-off-by: Alberto Garcia Suggested-by: Maxim Levitsky Reviewed-by: Maxim Levitsky --- v2: Rebase on top of the latest master tests/qemu-iotests/313 | 103 + tests/qemu-iotests/313.out | 29 +++

Re: [PATCH v5] block: report errno when flock fcntl fails

2021-01-18 Thread Max Reitz
On 13.01.21 17:44, David Edmondson wrote: When a call to fcntl(2) for the purpose of adding file locks fails with an error other than EAGAIN or EACCES, report the error returned by fcntl. EAGAIN or EACCES are elided as they are considered to be common failures, indicating that a conflicting

Re: [PATCH v4 00/23] backup performance: block_status + async

2021-01-18 Thread Max Reitz
On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: Hi Max! I applied my series onto yours 129-fixing and found, that 129 fails for backup. And setting small max-chunk and even max-workers to 1 doesn't help! (setting speed like in v3 still helps). And I found, that the problem is that

Re: [PATCH v4 09/23] job: call job_enter from job_pause

2021-01-18 Thread Max Reitz
On 18.01.21 14:45, Max Reitz wrote: On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: If main job coroutine called job_yield (while some background process is in progress), we should give it a chance to call job_pause_point(). It will be used in backup, when moved on async block-copy

Re: [PATCH v4 23/23] simplebench: add bench-backup.py

2021-01-18 Thread Max Reitz
On 16.01.21 22:47, Vladimir Sementsov-Ogievskiy wrote: Add script to benchmark new backup architecture. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/simplebench/bench-backup.py | 167 1 file changed, 167 insertions(+) create mode 100755

Re: [PATCH v4 09/23] job: call job_enter from job_pause

2021-01-18 Thread Max Reitz
On 16.01.21 22:46, Vladimir Sementsov-Ogievskiy wrote: If main job coroutine called job_yield (while some background process is in progress), we should give it a chance to call job_pause_point(). It will be used in backup, when moved on async block-copy. Note, that job_user_pause is not enough:

[PATCH v5 09/10] iotests/129: Clean up pylint and mypy complaints

2021-01-18 Thread Max Reitz
And consequentially drop it from 297's skip list. Signed-off-by: Max Reitz Reviewed-by: Willian Rampazzo Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 4 ++-- tests/qemu-iotests/297 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu

[PATCH v5 08/10] iotests/129: Limit mirror job's buffer size

2021-01-18 Thread Max Reitz
the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH v5 10/10] iotests/300: Clean up pylint and mypy complaints

2021-01-18 Thread Max Reitz
And consequentially drop it from 297's skip list. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/300 | 18 +++--- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git

[PATCH v5 05/10] iotests/129: Do not check @busy

2021-01-18 Thread Max Reitz
field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH v5 02/10] iotests/297: Rewrite in Python and extend reach

2021-01-18 Thread Max Reitz
y's output is suppressed, because (A) it does not add useful information, and (B) it would leak information about the files having been tested to the reference output, which we decidedly do not want. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- tests/qemu-iotests/297

[PATCH v5 07/10] iotests/129: Actually test a commit job

2021-01-18 Thread Max Reitz
(and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 27 +-- 1 file changed, 25 insertions(+), 2 deletions

[PATCH v5 06/10] iotests/129: Use throttle node

2021-01-18 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy

[PATCH v5 00/10] iotests: Fix 129 and expand 297’s reach

2021-01-18 Thread Max Reitz
buffer size' 009/10:[] [--] 'iotests/129: Clean up pylint and mypy complaints' 010/10:[] [--] 'iotests/300: Clean up pylint and mypy complaints' Max Reitz (10): iotests.py: Assume a couple of variables as given iotests/297: Rewrite in Python and extend reach iotests: Move try_

[PATCH v5 04/10] iotests/129: Remove test images in tearDown()

2021-01-18 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755

[PATCH v5 01/10] iotests.py: Assume a couple of variables as given

2021-01-18 Thread Max Reitz
it and move the abort code from execute_setup_common() there. Drop the 'assert iotests.sock_dir is not None' from iotest 300, because that sort of thing is precisely what this patch wants to prevent. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo

[PATCH v5 03/10] iotests: Move try_remove to iotests.py

2021-01-18 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu

Re: [PATCH v4 02/10] iotests/297: Rewrite in Python and extend reach

2021-01-18 Thread Max Reitz
On 15.01.21 20:27, Willian Rampazzo wrote: On Fri, Jan 15, 2021 at 2:43 PM Max Reitz wrote: Instead of checking iotests.py only, check all Python files in the qemu-iotests/ directory. Of course, most of them do not pass, so there is an extensive skip list for now. (The only files that do

[PATCH v4 10/10] iotests/300: Clean up pylint and mypy complaints

2021-01-15 Thread Max Reitz
And consequentially drop it from 297's skip list. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/300 | 18 +++--- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b

[PATCH v4 08/10] iotests/129: Limit mirror job's buffer size

2021-01-15 Thread Max Reitz
the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH v4 05/10] iotests/129: Do not check @busy

2021-01-15 Thread Max Reitz
field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b

[PATCH v4 07/10] iotests/129: Actually test a commit job

2021-01-15 Thread Max Reitz
(and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 27 +-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/tests/qemu

[PATCH v4 09/10] iotests/129: Clean up pylint and mypy complaints

2021-01-15 Thread Max Reitz
And consequentially drop it from 297's skip list. Signed-off-by: Max Reitz Reviewed-by: Willian Rampazzo Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 4 ++-- tests/qemu-iotests/297 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu

[PATCH v4 00/10] iotests: Fix 129 and expand 297’s reach

2021-01-15 Thread Max Reitz
009/10:[] [--] 'iotests/129: Clean up pylint and mypy complaints' 010/10:[0006] [FC] 'iotests/300: Clean up pylint and mypy complaints' Max Reitz (10): iotests.py: Assume a couple of variables as given iotests/297: Rewrite in Python and extend reach iotests: Move try_remove to iotests.py iotes

[PATCH v4 06/10] iotests/129: Use throttle node

2021-01-15 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy

[PATCH v4 04/10] iotests/129: Remove test images in tearDown()

2021-01-15 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755

[PATCH v4 03/10] iotests: Move try_remove to iotests.py

2021-01-15 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu

[PATCH v4 01/10] iotests.py: Assume a couple of variables as given

2021-01-15 Thread Max Reitz
it and move the abort code from execute_setup_common() there. Drop the 'assert iotests.sock_dir is not None' from iotest 300, because that sort of thing is precisely what this patch wants to prevent. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/300

[PATCH v4 02/10] iotests/297: Rewrite in Python and extend reach

2021-01-15 Thread Max Reitz
y's output is suppressed, because (A) it does not add useful information, and (B) it would leak information about the files having been tested to the reference output, which we decidedly do not want. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz Reviewed-by: Vladimir

Re: [PATCH v3 11/10] iotests: add flake8 linter

2021-01-15 Thread Max Reitz
On 15.01.21 12:53, Vladimir Sementsov-Ogievskiy wrote: pylint is good, but doesn't cover the PEP8. Let's add flake8, to be sure that our code sutisfy PEP8. Add new linter and fix some code style in checked files. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi! Here is my small addition

Re: [PATCH v3 10/10] iotests/300: Clean up pylint and mypy complaints

2021-01-15 Thread Max Reitz
On 15.01.21 12:30, Vladimir Sementsov-Ogievskiy wrote: 14.01.2021 20:03, Max Reitz wrote: Signed-off-by: Max Reitz ---   tests/qemu-iotests/297 |  2 +-   tests/qemu-iotests/300 | 18 +++---   2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b

Re: [PATCH v3 06/10] iotests/129: Use throttle node

2021-01-15 Thread Max Reitz
On 15.01.21 12:16, Vladimir Sementsov-Ogievskiy wrote: 14.01.2021 20:03, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node

Re: [PATCH v3 02/10] iotests/297: Rewrite in Python and extend reach

2021-01-15 Thread Max Reitz
On 15.01.21 12:13, Vladimir Sementsov-Ogievskiy wrote: 14.01.2021 20:02, Max Reitz wrote: Instead of checking iotests.py only, check all Python files in the qemu-iotests/ directory.  Of course, most of them do not pass, so there is an extensive skip list for now.  (The only files that do pass

Re: [PATCH v3 09/10] iotests/129: Clean up pylint and mypy complaints

2021-01-15 Thread Max Reitz
On 14.01.21 21:02, Willian Rampazzo wrote: On Thu, Jan 14, 2021 at 2:41 PM Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 4 ++-- tests/qemu-iotests/297 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu

[PATCH v3 08/10] iotests/129: Limit mirror job's buffer size

2021-01-14 Thread Max Reitz
the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests

[PATCH v3 07/10] iotests/129: Actually test a commit job

2021-01-14 Thread Max Reitz
(and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 27 +-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/tests/qemu

[PATCH v3 06/10] iotests/129: Use throttle node

2021-01-14 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/129 | 37

[PATCH v3 00/10] iotests: Fix 129 and expand 297’s reach

2021-01-14 Thread Max Reitz
mirror job's buffer size' 009/10:[0007] [FC] 'iotests/129: Clean up pylint and mypy complaints' 010/10:[down] 'iotests/300: Clean up pylint and mypy complaints' Max Reitz (10): iotests.py: Assume a couple of variables as given iotests/297: Rewrite in Python and extend reach iotests: Move

[PATCH v3 09/10] iotests/129: Clean up pylint and mypy complaints

2021-01-14 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 4 ++-- tests/qemu-iotests/297 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 6d21470cd7..201d9e0a0b 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests

[PATCH v3 10/10] iotests/300: Clean up pylint and mypy complaints

2021-01-14 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/300 | 18 +++--- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 1dce1d1b1c..03d8604538 100755 --- a/tests/qemu-iotests/297 +++ b

[PATCH v3 04/10] iotests/129: Remove test images in tearDown()

2021-01-14 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++ b

[PATCH v3 01/10] iotests.py: Assume a couple of variables as given

2021-01-14 Thread Max Reitz
it and move the abort code from execute_setup_common() there. Drop the 'assert iotests.sock_dir is not None' from iotest 300, because that sort of thing is precisely what this patch wants to prevent. Signed-off-by: Max Reitz --- tests/qemu-iotests/300| 1 - tests/qemu-iotests/iotests.py

[PATCH v3 05/10] iotests/129: Do not check @busy

2021-01-14 Thread Max Reitz
field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b

[PATCH v3 03/10] iotests: Move try_remove to iotests.py

2021-01-14 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755

[PATCH v3 02/10] iotests/297: Rewrite in Python and extend reach

2021-01-14 Thread Max Reitz
y's output is suppressed, because (A) it does not add useful information, and (B) it would leak information about the files having been tested to the reference output, which we decidedly do not want. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz --- tests/qemu-iotests/297

Re: [PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-14 Thread Max Reitz
On 13.01.21 21:28, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 22:27, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 20:57, Max Reitz wrote: I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now.  (The only files

Re: [PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-14 Thread Max Reitz
On 13.01.21 20:27, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 20:57, Max Reitz wrote: I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now.  (The only files that do pass are 209, 254, 283, and iotests.py

Re: [PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-14 Thread Max Reitz
On 13.01.21 20:01, Eric Blake wrote: On 1/13/21 11:57 AM, Max Reitz wrote: I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now. (The only files that do pass are 209, 254, 283, and iotests.py.) (Alternatively, we

[PATCH v2 4/8] iotests/129: Do not check @busy

2021-01-13 Thread Max Reitz
field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b

[PATCH v2 8/8] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 6 -- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/297.out | 1 + 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 6d21470cd7..64578493c1 100755 --- a/tests

[PATCH v2 5/8] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 37

[PATCH v2 7/8] iotests/129: Limit mirror job's buffer size

2021-01-13 Thread Max Reitz
the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests

[PATCH v2 6/8] iotests/129: Actually test a commit job

2021-01-13 Thread Max Reitz
(and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 27 +-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/tests/qemu

[PATCH v2 3/8] iotests/129: Remove test images in tearDown()

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++ b

[PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-13 Thread Max Reitz
hand, I decided to hide mypy's "Success" lines from the reference output, because they do not add anything useful. Signed-off-by: Max Reitz --- tests/qemu-iotests/297 | 66 ++ tests/qemu-iotests/297.out | 6 +++- 2 files changed, 65 insert

[PATCH v2 2/8] iotests: Move try_remove to iotests.py

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755

[PATCH v2 0/8] iotests: Fix 129 and expand 297’s reach

2021-01-13 Thread Max Reitz
/129: Limit mirror job's buffer size' 008/8:[0003] [FC] 'iotests/129: Clean up pylint and mypy complaints' Max Reitz (8): iotests/297: Allow checking all Python test files iotests: Move try_remove to iotests.py iotests/129: Remove test images in tearDown() iotests/129: Do not check @busy

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
On 13.01.21 17:46, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Max Reitz
On 13.01.21 15:31, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:06, Max Reitz wrote: Hi, There are some problems with iotests 129 (perhaps more than these, but these are the ones I know of): 1. It checks @busy to see whether a block job is still running; however,     block jobs tend

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Max Reitz
On 13.01.21 15:38, Kevin Wolf wrote: Am 13.01.2021 um 15:06 hat Max Reitz geschrieben: - pylint and mypy complain. (Running mypy with the options given in 297.) [Patch 4 removes one pylint complaint; patch 7 the rest.] Should we add it to 297 then to make sure we won't regress? Sounds

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
On 13.01.21 16:07, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:10, Max Reitz wrote: On 13.01.21 15:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running

[PATCH 3/7] iotests/129: Do not check @busy

2021-01-13 Thread Max Reitz
field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 2fc65ada6a..dd23bb2e5a 100755 --- a/tests

[PATCH 5/7] iotests/129: Actually test a commit job

2021-01-13 Thread Max Reitz
(and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 27 +-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index febc806398..e6753c90ef

[PATCH 7/7] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index c3ad584ba2..ec303069e9 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -20,9 +20,10 @@ import

[PATCH 6/7] iotests/129: Limit mirror job's buffer size

2021-01-13 Thread Max Reitz
chunk, too.) Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index e6753c90ef..c3ad584ba2 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -69,7 +69,7 @@ class

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
On 13.01.21 15:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 39

[PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 39

[PATCH 2/7] iotests/129: Remove test images in tearDown()

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -47,6 +47,8 @@ class TestStopWithBlockJob

[PATCH 1/7] iotests: Move try_remove to iotests.py

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755 --- a/tests/qemu-iotests

[PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Max Reitz
] - pylint and mypy complain. (Running mypy with the options given in 297.) [Patch 4 removes one pylint complaint; patch 7 the rest.] Max Reitz (7): iotests: Move try_remove to iotests.py iotests/129: Remove test images in tearDown() iotests/129: Do not check @busy iotests/129: Use throttle

Re: iotest 129

2021-01-13 Thread Max Reitz
On 13.01.21 10:53, Vladimir Sementsov-Ogievskiy wrote: 12.01.2021 20:44, Max Reitz wrote: Hi, tl;dr: I have some troubles debugging what’s wrong with iotest 129. It wants to check that 'stop' does not drain a block job, but to me it seems like that’s exactly what’s happening with the mirror

iotest 129

2021-01-12 Thread Max Reitz
filter-node-show-progress.patch for just test_drive_mirror(), but that only works back until fe4f0614ef9e361d (or rather 0f12264e7a4145 if you prefer not to get a SIGABRT). Before that commit, it hangs.) Max commit 9697ee270fc59a7a9e5b6b4a56373e38f2e50327 Author: Max Reitz Date: Tue Jan 12

Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default

2021-01-12 Thread Max Reitz
On 12.01.21 16:44, Vladimir Sementsov-Ogievskiy wrote: 12.01.2021 17:05, Max Reitz wrote: On 26.10.20 18:18, Vladimir Sementsov-Ogievskiy wrote: Further commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests

Re: [PATCH v3 25/25] simplebench: add bench-backup.py

2021-01-12 Thread Max Reitz
On 26.10.20 18:18, Vladimir Sementsov-Ogievskiy wrote: Add script to benchmark new backup architecture. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/simplebench/bench-backup.py | 165 1 file changed, 165 insertions(+) create mode 100755

Re: [PATCH v3 23/25] simplebench/bench_block_job: use correct shebang line with python3

2021-01-12 Thread Max Reitz
On 26.10.20 18:18, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/simplebench/bench_block_job.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Max Reitz

Re: [PATCH v3 21/25] block/block-copy: drop unused block_copy_set_progress_callback()

2021-01-12 Thread Max Reitz
On 26.10.20 18:18, Vladimir Sementsov-Ogievskiy wrote: Drop unused code. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 6 -- block/block-copy.c | 15 --- 2 files changed, 21 deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default

2021-01-12 Thread Max Reitz
On 26.10.20 18:18, Vladimir Sementsov-Ogievskiy wrote: Further commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabled copy_range. So, let's disable copy_range by default.

Re: [PATCH v3 19/25] backup: move to block-copy

2021-01-12 Thread Max Reitz
- 1 file changed, 121 insertions(+), 66 deletions(-) Reviewed-by: Max Reitz Irrelevant notes below. diff --git a/block/backup.c b/block/backup.c index 449b763ce4..0d1cd64eab 100644 --- a/block/backup.c +++ b/block/backup.c (Just something I noted when looking for remaining

Re: [PATCH v3 18/25] block/backup: drop extra gotos from backup_run()

2021-01-12 Thread Max Reitz
On 26.10.20 18:18, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v3 17/25] block/block-copy: make progress_bytes_callback optional

2021-01-12 Thread Max Reitz
-copy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) Reviewed-by: Max Reitz

Re: [PATCH v3 15/25] iotests: 219: prepare for backup over block-copy

2021-01-12 Thread Max Reitz
-size. Update it for explicit chunk-size for backup as for mirror. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/219 | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v3 13/25] iotests: 129: prepare for backup over block-copy

2021-01-12 Thread Max Reitz
t finish immediately. I feel like there’s a bigger problem here. Then again, I have 129 disabled all the time anyway because its throttling simply doesn’t work, so I guess Acked-by: Max Reitz

Re: [PATCH v3 12/25] iotests: 56: prepare for backup over block-copy

2021-01-12 Thread Max Reitz
already paused. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/056 | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v3 11/25] qapi: backup: add max-chunk and max-workers to x-perf struct

2021-01-12 Thread Max Reitz
, return NULL; } +cluster_size = backup_calculate_cluster_size(target, errp); +if (cluster_size < 0) { +goto error; +} + +if (perf->max_workers < 1) { +error_setg(errp, "max-worker must be greater than zero"); *max-workers With that fixed

Re: [PATCH v3 08/25] block/block-copy: add block_copy_cancel

2021-01-12 Thread Max Reitz
files changed, 32 insertions(+), 5 deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v3 07/25] block/block-copy: add ratelimit to block-copy

2021-01-12 Thread Max Reitz
-by: Max Reitz

<    1   2   3   4   5   6   7   8   9   10   >