[RFC PATCH v2 09/11] qemu_iotests: allow valgrint to print/delete the generated log file

2021-04-07 Thread Emanuele Giuseppe Esposito
When using valgrind on the test scripts, it generates a log file in $TEST_DIR that is either print (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests

[RFC PATCH v2 07/11] qemu_iotests: extend the check script to support valgrind for python tests

2021-04-07 Thread Emanuele Giuseppe Esposito
lgrind PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/iotests.py | 11 +++ tests/qemu-iotests/testenv.py | 1 + 2 files ch

Re: [RFC PATCH v2 03/11] qemu-iotests: add option to attach gdbserver

2021-04-08 Thread Emanuele Giuseppe Esposito
On 08/04/2021 17:40, Paolo Bonzini wrote: On 07/04/21 15:50, Emanuele Giuseppe Esposito wrote: +    self.gdb_qemu = os.getenv('GDB_QEMU') + +    if gdb and not self.gdb_qemu: +    self.gdb_qemu = 'localhost:12345' +    elif self.gdb_qemu and not gdb: +    del

Re: [RFC PATCH v2 04/11] qemu-iotests: delay QMP socket timers

2021-04-08 Thread Emanuele Giuseppe Esposito
On 08/04/2021 17:40, Paolo Bonzini wrote: On 07/04/21 15:50, Emanuele Giuseppe Esposito wrote:   def get_qmp_events_filtered(self, wait=60.0):   result = [] -    for ev in self.get_qmp_events(wait=wait): +    qmp_wait = wait +    if qemu_gdb: +    qmp_wait

[PATCH 1/5] blkdebug: refactor removal of a suspended request

2021-04-08 Thread Emanuele Giuseppe Esposito
lo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/blkdebug.c | 28 +--- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 2c0b9b0ee8..8f19d991fa 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -79

[PATCH 3/5] blkdebug: track all actions

2021-04-08 Thread Emanuele Giuseppe Esposito
Add a counter for each action that a rule can trigger. This is mainly used to keep track of how many coroutine_yeld() we need to perform after processing all rules in the list. Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/blkdebug.c | 17

[PATCH 2/5] blkdebug: move post-resume handling to resume_req_by_tag

2021-04-08 Thread Emanuele Giuseppe Esposito
to handle the yielding. Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/blkdebug.c | 31 ++- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 8f19d991fa..e37f999254 100644

[PATCH 0/5] blkdebug: fix racing condition when iterating on

2021-04-08 Thread Emanuele Giuseppe Esposito
to protect rules and suspended_reqs. Signed-off-by: Emanuele Giuseppe Esposito Emanuele Giuseppe Esposito (5): blkdebug: refactor removal of a suspended request blkdebug: move post-resume handling to resume_req_by_tag blkdebug: track all actions blkdebug: do not suspend in the middle

[PATCH 5/5] blkdebug: protect rules and suspended_reqs with a lock

2021-04-08 Thread Emanuele Giuseppe Esposito
Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/blkdebug.c | 32 +++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index dffd869b32..e92a35ccbb 100644 --- a/block/blkdebug.c +++ b

[PATCH 4/5] blkdebug: do not suspend in the middle of QLIST_FOREACH_SAFE

2021-04-08 Thread Emanuele Giuseppe Esposito
Use actions_count to see how many yeld to issue. Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/blkdebug.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 388b5ed615..dffd869b32 100644

[PATCH v2 6/8] block: do not acquire AioContext in check_to_replace_node

2021-04-19 Thread Emanuele Giuseppe Esposito
Make the (only) caller do it. Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 22 +- blockdev.c| 7 ++- include/block/block.h | 1 + 3 files changed, 12 insertions(+), 18

[PATCH v2 8/8] block: do not take AioContext around reopen

2021-04-19 Thread Emanuele Giuseppe Esposito
Reopen needs to handle AioContext carefully due to calling bdrv_drain_all_begin/end. By not taking AioContext around calls to bdrv_reopen_multiple, we can drop the function's release/acquire pair and the AioContext argument too. Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe

[PATCH v2 0/8] Block layer thread-safety, continued

2021-04-19 Thread Emanuele Giuseppe Esposito
://lists.gnu.org/archive/html/qemu-devel/2017-07/msg01398.html Signed-off-by: Emanuele Giuseppe Esposito --- v1 (2017) -> v2 (2021): - v1 Patch "block-backup: add reqs_lock" has been dropped, because now is completely different from the old version and all functions that were affected by

[PATCH v2 1/8] block: prepare write threshold code for thread safety

2021-04-19 Thread Emanuele Giuseppe Esposito
Reviewed-by: Stefan Hajnoczi Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/write-threshold.c | 28 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/block/write-threshold.c b/block/write-threshold.c index

[PATCH v2 2/8] block: protect write threshold QMP commands from concurrent requests

2021-04-19 Thread Emanuele Giuseppe Esposito
For simplicity, use bdrv_drained_begin/end to avoid concurrent writes to the write threshold, or reading it while it is being set. qmp_block_set_write_threshold is protected by the BQL. Reviewed-by: Stefan Hajnoczi Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito

[PATCH v2 7/8] block/replication: do not acquire AioContext

2021-04-19 Thread Emanuele Giuseppe Esposito
Replication functions are mostly called when the BDS is quiescent and does not have any pending I/O. They do not need to synchronize on anything since BDS and BB are now thread-safe. Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/replication.c | 54

[PATCH v2 5/8] block: add a few more notes on locking

2021-04-19 Thread Emanuele Giuseppe Esposito
Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int.h | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index a1aad5ad2d..67a0777e12 100644

[PATCH v2 3/8] util: use RCU accessors for notifiers

2021-04-19 Thread Emanuele Giuseppe Esposito
complete, no matter how "readers" are defined. In the next patch, for example, synchronize_rcu's role is taken by bdrv_drain (which is a superset of synchronize_rcu, since it also blocks new incoming readers). Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Signed-off-by: Emanuel

[PATCH v2 4/8] block: make before-write notifiers thread-safe

2021-04-19 Thread Emanuele Giuseppe Esposito
Reads access the list in RCU style, so be careful to avoid use-after-free scenarios in the backup block job. Apart from this, all that's needed is protecting updates with a mutex. Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block.c

Re: [PATCH] ratelimit: protect with a mutex

2021-04-14 Thread Emanuele Giuseppe Esposito
limit->slice_ns = slice_ns; limit->slice_quota = MAX(((double)speed * slice_ns) / 10ULL, 1); } Reviewed-by: Emanuele Giuseppe Esposito

[PATCH v3 07/15] qemu-iotests: add gdbserver option to script tests too

2021-04-14 Thread Emanuele Giuseppe Esposito
The only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/common.rc | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff

[PATCH v3 12/15] qemu_iotests: insert valgrind command line as wrapper for qemu binary

2021-04-14 Thread Emanuele Giuseppe Esposito
The priority will be given to gdb command line, meaning if the -gdb parameter and -valgrind are given, gdb will be wrapped around the qemu binary. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git

[PATCH v3 08/15] docs/devel/testing: add -gdb option to the debugging section of QEMU iotests

2021-04-14 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- docs/devel/testing.rst | 7 +++ 1 file changed, 7 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index b7e2370e7e..2ee77a057b 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -229,6 +229,13

[PATCH v3 11/15] qemu_iotests: allow valgrind to read/delete the generated log file

2021-04-14 Thread Emanuele Giuseppe Esposito
When using -valgrind on the script tests, it generates a log file in $TEST_DIR that is either read (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests

[PATCH v3 13/15] docs/devel/testing: add -valgrind option to the debug section of QEMU iotests

2021-04-14 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- docs/devel/testing.rst | 7 +++ 1 file changed, 7 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 2ee77a057b..62902cfd2d 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -236,6 +236,13 @@ given

[PATCH v3 04/15] qemu-iotests: add option to attach gdbserver

2021-04-14 Thread Emanuele Giuseppe Esposito
Add -gdb flag and GDB_QEMU environmental variable to python tests to attach a gdbserver to each qemu instance. if -gdb is not provided but $GDB_QEMU is set, ignore the environmental variable. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/check | 6 +- tests/qemu

[PATCH v3 06/15] qemu_iotests: insert gdbserver command line as wrapper for qemu binary

2021-04-14 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 380527245e..4f3fb13915 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu

[PATCH v3 03/15] docs/devel/testing: add debug section to the QEMU iotests chapter

2021-04-14 Thread Emanuele Giuseppe Esposito
Introduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches. Signed-off-by: Emanuele Giuseppe Esposito --- docs/devel/testing.rst | 8 1 file changed, 8 insertions(+) diff --git a/docs/devel/testing.rst b/

[PATCH v3 02/15] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

2021-04-14 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- python/qemu/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index 39a0cf62fe..c18eae96c6 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -111,6 +111,7

[PATCH v3 01/15] python: qemu: add timer parameter for qmp.accept socket

2021-04-14 Thread Emanuele Giuseppe Esposito
Add a new _qmp_timer field to the QEMUMachine class. The default timer is 15 sec, as per the default in the qmp accept() function. Signed-off-by: Emanuele Giuseppe Esposito --- python/qemu/machine.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine.py b

[PATCH v3 00/15] qemu_iotests: improve debugging options

2021-04-14 Thread Emanuele Giuseppe Esposito
. This series is tested on the previous serie "qemu-iotests: quality of life improvements" but independent from it, so it can be applied separately. Signed-off-by: Emanuele Giuseppe Esposito --- v3: - Introduce the class field _qmp_timer instead of a function parameter in the _post_launch() func

[PATCH v3 09/15] qemu_iotests: extend the check script to support valgrind for python tests

2021-04-14 Thread Emanuele Giuseppe Esposito
lgring PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/check | 7 --- tests/qemu-iotests/iotests.py | 11 +++ tests/qemu-iot

[PATCH v3 14/15] qemu_iotests: add option to show qemu binary logs on stdout

2021-04-14 Thread Emanuele Giuseppe Esposito
Using the flag -p, allow the qemu binary to print to stdout. This helps especially when doing print-debugging. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests/check | 3 ++- tests/qemu-iotests/iotests.py | 9 + tests/qemu-iotests/testenv.py | 9 +++-- 3 files

[PATCH v3 15/15] docs/devel/testing: add -p option to the debug section of QEMU iotests

2021-04-14 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- docs/devel/testing.rst | 4 1 file changed, 4 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 62902cfd2d..0c18fc4571 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -246,6 +246,10 @@ given

[PATCH v3 05/15] qemu-iotests: delay QMP socket timers

2021-04-14 Thread Emanuele Giuseppe Esposito
Attaching a gdbserver implies that the qmp socket should wait indefinitely for an answer from QEMU. Signed-off-by: Emanuele Giuseppe Esposito --- python/qemu/machine.py| 3 +++ tests/qemu-iotests/iotests.py | 10 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git

[PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind

2021-04-14 Thread Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon. Signed-off-by: Emanuele Giuseppe Esposito --- python/qemu/machine.py| 2 +- tests/qemu-iotests/iotests.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git

[RFC PATCH 0/3] block-copy: lock tasks and calls list

2021-04-20 Thread Emanuele Giuseppe Esposito
This serie of patches continues Paolo's series on making the block layer thread safe. Add a CoMutex lock for both tasks and calls list present in block/block-copy.c Signed-off-by: Emanuele Giuseppe Esposito Emanuele Giuseppe Esposito (3): block-copy: improve documentation for BlockCopyTask

[RFC PATCH 1/3] block-copy: improve documentation for BlockCopyTask type and functions

2021-04-20 Thread Emanuele Giuseppe Esposito
. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 39ae481c8b..03df50a354 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -48,25 +48,32

[RFC PATCH 2/3] block-copy: add a CoMutex to the BlockCopyTask list

2021-04-20 Thread Emanuele Giuseppe Esposito
Because the list of tasks is only modified by coroutine functions, add a CoMutex in order to protect the list of tasks. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 23 +++ 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/block/block

[RFC PATCH 3/3] block-copy: add CoMutex lock for BlockCopyCallState list

2021-04-20 Thread Emanuele Giuseppe Esposito
Use the same tasks_lock, since the calls list is just used for debug purposes. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index e785ac57e0..555f5fb747 100644

Re: [RFC PATCH 1/3] block-copy: improve documentation for BlockCopyTask type and functions

2021-04-20 Thread Emanuele Giuseppe Esposito
On 20/04/2021 14:03, Vladimir Sementsov-Ogievskiy wrote: 20.04.2021 13:04, Emanuele Giuseppe Esposito wrote: As done in BlockCopyCallState, categorize BlockCopyTask in IN, State and OUT fields. This is just to understand which field has to be protected with a lock. Also add coroutine_fn

Re: [RFC PATCH v2 00/11] qemu_iotests: improve debugging options

2021-04-08 Thread Emanuele Giuseppe Esposito
On 08/04/2021 10:26, Markus Armbruster wrote: Emanuele Giuseppe Esposito writes: This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests. It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file

Re: [RFC PATCH v2 02/11] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

2021-04-09 Thread Emanuele Giuseppe Esposito
diff --git a/python/qemu/machine.py b/python/qemu/machine.py index c721e07d63..18d32ebe45 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -109,7 +109,7 @@ def __init__(self,   self._binary = binary   self._args = list(args) -    self._wrapper = wrapper + 

Re: [RFC PATCH v2 04/11] qemu-iotests: delay QMP socket timers

2021-04-09 Thread Emanuele Giuseppe Esposito
On 08/04/2021 21:03, Paolo Bonzini wrote: Il gio 8 apr 2021, 18:06 Emanuele Giuseppe Esposito <mailto:eespo...@redhat.com>> ha scritto: On 08/04/2021 17:40, Paolo Bonzini wrote: > On 07/04/21 15:50, Emanuele Giuseppe Esposito wrote: >>   def get_q

Re: [RFC PATCH v2 01/11] python: qemu: add timer parameter for qmp.accept socket

2021-04-09 Thread Emanuele Giuseppe Esposito
--- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -138,9 +138,9 @@ def _pre_launch(self) -> None:   super()._pre_launch()   self._qtest = QEMUQtestProtocol(self._qtest_path, server=True) -    def _post_launch(self) -> None: +    def _post_launch(self, timer) -> None:

Re: [RFC PATCH v2 00/11] qemu_iotests: improve debugging options

2021-04-08 Thread Emanuele Giuseppe Esposito
On 08/04/2021 14:39, Markus Armbruster wrote: Emanuele Giuseppe Esposito writes: On 08/04/2021 10:26, Markus Armbruster wrote: Emanuele Giuseppe Esposito writes: This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests. It also allows

[PATCH] include/block.h: remove outdated comment in bdrv_drained_begin

2021-08-16 Thread Emanuele Giuseppe Esposito
in blockjob.c. So no additional pause should be required. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 3477290f9a..1b0d05b545 100644 --- a/include/block

[PATCH v2] include/block.h: remove outdated comment

2021-09-03 Thread Emanuele Giuseppe Esposito
in blockjob.c. So no additional pause should be required. Signed-off-by: Emanuele Giuseppe Esposito --- v2: + add "block jobs" to the external request sources include/block/block.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/include/block/block.h b/include/blo

[RFC PATCH 4/4] block/block-backend.c: assertions for block-backend

2021-09-08 Thread Emanuele Giuseppe Esposito
All the graph API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 93

[RFC PATCH 1/4] main-loop.h: introduce qemu_in_main_thread()

2021-09-08 Thread Emanuele Giuseppe Esposito
in next patches fail despite the AioContext is still the main loop. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/main-loop.h | 13 + softmmu/cpus.c | 5 + stubs/iothread-lock.c| 5 + 3 files changed, 23 insertions(+) diff --git a/include/qemu/main

[RFC PATCH 3/4] include/sysemu/block-backend: split header into I/O and graph API

2021-09-08 Thread Emanuele Giuseppe Esposito
and avoid to change all includes in all c files that use the block-backend APIs. Assertions are added in the next patch. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 9 +- include/sysemu/block-backend-common.h | 74 include/sysemu/block-backend

[RFC PATCH 0/4] block layer: split block APIs in graph and I/O

2021-09-08 Thread Emanuele Giuseppe Esposito
.h and block_int.h 2) audit graph API and replace the AioContext lock with drains, or remove them when not necessary (requires further discussion, not necessary now). 3) [optional as it should be already the case] audit the I/O API and check that thread safety is guaranteed Signed-off-by: Emanue

[RFC PATCH 2/4] migration: block-dirty-bitmap: add missing qemu_mutex_lock_iothread

2021-09-08 Thread Emanuele Giuseppe Esposito
init_dirty_bitmap_migration assumes the iothread lock (BQL) to be held, but instead it isn't. Instead of adding the lock to qemu_savevm_state_setup(), follow the same pattern as the other ->save_setup callbacks and lock+unlock inside dirty_bitmap_save_setup(). Signed-off-by: Emanuele Giuse

Re: [RFC PATCH 0/4] block layer: split block APIs in graph and I/O

2021-09-16 Thread Emanuele Giuseppe Esposito
On 15/09/2021 16:43, Stefan Hajnoczi wrote: On Wed, Sep 15, 2021 at 02:11:41PM +0200, Paolo Bonzini wrote: On 13/09/21 15:10, Stefan Hajnoczi wrote: On Wed, Sep 08, 2021 at 09:10:17AM -0400, Emanuele Giuseppe Esposito wrote: Currently, block layer APIs like block-backend.h contain a mix

Re: [PATCH v8 08/16] qemu-iotests: add gdbserver option to script tests too

2021-08-09 Thread Emanuele Giuseppe Esposito
diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index 0fc52d20d7..cbca757b49 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -85,7 +85,12 @@ _timed_wait_for()   timeout=yes   QEMU_STATUS[$h]=0 -    while IFS= read -t

[PATCH v9 05/16] qemu-iotests: add option to attach gdbserver

2021-08-09 Thread Emanuele Giuseppe Esposito
-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/check | 6 +- tests/qemu-iotests/iotests.py | 5 + tests/qemu-iotests/testenv.py | 17 +++-- 3 files changed, 25 insertions(+), 3 deletions(-) diff

[PATCH v9 02/16] python: Reduce strictness of pylint's duplicate-code check

2021-08-09 Thread Emanuele Giuseppe Esposito
From: John Snow Pylint prior to 2.8.3 (We pin at >= 2.8.0) includes function and method signatures as part of its duplicate checking algorithm. This check does not listen to pragmas, so the only way to disable it is to turn it off completely or increase the minimum duplicate lines so that it

[PATCH v9 01/16] python: qemu: add timer parameter for qmp.accept socket

2021-08-09 Thread Emanuele Giuseppe Esposito
to the QMP monitor test command execution. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Acked-by: John Snow Reviewed-by: Max Reitz --- python/qemu/machine/machine.py | 7 +-- python/qemu/machine/qtest.py | 5 +++-- tests/qemu

[PATCH v9 00/16] qemu_iotests: improve debugging options

2021-08-09 Thread Emanuele Giuseppe Esposito
. Signed-off-by: Emanuele Giuseppe Esposito --- v9: * Replace `! -z` with `-n` in bash scripts (patch 8), and quote $GDB_OPTIONS in the same if condition [Max] * Add r-b from Max to all patches except 8, remove r-b from Vladimir on patch 8 Emanuele Giuseppe Esposito (15): python: qemu: add timer

[PATCH v9 03/16] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

2021-08-09 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz Acked-by: John Snow --- python/qemu/machine/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/qtest.py b/python

[PATCH v9 10/16] qemu-iotests: extend the check script to prepare supporting valgrind for python tests

2021-08-09 Thread Emanuele Giuseppe Esposito
lgrind PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/check | 7

[PATCH v9 15/16] qemu-iotests: add option to show qemu binary logs on stdout

2021-08-09 Thread Emanuele Giuseppe Esposito
Using the flag -p, allow the qemu binary to print to stdout. Also create the common function _close_qemu_log_file() to avoid accessing machine.py private fields directly and have duplicate code. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed

[PATCH v9 14/16] docs/devel/testing: add -valgrind option to the debug section of QEMU iotests

2021-08-09 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- docs/devel/testing.rst | 6 ++ 1 file changed, 6 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 01e1919873..8ebcf85a31 100644 --- a/docs/devel

[PATCH v9 12/16] qemu-iotests: allow valgrind to read/delete the generated log file

2021-08-09 Thread Emanuele Giuseppe Esposito
When using -valgrind on the script tests, it generates a log file in $TEST_DIR that is either read (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir

[PATCH v9 16/16] docs/devel/testing: add -p option to the debug section of QEMU iotests

2021-08-09 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- docs/devel/testing.rst | 4 1 file changed, 4 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 8ebcf85a31..4a0abbf23d 100644 --- a/docs/devel

[PATCH v9 13/16] qemu-iotests: insert valgrind command line as wrapper for qemu binary

2021-08-09 Thread Emanuele Giuseppe Esposito
If -gdb and -valgrind are both defined, return an error. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests

[PATCH v9 08/16] qemu-iotests: add gdbserver option to script tests too

2021-08-09 Thread Emanuele Giuseppe Esposito
Remove read timer in test script when GDB_OPTIONS are set, so that the bash tests won't timeout while running gdb. The only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail. Signed-off-by: Emanuele Giuseppe

[PATCH v9 09/16] docs/devel/testing: add -gdb option to the debugging section of QEMU iotests

2021-08-09 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- docs/devel/testing.rst | 11 +++ 1 file changed, 11 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 8359f2ae37..01e1919873 100644 --- a/docs

[PATCH v9 11/16] qemu-iotests: extend QMP socket timeout when using valgrind

2021-08-09 Thread Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout and the generic class Timeout in iotests.py timeouts too soon. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py

[PATCH v9 06/16] qemu-iotests: delay QMP socket timers

2021-08-09 Thread Emanuele Giuseppe Esposito
Attaching gdbserver implies that the qmp socket should wait indefinitely for an answer from QEMU. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests

[PATCH v9 07/16] qemu_iotests: insert gdbserver command line as wrapper for qemu binary

2021-08-09 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e176a84620

[PATCH v9 04/16] docs/devel/testing: add debug section to the QEMU iotests chapter

2021-08-09 Thread Emanuele Giuseppe Esposito
Introduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- docs/devel/testing.rst | 8 1 file

[PATCH v2 1/2] pylint: fix errors and warnings generated by tests/qemu-iotests/297

2021-10-08 Thread Emanuele Giuseppe Esposito
Test 297 in tests/qemu-iotests currently fails: pylint has learned new things to check, or we simply missed them. All fixes in this patch are related to additional spaces used or wrong indentation. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu-iotests

Re: [RFC PATCH v2 11/25] include/block/blockjob.h: global state API

2021-10-08 Thread Emanuele Giuseppe Esposito
+/* + * Global state (GS) API. These functions run under the BQL lock. + * + * If a function modifies the graph, it also uses drain and/or + * aio_context_acquire/release to be sure it has unique access. + * aio_context locking is needed together with BQL because of + * the thread-safe I/O API

[PATCH v2 0/2] pylint: fix new errors and warnings in qemu-iotests

2021-10-08 Thread Emanuele Giuseppe Esposito
There are some warnings and errors that we either miss or are new in pylint. Anyways, test 297 of qemu-iotests fails because of that, so we need to fix it. All these fixes involve just indentation or additional spaces added. Signed-off-by: Emanuele Giuseppe Esposito --- v2: * temporarly enable

Re: [RFC PATCH v2 04/25] include/block/block: split header into I/O and global state API

2021-10-08 Thread Emanuele Giuseppe Esposito
On 07/10/2021 15:01, Stefan Hajnoczi wrote: On Thu, Oct 07, 2021 at 12:43:43PM +0100, Daniel P. Berrangé wrote: On Tue, Oct 05, 2021 at 10:31:54AM -0400, Emanuele Giuseppe Esposito wrote: Similarly to the previous patch, split block.h in block-io.h and block-global-state.h block-common.h

[PATCH v2 2/2] qemu-iotests: fix image-fleecing pylint errors

2021-10-08 Thread Emanuele Giuseppe Esposito
The problem here is that some variables are formatted with unnecessary spaces to make it prettier and easier to read. However, pylint complains about those additional spaces. Temporarly remove the check, but just on those variables. Signed-off-by: Emanuele Giuseppe Esposito --- tests/qemu

Re: [RFC PATCH v2 17/25] include/block/transactions: global state API + assertions

2021-10-08 Thread Emanuele Giuseppe Esposito
On 07/10/2021 16:47, Stefan Hajnoczi wrote: On Tue, Oct 05, 2021 at 10:32:07AM -0400, Emanuele Giuseppe Esposito wrote: transactions run always under the BQL lock, so they are all in the global state API. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/transactions.h | 24

[PATCH v3 00/25] block layer: split block APIs in global state and I/O

2021-10-12 Thread Emanuele Giuseppe Esposito
m when not necessary (requires further discussion). 2) [optional as it should be already the case] audit the I/O API and check that thread safety is guaranteed Signed-off-by: Emanuele Giuseppe Esposito --- v2 -> v3: * rename "graph API" into "global state API" * change o

[PATCH v3 08/25] block: introduce assert_bdrv_graph_writable

2021-10-12 Thread Emanuele Giuseppe Esposito
. Because adding drains requires additional discussions, they will be added in future series. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 5 + block/io.c | 11 +++ include/block/block_int-global-state.h | 10

[PATCH v3 10/25] assertions for blockjob_int.h

2021-10-12 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 4 1 file changed, 4 insertions(+) diff --git a/blockjob.c b/blockjob.c index 4bad1408cb..fbd6c7d873 100644 --- a/blockjob.c +++ b/blockjob.c @@ -83,6 +83,7 @@ BlockJob *block_job_get(const char *id

[PATCH v3 06/25] include/block/block_int: split header into I/O and global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- blockdev.c |5 + include/block/block_int-common.h | 1164 +++ include/block/block_int-global-state.h | 319 + include/block/block_int-io.h | 163 +++ include/block/block_int.h | 1478

[PATCH v3 09/25] include/block/blockjob_int.h: split header into I/O and GS API

2021-10-12 Thread Emanuele Giuseppe Esposito
Since the I/O functions are not many, keep a single file. Also split the function pointers in BlockJobDriver. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/blockjob_int.h | 28 1 file changed, 28 insertions(+) diff --git

[PATCH v3 20/25] block_int-common.h: assertion in the callers of BlockDriver function pointers

2021-10-12 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block.c | 17 + 1 file changed, 17 insertions(+) diff --git a/block.c b/block.c index 94bff5c757..40c4729b8d 100644 --- a/block.c +++ b/block.c @@ -1074,6 +1074,7 @@ int refresh_total_sectors

[PATCH v3 18/25] block/coroutines: I/O API

2021-10-12 Thread Emanuele Giuseppe Esposito
block coroutines functions run in different aiocontext, and are not protected by the BQL. Therefore are I/O. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/coroutines.h | 7 +++ 1 file changed, 7 insertions(+) diff --git

[PATCH v3 15/25] assertions for blockdev.h global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 2 ++ blockdev.c| 12 2 files changed, 14 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 6f36a54e82

[PATCH v3 25/25] job.h: assertions in the callers of JobDriver funcion pointers

2021-10-12 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- job.c | 9 + 1 file changed, 9 insertions(+) diff --git a/job.c b/job.c index dbfa67bb0a..94b142684f 100644 --- a/job.c +++ b/job.c @@ -380,6 +380,8 @@ void job_ref(Job *job) void job_unref(Job *job

Re: [RFC PATCH v2 24/25] job.h: split function pointers in JobDriver

2021-10-08 Thread Emanuele Giuseppe Esposito
On 07/10/2021 16:54, Stefan Hajnoczi wrote: On Tue, Oct 05, 2021 at 10:32:14AM -0400, Emanuele Giuseppe Esposito wrote: The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 31 +++ 1 file

[PATCH v3 02/25] include/block/block: split header into I/O and global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
earer view on what needs what kind of protection. block-common.h instead contains common structures shared by both headers. block.h is left there for legacy and to avoid changing all includes in all c files that use the block APIs. Assertions are added in the next patch. Signed-off-by: Emanuele Giusepp

[PATCH v3 05/25] block/block-backend.c: assertions for block-backend

2021-10-12 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 90

[PATCH v3 22/25] block_int-common.h: assertions in the callers of BdrvChildClass function pointers

2021-10-12 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/block.c b/block.c index 40c4729b8d..da80e89ad4 100644 --- a/block.c +++ b/block.c @@ -1462,6 +1462,7 @@ const BdrvChildClass child_of_bds

[PATCH v3 24/25] job.h: split function pointers in JobDriver

2021-10-12 Thread Emanuele Giuseppe Esposito
The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 16 1 file changed, 16 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index 6e67b6977f..7e9e59f4b8 100644 --- a/include/qemu/job.h

[PATCH v3 01/25] main-loop.h: introduce qemu_in_main_thread()

2021-10-12 Thread Emanuele Giuseppe Esposito
in next patches fail despite the AioContext is still the main loop. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/qemu/main-loop.h | 13 + softmmu/cpus.c | 5 + stubs/iothread-lock.c| 5 + 3 files changed, 23 insertions(+) diff

[PATCH v3 13/25] include/sysemu/blockdev.h: move drive_add and inline drive_def

2021-10-12 Thread Emanuele Giuseppe Esposito
simplifying next patch. Signed-off-by: Emanuele Giuseppe Esposito --- block/monitor/block-hmp-cmds.c | 2 +- blockdev.c | 27 +-- include/sysemu/blockdev.h | 6 ++ softmmu/vl.c | 25 - 4 files changed, 28

[PATCH v3 11/25] include/block/blockjob.h: global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
blockjob functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/blockjob.h | 9 + 1 file changed, 9 insertions(+) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index d200f33c10..fa0c3f7a47 100644 --- a/include/block

[PATCH v3 12/25] assertions for blockob.h global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockjob.c b/blockjob.c index fbd6c7d873..4982f6a2b5 100644 --- a/blockjob.c +++ b/blockjob.c @@ -61,6 +61,7 @@ static bool is_block_job(Job *job

[PATCH v3 16/25] include/block/snapshot: global state API + assertions

2021-10-12 Thread Emanuele Giuseppe Esposito
Snapshots run also under the BQL lock, so they all are in the global state API. The aiocontext lock that they hold is currently an overkill and in future could be removed. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/snapshot.c

[PATCH v3 03/25] assertions for block global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block.c| 136

[PATCH v3 04/25] include/sysemu/block-backend: split header into I/O and global state (GS) API

2021-10-12 Thread Emanuele Giuseppe Esposito
ared between the two headers, and the functions that can't be categorized as I/O or global state. Assertions are added in the next patch. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 9 +- include/sysemu/block-backend-common.h | 74 ++

[PATCH v3 14/25] include/systemu/blockdev.h: global state API

2021-10-12 Thread Emanuele Giuseppe Esposito
blockdev functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito --- include/sysemu/blockdev.h | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index 960b54d320..cc9dfb5181 100644

<    1   2   3   4   5   6   7   8   9   10   >