Re: [PATCH 2/4] qtest: replace gettimeofday with GTimer

2022-03-04 Thread Laurent Vivier
Le 04/03/2022 à 16:54, Laurent Vivier a écrit : Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : From: Marc-André Lureau glib provides a convenience helper to measure elapsed time. It isn't subject to wall-clock time changes. Note that this changes the initial OPENED time, which

Re: [PATCH 4/4] oslib: drop qemu_gettimeofday()

2022-03-04 Thread Stefan Weil
Am 04.03.22 um 16:27 schrieb marcandre.lur...@redhat.com: From: Marc-André Lureau No longer used after the previous patches. Signed-off-by: Marc-André Lureau --- include/sysemu/os-posix.h | 3 --- include/sysemu/os-win32.h | 6 -- util/oslib-win32.c| 20

[PULL 38/50] block_int-common.h: split function pointers in BlockDriver

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Similar to the header split, also the function pointers in BlockDriver can be split in I/O and global state. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-26-eespo...@redhat.com> Signed-off-by: Kevin Wolf ---

[PULL 39/50] block_int-common.h: assertions in the callers of BlockDriver function pointers

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-27-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- block.c| 17 + block/create.c | 2 ++ 2 files changed, 19 insertions(+) diff --git a/block.c b/block.c

Re: [PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

2022-03-04 Thread Marc-André Lureau
Hi On Fri, Mar 4, 2022 at 8:10 PM Laurent Vivier wrote: > Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : > > From: Marc-André Lureau > > > > GLib g_get_real_time() is an alternative to gettimeofday(). > > > > For semihosting, a few bits are lost on POSIX host, but this shouldn't

[PATCH v2 11/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets

2022-03-04 Thread Daniel P . Berrangé
This validates that connections to an NBD server running on a UNIX socket can use TLS, and require a TLS hostname override to pass certificate validation. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/233 | 24

[PATCH v2 02/12] block: pass desired TLS hostname through from block driver client

2022-03-04 Thread Daniel P . Berrangé
In commit a71d597b989fd701b923f09b3c20ac4fcaa55e81 Author: Vladimir Sementsov-Ogievskiy Date: Thu Jun 10 13:08:00 2021 +0300 block/nbd: reuse nbd_co_do_establish_connection() in nbd_open() the use of the 'hostname' field from the BDRVNBDState struct was lost, and 'nbd_connect' just

[PATCH v2 10/12] tests/qemu-iotests: validate NBD TLS with hostname mismatch

2022-03-04 Thread Daniel P . Berrangé
This validates that connections to an NBD server where the certificate hostname does not match will fail. It further validates that using the new 'tls-hostname' override option can solve the failure. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/233|

[PATCH v2 07/12] tests/qemu-iotests: expand _filter_nbd rules

2022-03-04 Thread Daniel P . Berrangé
Some tests will want to use 'localhost' instead of '127.0.0.1', and some will use the image options syntax rather than the classic URI syntax. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/common.filter | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v2 3/5] iotests: Remove explicit checks for qemu_img() == 0

2022-03-04 Thread John Snow
qemu_img() returning zero ought to be the rule, not the exception. Remove all explicit checks against the condition in preparation for making non-zero returns an Exception. Signed-off-by: John Snow --- tests/qemu-iotests/163 | 9 +++-- tests/qemu-iotests/216

Re: [PATCH v2 3/9] util: Unify implementations of qemu_memalign()

2022-03-04 Thread Philippe Mathieu-Daudé
On 4/3/22 12:21, Peter Maydell wrote: We implement qemu_memalign() in both oslib-posix.c and oslib-win32.c, but the two versions are essentially the same: they call qemu_try_memalign(), and abort() after printing an error message if it fails. The only difference is that the win32 version prints

[PATCH 5/5] iotests/296: Accept early failure

2022-03-04 Thread Hanna Reitz
As of "crypto: perform permission checks under BQL", permission failures occur early in the blockdev-amend job to update a LUKS volume's keys. Expect them in x-blockdev-amend's QMP reply instead of waiting for the actual job to fail later. (Note that the job will still be created, so we need to

[PATCH 0/5] block/amend: Fix failures seen in iotest 296

2022-03-04 Thread Hanna Reitz
Hi, I’ve tried basing my block branch on Kevin’s and noticed that after “crypto: perform permission checks under BQL”, iotest 296 was failing. I/We have debugged those failures and here are fixes for it. Hence, this series is based on Kevin’s block branch

[PULL 04/50] block: rename bdrv_invalidate_cache_all, blk_invalidate_cache and test_sync_op_invalidate_cache

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Following the bdrv_activate renaming, change also the name of the respective callers. bdrv_invalidate_cache_all -> bdrv_activate_all blk_invalidate_cache -> blk_activate test_sync_op_invalidate_cache -> test_sync_op_activate No functional change intended.

[PULL 12/50] qsd: Add --daemonize

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz To implement this, we reuse the existing daemonizing functions from the system emulator, which mainly do the following: - Fork off a child process, and set up a pipe between parent and child - The parent process waits until the child sends a status byte over the pipe (0 means

[PULL 01/50] crypto: perform permission checks under BQL

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Move the permission API calls into driver-specific callbacks that always run under BQL. In this case, bdrv_crypto_luks needs to perform permission checks before and after qcrypto_block_amend_options(). The problem is that the caller,

[PULL 05/50] block: move BQL logic of bdrv_co_invalidate_cache in bdrv_activate

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Split bdrv_co_invalidate cache in two: the Global State (under BQL) code that takes care of permissions and running GS callbacks, and leave only the I/O code (->bdrv_co_invalidate_cache) running in the I/O coroutine. The only side effect is that

[PULL 09/50] cpus: use coroutine TLS macros for iothread_locked

2022-03-04 Thread Kevin Wolf
From: Stefan Hajnoczi qemu_mutex_iothread_locked() may be used from coroutines. Standard __thread variables cannot be used by coroutines. Use the coroutine TLS macros instead. Signed-off-by: Stefan Hajnoczi Message-Id: <20220222140150.27240-5-stefa...@redhat.com> Signed-off-by: Kevin Wolf ---

Re: [PULL 0/6] hw/nvme updates

2022-03-04 Thread Peter Maydell
On Fri, 4 Mar 2022 at 06:24, Klaus Jensen wrote: > > From: Klaus Jensen > > Hi Peter, > > Last round of hw/nvme updates for v7.0. > > The following changes since commit 64ada298b98a51eb2512607f6e6180cb330c47b1: > > Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220302' into >

[PULL 02/50] crypto: distinguish between main loop and I/O in block_crypto_amend_options_generic_luks

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito block_crypto_amend_options_generic_luks uses the block layer permission API, therefore it should be called with the BQL held. However, the same function is being called by two BlockDriver callbacks: bdrv_amend_options (under BQL) and bdrv_co_amend (I/O). The

[PULL 03/50] block: introduce bdrv_activate

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito This function is currently just a wrapper for bdrv_invalidate_cache(), but in future will contain the code of bdrv_co_invalidate_cache() that has to always be protected by BQL, and leave the rest in the I/O coroutine. Replace all bdrv_invalidate_cache()

[PULL 34/50] assertions for blockdev.h global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-22-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- block/block-backend.c | 3 +++ blockdev.c| 16 2 files changed, 19 insertions(+) diff --git

[PATCH v2 5/5] iotests: fortify compare_images() against crashes

2022-03-04 Thread John Snow
Fortify compare_images() to be more discerning about the status codes it receives. If qemu_img() returns an exit code that implies it didn't actually perform the comparison, treat that as an exceptional circumstance and force the caller to be aware of the peril. If a negative test is desired

Re: [PATCH 4/4] oslib: drop qemu_gettimeofday()

2022-03-04 Thread Laurent Vivier
Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : From: Marc-André Lureau No longer used after the previous patches. Signed-off-by: Marc-André Lureau --- include/sysemu/os-posix.h | 3 --- include/sysemu/os-win32.h | 6 -- util/oslib-win32.c| 20

Re: [PATCH 07/12] tests/qemu-iotests: expand _filter_nbd rules

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:25PM +, Daniel P. Berrangé wrote: > Some tests will want to use 'localhost' instead of '127.0.0.1', and > some will use the image options syntax rather than the classic URI > syntax. > > Signed-off-by: Daniel P. Berrangé > --- > tests/qemu-iotests/common.filter

[PULL 27/50] block: introduce assert_bdrv_graph_writable

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito We want to be sure that the functions that write the child and parent list of a bs are under BQL and drain. BQL prevents from concurrent writings from the GS API, while drains protect from I/O. TODO: drains are missing in some functions using this assert.

[PULL 44/50] job.h: assertions in the callers of JobDriver function pointers

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-32-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- job.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/job.c b/job.c index 54db80df66..075c6f3a20 100644 ---

[PULL 43/50] job.h: split function pointers in JobDriver

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-31-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/qemu/job.h | 22 ++ 1 file changed, 22

[PULL 40/50] block_int-common.h: split function pointers in BdrvChildClass

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-28-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/block/block_int-common.h | 81 ++-- 1 file changed, 47 insertions(+), 34 deletions(-) diff

[PULL 33/50] include/sysemu/blockdev.h: global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito blockdev functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-21-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/sysemu/blockdev.h | 13 ++--- 1 file changed, 10

[PULL 42/50] block-backend-common.h: split function pointers in BlockDevOps

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Assertions in the callers of the function pointrs are already added by previous patches. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20220303151616.325444-30-eespo...@redhat.com>

[PATCH v2 4/5] iotests: make qemu_img raise on non-zero rc by default

2022-03-04 Thread John Snow
re-write qemu_img() as a function that will by default raise a VerboseProcessException (extended from CalledProcessException) on non-zero return codes. This will produce a stack trace that will show the command line arguments and return code from the failed process run. Users that want something

[PATCH 2/5] block/amend: Always call .bdrv_amend_clean()

2022-03-04 Thread Hanna Reitz
.bdrv_amend_clean() says block drivers can use it to clean up what was done in .bdrv_amend_pre_run(). Therefore, it should always be called after .bdrv_amend_pre_run(), which means we need it to call it in the JobDriver.free() callback, not in JobDriver.clean(). Signed-off-by: Hanna Reitz ---

[PATCH 3/5] block/crypto: Reset updating_keys on perm failure

2022-03-04 Thread Hanna Reitz
When the permissions could not be changed as would be necessary for updating the keys, reset updating_keys to false so block_crypto_child_perms() will not continue to try claiming these permissions. Signed-off-by: Hanna Reitz --- block/crypto.c | 8 +++- 1 file changed, 7 insertions(+), 1

[PATCH 4/5] block/amend: Keep strong reference to BDS

2022-03-04 Thread Hanna Reitz
Otherwise, the BDS might be freed while the job is running, which would cause a use-after-free. Signed-off-by: Hanna Reitz --- block/amend.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/amend.c b/block/amend.c index 553890ded9..f696a006e3 100644 --- a/block/amend.c +++

Re: [PATCH 1/4] m68k/nios2-semi: fix gettimeofday() result check

2022-03-04 Thread Laurent Vivier
Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : From: Marc-André Lureau gettimeofday() returns 0 for success. Signed-off-by: Marc-André Lureau --- target/m68k/m68k-semi.c | 2 +- target/nios2/nios2-semi.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git

[PULL 08/50] rcu: use coroutine TLS macros

2022-03-04 Thread Kevin Wolf
From: Stefan Hajnoczi RCU may be used from coroutines. Standard __thread variables cannot be used by coroutines. Use the coroutine TLS macros instead. Signed-off-by: Stefan Hajnoczi Message-Id: <20220222140150.27240-4-stefa...@redhat.com> Signed-off-by: Kevin Wolf --- include/qemu/rcu.h

[PULL 21/50] block/block-backend.c: assertions for block-backend

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito Message-Id:

[PULL 46/50] iotests: Allow using QMP with the QSD

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz Add a parameter to optionally open a QMP connection when creating a QemuStorageDaemon instance. Signed-off-by: Hanna Reitz Message-Id: <20220216105355.30729-3-hre...@redhat.com> Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf ---

[PULL 47/50] iotests/graph-changes-while-io: New test

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz Test the following scenario: 1. Some block node (null-co) attached to a user (here: NBD server) that performs I/O and keeps the node in an I/O thread 2. Repeatedly run blockdev-add/blockdev-del to add/remove an overlay to/from that node Each blockdev-add triggers

[PULL 45/50] block: Make bdrv_refresh_limits() non-recursive

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz bdrv_refresh_limits() recurses down to the node's children. That does not seem necessary: When we refresh limits on some node, and then recurse down and were to change one of its children's BlockLimits, then that would mean we noticed the changed limits by pure chance. The

[PULL 32/50] assertions for blockjob.h global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-20-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- blockjob.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/blockjob.c b/blockjob.c index d79a52d204..4868453d74

[PULL 50/50] block/amend: Keep strong reference to BDS

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz Otherwise, the BDS might be freed while the job is running, which would cause a use-after-free. Signed-off-by: Hanna Reitz Message-Id: <20220304153729.711387-5-hre...@redhat.com> Signed-off-by: Kevin Wolf --- block/amend.c | 3 +++ 1 file changed, 3 insertions(+) diff

Re: [PATCH 02/12] block: pass desired TLS hostname through from block driver client

2022-03-04 Thread Daniel P . Berrangé
On Thu, Mar 03, 2022 at 02:14:34PM -0600, Eric Blake wrote: > On Thu, Mar 03, 2022 at 04:03:20PM +, Daniel P. Berrangé wrote: > > In > > > > commit a71d597b989fd701b923f09b3c20ac4fcaa55e81 > > Author: Vladimir Sementsov-Ogievskiy > > Date: Thu Jun 10 13:08:00 2021 +0300 > > > >

[PATCH v2 03/12] block/nbd: support override of hostname for TLS certificate validation

2022-03-04 Thread Daniel P . Berrangé
When connecting to an NBD server with TLS and x509 credentials, the client must validate the hostname it uses for the connection, against that published in the server's certificate. If the client is tunnelling its connection over some other channel, however, the hostname it uses may not match the

[PATCH v2 09/12] tests/qemu-iotests: convert NBD TLS test to use standard filters

2022-03-04 Thread Daniel P . Berrangé
Using standard filters is more future proof than rolling our own. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/233 | 29 - tests/qemu-iotests/233.out | 8 2 files changed, 16 insertions(+), 21 deletions(-) diff

Re: [PATCH v2 6/9] util: Share qemu_try_memalign() implementation between POSIX and Windows

2022-03-04 Thread Philippe Mathieu-Daudé
On 4/3/22 12:21, Peter Maydell wrote: The qemu_try_memalign() functions for POSIX and Windows used to be significantly different, but these days they are identical except for the actual allocation function called, and the POSIX version already has to have ifdeffery for different allocation

Re: [PATCH v2 1/3] block: Make bdrv_refresh_limits() non-recursive

2022-03-04 Thread Hanna Reitz
On 04.03.22 15:14, Kevin Wolf wrote: Am 04.03.2022 um 13:44 hat Hanna Reitz geschrieben: On 03.03.22 17:56, Kevin Wolf wrote: Am 16.02.2022 um 11:53 hat Hanna Reitz geschrieben: bdrv_refresh_limits() recurses down to the node's children. That does not seem necessary: When we refresh limits

[PATCH 1/5] block/amend: Clean up job on early failure

2022-03-04 Thread Hanna Reitz
Signed-off-by: Hanna Reitz --- block/amend.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/amend.c b/block/amend.c index 329bca53dc..f465738665 100644 --- a/block/amend.c +++ b/block/amend.c @@ -134,6 +134,7 @@ void qmp_x_blockdev_amend(const char *job_id, s->force = has_force ?

Re: [PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

2022-03-04 Thread Laurent Vivier
Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : From: Marc-André Lureau GLib g_get_real_time() is an alternative to gettimeofday(). For semihosting, a few bits are lost on POSIX host, but this shouldn't be a big concern. Signed-off-by: Marc-André Lureau --- blockdev.c

Re: [PATCH 0/5] block/amend: Fix failures seen in iotest 296

2022-03-04 Thread Kevin Wolf
Am 04.03.2022 um 16:37 hat Hanna Reitz geschrieben: > Hi, > > I’ve tried basing my block branch on Kevin’s and noticed that after > “crypto: perform permission checks under BQL”, iotest 296 was failing. > I/We have debugged those failures and here are fixes for it. > > Hence, this series is

Re: [PATCH 06/12] tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:24PM +, Daniel P. Berrangé wrote: > When developing an I/O test it is typical to add some logic to the > test script, run it to view the output diff, and then apply the > output diff to the reference file. This can be drastically simplified > by letting the test

[PULL 00/50] Block layer patches

2022-03-04 Thread Kevin Wolf
The following changes since commit 6629bf78aac7e53f83fd0bcbdbe322e2302dfd1f: Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20220302' into staging (2022-03-03 14:46:48 +) are available in the Git repository at: https://gitlab.com/kmwolf/qemu.git tags/for-upstream

[PULL 18/50] IO_CODE and IO_OR_GS_CODE for block I/O API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Mark all I/O functions with IO_CODE, and all "I/O OR GS" with IO_OR_GS_CODE. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-6-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/block/block-io.h | 1 + block.c

[PULL 17/50] assertions for block global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito Message-Id:

[PULL 20/50] include/sysemu/block-backend: split header into I/O and global state (GS) API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Similarly to the previous patches, split block-backend.h in block-backend-io.h and block-backend-global-state.h In addition, remove "block/block.h" include as it seems it is not necessary anymore, together with "qemu/iov.h" block-backend-common.h contains the

[PULL 22/50] IO_CODE and IO_OR_GS_CODE for block-backend I/O API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Mark all I/O functions with IO_CODE, and all "I/O OR GS" with IO_OR_GS_CODE. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-10-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/sysemu/block-backend-io.h | 2 ++

[PULL 26/50] IO_CODE and IO_OR_GS_CODE for block_int I/O API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Mark all I/O functions with IO_CODE, and all "I/O OR GS" with IO_OR_GS_CODE. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-14-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/block/block_int-io.h | 6 ++ block.c

[PULL 28/50] include/block/blockjob_int.h: split header into I/O and GS API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Since the I/O functions are not many, keep a single file. Also split the function pointers in BlockJobDriver. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Message-Id: <20220303151616.325444-16-eespo...@redhat.com> Signed-off-by: Kevin

Re: [PATCH 10/12] tests/qemu-iotests: validate NBD TLS with hostname mismatch

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:28PM +, Daniel P. Berrangé wrote: > This validates that connections to an NBD server where the certificate > hostname does not match will fail. It further validates that using the > new 'tls-hostname' override option can solve the failure. > > Signed-off-by:

Re: [PATCH 11/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:29PM +, Daniel P. Berrangé wrote: > This validates that connections to an NBD server running on a UNIX > socket can use TLS, and require a TLS hostname override to pass > certificate validation. > > Signed-off-by: Daniel P. Berrangé > --- >

Re: [PATCH 12/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:30PM +, Daniel P. Berrangé wrote: > This validates that connections to an NBD server running on a UNIX > socket can use TLS with pre-shared keys (PSK). > > Signed-off-by: Daniel P. Berrangé > --- > tests/qemu-iotests/233| 28

Re: [PATCH] hw/block: m25p80: Add support for w25q01jvq

2022-03-04 Thread Philippe Mathieu-Daudé
On 4/3/22 19:09, Patrick Williams wrote: The w25q01jvq is a 128MB part. Support is being added to the kernel[1] and the two have been tested together. 1. https://lore.kernel.org/lkml/2022022209.23108-1-potin@quantatw.com/ Signed-off-by: Patrick Williams Cc: Potin Lai ---

Re: [PATCH 08/12] tests/qemu-iotests: introduce filter for qemu-nbd export list

2022-03-04 Thread Daniel P . Berrangé
On Fri, Mar 04, 2022 at 10:43:45AM -0600, Eric Blake wrote: > On Thu, Mar 03, 2022 at 04:03:26PM +, Daniel P. Berrangé wrote: > > Introduce a filter for the output of qemu-nbd export list so it can be > > reused in multiple tests. > > > > The filter is a bit more permissive that what test 241

[PATCH v2 05/12] block/nbd: don't restrict TLS usage to IP sockets

2022-03-04 Thread Daniel P . Berrangé
The TLS usage for NBD was restricted to IP sockets because validating x509 certificates requires knowledge of the hostname that the client is connecting to. TLS does not have to use x509 certificates though, as PSK (pre-shared keys) provide an alternative credential option. These have no

[PATCH v2 00/12] nbd: enable use of TLS on non-TCP transports and other TLS improvements

2022-03-04 Thread Daniel P . Berrangé
This series was principally motivated by a desire to enabl use of TLS on non-TCP transports. For x509 certificates this means we need a way to set the hostname to use for validation. This also lets us override the hostname when connecting on a TCP transport that is tunnelled or port-forwarded. It

[PATCH v2 12/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

2022-03-04 Thread Daniel P . Berrangé
This validates that connections to an NBD server running on a UNIX socket can use TLS with pre-shared keys (PSK). Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/233| 28 tests/qemu-iotests/233.out| 17 +

[PATCH 1/4] m68k/nios2-semi: fix gettimeofday() result check

2022-03-04 Thread marcandre . lureau
From: Marc-André Lureau gettimeofday() returns 0 for success. Signed-off-by: Marc-André Lureau --- target/m68k/m68k-semi.c | 2 +- target/nios2/nios2-semi.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/target/m68k/m68k-semi.c b/target/m68k/m68k-semi.c index

[PATCH 4/4] oslib: drop qemu_gettimeofday()

2022-03-04 Thread marcandre . lureau
From: Marc-André Lureau No longer used after the previous patches. Signed-off-by: Marc-André Lureau --- include/sysemu/os-posix.h | 3 --- include/sysemu/os-win32.h | 6 -- util/oslib-win32.c| 20 3 files changed, 29 deletions(-) diff --git

[PATCH 0/4] RFC: remove qemu_gettimeofday()

2022-03-04 Thread marcandre . lureau
From: Marc-André Lureau Hi, Here is a few patches to replace qemu_gettimeofday() helper with functions provided by GLib. Marc-André Lureau (4): m68k/nios2-semi: fix gettimeofday() result check qtest: replace gettimeofday with GTimer Replace qemu_gettimeofday() with g_get_real_time()

Re: [PATCH 2/4] qtest: replace gettimeofday with GTimer

2022-03-04 Thread Laurent Vivier
Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : From: Marc-André Lureau glib provides a convenience helper to measure elapsed time. It isn't subject to wall-clock time changes. Note that this changes the initial OPENED time, which used to print the current time. Time is

Re: [PATCH 05/12] block/nbd: don't restrict TLS usage to IP sockets

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:23PM +, Daniel P. Berrangé wrote: > The TLS usage for NBD was restricted to IP sockets because validating > x509 certificates requires knowledge of the hostname that the client > is connecting to. > > TLS does not have to use x509 certificates though, as PSK

[PATCH 2/4] qtest: replace gettimeofday with GTimer

2022-03-04 Thread marcandre . lureau
From: Marc-André Lureau glib provides a convenience helper to measure elapsed time. It isn't subject to wall-clock time changes. Note that this changes the initial OPENED time, which used to print the current time. Signed-off-by: Marc-André Lureau --- softmmu/qtest.c | 39

Re: [PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

2022-03-04 Thread Laurent Vivier
Le 04/03/2022 à 17:08, Laurent Vivier a écrit : Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit : From: Marc-André Lureau GLib g_get_real_time() is an alternative to gettimeofday(). For semihosting, a few bits are lost on POSIX host, but this shouldn't be a big concern.

Re: [PATCH 1/5] python/utils: add add_visual_margin() text decoration utility

2022-03-04 Thread John Snow
On Thu, Mar 3, 2022, 5:43 PM Eric Blake wrote: > On Thu, Mar 03, 2022 at 03:58:58PM -0500, John Snow wrote: > > >>> print(add_visual_margin(msg, width=72, name="Commit Message")) > > ┏━ Commit Message ━━ > > ┃ add_visual_margin() takes a chunk

[PULL 13/50] iotests/185: Add post-READY quit tests

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz 185 tests quitting qemu while a block job is active. It does not specifically test quitting qemu while a mirror or active commit job is in its READY phase. Add two test cases for this, where we respectively mirror or commit to an external QSD instance, which provides a

[PULL 06/50] tls: add macros for coroutine-safe TLS variables

2022-03-04 Thread Kevin Wolf
From: Stefan Hajnoczi Compiler optimizations can cache TLS values across coroutine yield points, resulting in stale values from the previous thread when a coroutine is re-entered by a new thread. Serge Guelton developed an __attribute__((noinline)) wrapper and tested it with clang and gcc. I

[PULL 23/50] block.c: assertions to the block layer permissions API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Now that we "covered" the three main cases where the permission API was being used under BQL (fuse, amend and invalidate_cache), we can safely assert for the permission functions implemented in block.c Signed-off-by: Emanuele Giuseppe Esposito Message-Id:

[PULL 35/50] include/block/snapshot: global state API + assertions

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Snapshots run also under the BQL, so they all are in the global state API. The aiocontext lock that they hold is currently an overkill and in future could be removed. Signed-off-by: Emanuele Giuseppe Esposito Message-Id:

[PULL 48/50] tests/qemu-iotests: Rework the checks and spots using GNU sed

2022-03-04 Thread Kevin Wolf
From: Thomas Huth Instead of failing the iotests if GNU sed is not available (or skipping them completely in the check-block.sh script), it would be better to simply skip the bash-based tests that rely on GNU sed, so that the other tests could still be run. Thus we now explicitely use "gsed"

[PULL 30/50] block.c: add assertions to static functions

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Following the assertion derived from the API split, propagate the assertion also in the static functions. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-18-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- block.c

[PULL 37/50] block/coroutines: I/O and "I/O or GS" API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito block coroutines functions run in different aiocontext, and are not protected by the BQL. Therefore are I/O. On the other side, generated_co_wrapper functions use BDRV_POLL_WHILE, meaning the caller can either be the main loop or a specific iothread.

[PULL 31/50] include/block/blockjob.h: global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito blockjob functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-19-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- include/block/blockjob.h | 29 ++--- 1 file changed,

[PULL 41/50] block_int-common.h: assertions in the callers of BdrvChildClass function pointers

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-29-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- block.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/block.c b/block.c index

[PATCH v2 2/5] python/utils: add VerboseProcessError

2022-03-04 Thread John Snow
This adds an Exception that extends the Python stdlib subprocess.CalledProcessError. The difference is that the str() method of this exception also adds the stdout/stderr logs. In effect, if this exception goes unhandled, Python will print the output in a visually distinct wrapper to the terminal

Re: [PATCH v2 4/9] util: Return valid allocation for qemu_try_memalign() with zero size

2022-03-04 Thread Philippe Mathieu-Daudé
On 4/3/22 12:21, Peter Maydell wrote: Currently qemu_try_memalign()'s behaviour if asked to allocate 0 bytes is rather variable: * on Windows, we will assert * on POSIX platforms, we get the underlying behaviour of the posix_memalign() or equivalent function, which may be either

[PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

2022-03-04 Thread marcandre . lureau
From: Marc-André Lureau GLib g_get_real_time() is an alternative to gettimeofday(). For semihosting, a few bits are lost on POSIX host, but this shouldn't be a big concern. Signed-off-by: Marc-André Lureau --- blockdev.c | 8 hw/rtc/m41t80.c| 6 +++---

[PULL 10/50] os-posix: Add os_set_daemonize()

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz The daemonizing functions in os-posix (os_daemonize() and os_setup_post()) only daemonize the process if the static `daemonize` variable is set. Right now, it can only be set by os_parse_cmd_args(). In order to use os_daemonize() and os_setup_post() from the storage daemon to

[PULL 11/50] qsd: Add pre-init argument parsing pass

2022-03-04 Thread Kevin Wolf
From: Hanna Reitz In contrast to qemu-nbd (where it is called --fork) and the system emulator, QSD does not have a --daemonize switch yet. Just like them, QSD allows setting up block devices and exports on the command line. When doing so, it is often necessary for whoever invoked the QSD to

[PULL 15/50] main loop: macros to mark GS and I/O functions

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Righ now, IO_CODE and IO_OR_GS_CODE are nop, as there isn't really a way to check that a function is only called in I/O. On the other side, we can use qemu_in_main_thread() to check if we are in the main loop. The usage of macros makes easy to extend them in the

[PULL 19/50] block/export/fuse.c: allow writable exports to take RESIZE permission

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Allow writable exports to get BLK_PERM_RESIZE permission from creation, in fuse_export_create(). In this way, there is no need to give the permission in fuse_do_truncate(), which might be run in an iothread. Permissions should be set only in the main thread, so

[PULL 25/50] assertions for block_int global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-13-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- block.c | 15 +++ block/backup.c | 1 + block/block-backend.c |

[PULL 14/50] main-loop.h: introduce qemu_in_main_thread()

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito When invoked from the main loop, this function is the same as qemu_mutex_iothread_locked, and returns true if the BQL is held. When invoked from iothreads or tests, it returns true only if the current AioContext is the Main Loop. This essentially just extends

[PULL 16/50] include/block/block: split header into I/O and global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito block.h currently contains a mix of functions: some of them run under the BQL and modify the block layer graph, others are instead thread-safe and perform I/O in iothreads. Some others can only be called by either the main loop or the iothread running the

[PULL 24/50] include/block/block_int: split header into I/O and global state API

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Similarly to the previous patch, split block_int.h in block_int-io.h and block_int-global-state.h block_int-common.h contains the structures shared between the two headers, and the functions that can't be categorized as I/O or global state. Assertions are added

[PULL 29/50] GS and IO CODE macros for blockjob_int.h

2022-03-04 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220303151616.325444-17-eespo...@redhat.com> Signed-off-by: Kevin Wolf --- blockjob.c | 5 + 1 file changed, 5 insertions(+) diff --git a/blockjob.c b/blockjob.c index 10815a89fe..d79a52d204 100644

[PULL 07/50] util/async: replace __thread with QEMU TLS macros

2022-03-04 Thread Kevin Wolf
From: Stefan Hajnoczi QEMU TLS macros must be used to make TLS variables safe with coroutines. Signed-off-by: Stefan Hajnoczi Message-Id: <20220222140150.27240-3-stefa...@redhat.com> Signed-off-by: Kevin Wolf --- util/async.c | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-)

[PATCH v2 01/12] crypto: mandate a hostname when checking x509 creds on a client

2022-03-04 Thread Daniel P . Berrangé
Currently the TLS session object assumes that the caller will always provide a hostname when using x509 creds on a client endpoint. This relies on the caller to detect and report an error if the user has configured QEMU with x509 credentials on a UNIX socket. The migration code has such a check,

Re: [PATCH v2 8/9] util: Put qemu_vfree() in memalign.c

2022-03-04 Thread Philippe Mathieu-Daudé
On 4/3/22 12:21, Peter Maydell wrote: qemu_vfree() is the companion free function to qemu_memalign(); put it in memalign.c so the allocation and free functions are together. Signed-off-by: Peter Maydell Reviewed-by: Richard Henderson Message-id:

Re: [PATCH 09/12] tests/qemu-iotests: convert NBD TLS test to use standard filters

2022-03-04 Thread Eric Blake
On Thu, Mar 03, 2022 at 04:03:27PM +, Daniel P. Berrangé wrote: > Using standard filters is more future proof than rolling our own. > > Signed-off-by: Daniel P. Berrangé > --- > tests/qemu-iotests/233 | 29 - > tests/qemu-iotests/233.out | 9 - > 2

  1   2   >