Re: [PATCH 2/4] qtest: replace gettimeofday with GTimer

[Laurent Vivier]

Le 04/03/2022 à 16:54, Laurent Vivier a écrit :

Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :

From: Marc-André Lureau 

glib provides a convenience helper to measure elapsed time. It isn't
subject to wall-clock time changes.

Note that this changes the initial OPENED time, which used to print the
current time.



Time is printed using FMT_timeval ("%ld.%06ld"), but g_timer_elapsed() returns 
a gdouble.

Are you sure it works properly?


Sorry, I missed this part:


-#define FMT_timeval "%ld.%06ld"
+#define FMT_timeval "%.06f"


Reviewed-by: Laurent Vivier 

Re: [PATCH 4/4] oslib: drop qemu_gettimeofday()

[Stefan Weil]

Am 04.03.22 um 16:27 schrieb marcandre.lur...@redhat.com:


From: Marc-André Lureau 

No longer used after the previous patches.

Signed-off-by: Marc-André Lureau 
---
  include/sysemu/os-posix.h |  3 ---
  include/sysemu/os-win32.h |  6 --
  util/oslib-win32.c| 20 
  3 files changed, 29 deletions(-)

diff --git a/include/sysemu/os-posix.h b/include/sysemu/os-posix.h
index 2edf33658a44..c9b1d63fedda 100644
--- a/include/sysemu/os-posix.h
+++ b/include/sysemu/os-posix.h
@@ -52,9 +52,6 @@ int os_mlock(void);
  #define closesocket(s) close(s)
  #define ioctlsocket(s, r, v) ioctl(s, r, v)
  
-typedef struct timeval qemu_timeval;

-#define qemu_gettimeofday(tp) gettimeofday(tp, NULL)
-
  bool is_daemonized(void);
  
  /**

diff --git a/include/sysemu/os-win32.h b/include/sysemu/os-win32.h
index 43f569b5c216..4d4be826f48c 100644
--- a/include/sysemu/os-win32.h
+++ b/include/sysemu/os-win32.h
@@ -71,12 +71,6 @@ int getpagesize(void);
  # define EPROTONOSUPPORT EINVAL
  #endif
  
-typedef struct {

-long tv_sec;
-long tv_usec;
-} qemu_timeval;
-int qemu_gettimeofday(qemu_timeval *tp);
-
  static inline bool is_daemonized(void)
  {
  return false;
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index af559ef3398d..7faf59e9aaea 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -265,26 +265,6 @@ void qemu_set_cloexec(int fd)
  {
  }
  
-/* Offset between 1/1/1601 and 1/1/1970 in 100 nanosec units */

-#define _W32_FT_OFFSET (1164447360ULL)
-
-int qemu_gettimeofday(qemu_timeval *tp)
-{
-  union {
-unsigned long long ns100; /*time since 1 Jan 1601 in 100ns units */
-FILETIME ft;
-  }  _now;
-
-  if(tp) {
-  GetSystemTimeAsFileTime (&_now.ft);
-  tp->tv_usec=(long)((_now.ns100 / 10ULL) % 100ULL );
-  tp->tv_sec= (long)((_now.ns100 - _W32_FT_OFFSET) / 1000ULL);
-  }
-  /* Always return 0 as per Open Group Base Specifications Issue 6.
- Do not set errno on error.  */
-  return 0;
-}
-
  int qemu_get_thread_id(void)
  {
  return GetCurrentThreadId();



Reviewed-by: Stefan Weil 




OpenPGP_0xE08C21D5677450AD.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

[PULL 38/50] block_int-common.h: split function pointers in BlockDriver

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Similar to the header split, also the function pointers in BlockDriver
can be split in I/O and global state.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-26-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int-common.h | 445 ---
 1 file changed, 237 insertions(+), 208 deletions(-)

diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index b92e3630fd..f05ebb0da3 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -96,6 +96,11 @@ typedef struct BdrvTrackedRequest {
 
 
 struct BlockDriver {
+/*
+ * These fields are initialized when this object is created,
+ * and are never changed afterwards.
+ */
+
 const char *format_name;
 int instance_size;
 
@@ -122,6 +127,69 @@ struct BlockDriver {
  */
 bool is_format;
 
+/*
+ * Drivers not implementing bdrv_parse_filename nor bdrv_open should have
+ * this field set to true, except ones that are defined only by their
+ * child's bs.
+ * An example of the last type will be the quorum block driver.
+ */
+bool bdrv_needs_filename;
+
+/*
+ * Set if a driver can support backing files. This also implies the
+ * following semantics:
+ *
+ *  - Return status 0 of .bdrv_co_block_status means that corresponding
+ *blocks are not allocated in this layer of backing-chain
+ *  - For such (unallocated) blocks, read will:
+ *- fill buffer with zeros if there is no backing file
+ *- read from the backing file otherwise, where the block layer
+ *  takes care of reading zeros beyond EOF if backing file is short
+ */
+bool supports_backing;
+
+bool has_variable_length;
+
+/*
+ * Drivers setting this field must be able to work with just a plain
+ * filename with ':' as a prefix, and no other options.
+ * Options may be extracted from the filename by implementing
+ * bdrv_parse_filename.
+ */
+const char *protocol_name;
+
+/* List of options for creating images, terminated by name == NULL */
+QemuOptsList *create_opts;
+
+/* List of options for image amend */
+QemuOptsList *amend_opts;
+
+/*
+ * If this driver supports reopening images this contains a
+ * NULL-terminated list of the runtime options that can be
+ * modified. If an option in this list is unspecified during
+ * reopen then it _must_ be reset to its default value or return
+ * an error.
+ */
+const char *const *mutable_opts;
+
+/*
+ * Pointer to a NULL-terminated array of names of strong options
+ * that can be specified for bdrv_open(). A strong option is one
+ * that changes the data of a BDS.
+ * If this pointer is NULL, the array is considered empty.
+ * "filename" and "driver" are always considered strong.
+ */
+const char *const *strong_runtime_opts;
+
+
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 /*
  * This function is invoked under BQL before .bdrv_co_amend()
  * (which in contrast does not necessarily run under the BQL)
@@ -143,7 +211,6 @@ struct BlockDriver {
 bool (*bdrv_recurse_can_replace)(BlockDriverState *bs,
  BlockDriverState *to_replace);
 
-int (*bdrv_probe)(const uint8_t *buf, int buf_size, const char *filename);
 int (*bdrv_probe_device)(const char *filename);
 
 /*
@@ -152,28 +219,8 @@ struct BlockDriver {
  */
 void (*bdrv_parse_filename)(const char *filename, QDict *options,
 Error **errp);
-/*
- * Drivers not implementing bdrv_parse_filename nor bdrv_open should have
- * this field set to true, except ones that are defined only by their
- * child's bs.
- * An example of the last type will be the quorum block driver.
- */
-bool bdrv_needs_filename;
-
-/*
- * Set if a driver can support backing files. This also implies the
- * following semantics:
- *
- *  - Return status 0 of .bdrv_co_block_status means that corresponding
- *blocks are not allocated in this layer of backing-chain
- *  - For such (unallocated) blocks, read will:
- *- fill buffer with zeros if there is no backing file
- *- read from the backing file otherwise, where the block layer
- *  takes care of reading zeros beyond EOF if backing file is short
- */
-bool supports_backing;
 
-/* For handling image reopen for split or non-split files */
+/* For handling image reopen for split or non-split files. */
 int (*bdrv_reopen_prepare)(BDRVReopenState *reopen_state,
BlockReopenQueue *queue, Error **errp);
 void 

[PULL 39/50] block_int-common.h: assertions in the callers of BlockDriver function pointers

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-27-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c| 17 +
 block/create.c |  2 ++
 2 files changed, 19 insertions(+)

diff --git a/block.c b/block.c
index b91f9b2d11..e5ba40d028 100644
--- a/block.c
+++ b/block.c
@@ -529,6 +529,7 @@ static void coroutine_fn bdrv_create_co_entry(void *opaque)
 
 CreateCo *cco = opaque;
 assert(cco->drv);
+GLOBAL_STATE_CODE();
 
 ret = cco->drv->bdrv_co_create_opts(cco->drv,
 cco->filename, cco->opts, _err);
@@ -1096,6 +1097,7 @@ int refresh_total_sectors(BlockDriverState *bs, int64_t 
hint)
 static void bdrv_join_options(BlockDriverState *bs, QDict *options,
   QDict *old_options)
 {
+GLOBAL_STATE_CODE();
 if (bs->drv && bs->drv->bdrv_join_options) {
 bs->drv->bdrv_join_options(options, old_options);
 } else {
@@ -1605,6 +1607,7 @@ static int bdrv_open_driver(BlockDriverState *bs, 
BlockDriver *drv,
 {
 Error *local_err = NULL;
 int i, ret;
+GLOBAL_STATE_CODE();
 
 bdrv_assign_node_name(bs, node_name, _err);
 if (local_err) {
@@ -1996,6 +1999,8 @@ static int bdrv_fill_options(QDict **options, const char 
*filename,
 BlockDriver *drv = NULL;
 Error *local_err = NULL;
 
+GLOBAL_STATE_CODE();
+
 /*
  * Caution: while qdict_get_try_str() is fine, getting non-string
  * types would require more care.  When @options come from
@@ -2192,6 +2197,7 @@ static void bdrv_child_perm(BlockDriverState *bs, 
BlockDriverState *child_bs,
 uint64_t *nperm, uint64_t *nshared)
 {
 assert(bs->drv && bs->drv->bdrv_child_perm);
+GLOBAL_STATE_CODE();
 bs->drv->bdrv_child_perm(bs, c, role, reopen_queue,
  parent_perm, parent_shared,
  nperm, nshared);
@@ -2280,6 +2286,7 @@ static void bdrv_drv_set_perm_commit(void *opaque)
 {
 BlockDriverState *bs = opaque;
 uint64_t cumulative_perms, cumulative_shared_perms;
+GLOBAL_STATE_CODE();
 
 if (bs->drv->bdrv_set_perm) {
 bdrv_get_cumulative_perm(bs, _perms,
@@ -2291,6 +2298,7 @@ static void bdrv_drv_set_perm_commit(void *opaque)
 static void bdrv_drv_set_perm_abort(void *opaque)
 {
 BlockDriverState *bs = opaque;
+GLOBAL_STATE_CODE();
 
 if (bs->drv->bdrv_abort_perm_update) {
 bs->drv->bdrv_abort_perm_update(bs);
@@ -2306,6 +2314,7 @@ static int bdrv_drv_set_perm(BlockDriverState *bs, 
uint64_t perm,
  uint64_t shared_perm, Transaction *tran,
  Error **errp)
 {
+GLOBAL_STATE_CODE();
 if (!bs->drv) {
 return 0;
 }
@@ -4372,6 +4381,7 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, 
Error **errp)
 
 assert(qemu_get_current_aio_context() == qemu_get_aio_context());
 assert(bs_queue != NULL);
+GLOBAL_STATE_CODE();
 
 QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
 ctx = bdrv_get_aio_context(bs_entry->state.bs);
@@ -4637,6 +4647,7 @@ static int bdrv_reopen_prepare(BDRVReopenState 
*reopen_state,
 
 assert(reopen_state != NULL);
 assert(reopen_state->bs->drv != NULL);
+GLOBAL_STATE_CODE();
 drv = reopen_state->bs->drv;
 
 /* This function and each driver's bdrv_reopen_prepare() remove
@@ -4847,6 +4858,7 @@ static void bdrv_reopen_commit(BDRVReopenState 
*reopen_state)
 bs = reopen_state->bs;
 drv = bs->drv;
 assert(drv != NULL);
+GLOBAL_STATE_CODE();
 
 /* If there are any driver level actions to take */
 if (drv->bdrv_reopen_commit) {
@@ -4888,6 +4900,7 @@ static void bdrv_reopen_abort(BDRVReopenState 
*reopen_state)
 assert(reopen_state != NULL);
 drv = reopen_state->bs->drv;
 assert(drv != NULL);
+GLOBAL_STATE_CODE();
 
 if (drv->bdrv_reopen_abort) {
 drv->bdrv_reopen_abort(reopen_state);
@@ -6723,6 +6736,8 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs)
 int ret;
 uint64_t cumulative_perms, cumulative_shared_perms;
 
+GLOBAL_STATE_CODE();
+
 if (!bs->drv) {
 return -ENOMEDIUM;
 }
@@ -7237,6 +7252,7 @@ static void bdrv_detach_aio_context(BlockDriverState *bs)
 BdrvAioNotifier *baf, *baf_tmp;
 
 assert(!bs->walking_aio_notifiers);
+GLOBAL_STATE_CODE();
 bs->walking_aio_notifiers = true;
 QLIST_FOREACH_SAFE(baf, >aio_notifiers, list, baf_tmp) {
 if (baf->deleted) {
@@ -7264,6 +7280,7 @@ static void bdrv_attach_aio_context(BlockDriverState *bs,
 AioContext *new_context)
 {
 BdrvAioNotifier *ban, *ban_tmp;
+GLOBAL_STATE_CODE();
 
 if (bs->quiesce_counter) {
 aio_disable_external(new_context);
diff --git a/block/create.c b/block/create.c
index 89812669df..4df43f11f4 100644
--- a/block/create.c
+++ b/block/create.c
@@ -42,6 +42,8 @@ static 

Re: [PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

[Marc-André Lureau]

Hi

On Fri, Mar 4, 2022 at 8:10 PM Laurent Vivier  wrote:

> Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :
> > From: Marc-André Lureau 
> >
> > GLib g_get_real_time() is an alternative to gettimeofday().
> >
> > For semihosting, a few bits are lost on POSIX host, but this shouldn't
> > be a big concern.
> >
> > Signed-off-by: Marc-André Lureau 
> > ---
> >   blockdev.c |  8 
> >   hw/rtc/m41t80.c|  6 +++---
> >   hw/virtio/virtio-balloon.c |  9 +
> >   qapi/qmp-event.c   | 12 +---
> >   qemu-img.c |  8 
> >   qga/commands-posix.c   | 11 +--
> >   target/m68k/m68k-semi.c| 22 ++
> >   target/nios2/nios2-semi.c  | 24 +++-
> >   8 files changed, 39 insertions(+), 61 deletions(-)
> >
> ...
> > diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> > index 75dbaab68ea9..3311a4ca0167 100644
> > --- a/qga/commands-posix.c
> > +++ b/qga/commands-posix.c
> > @@ -138,16 +138,7 @@ void qmp_guest_shutdown(bool has_mode, const char
> *mode, Error **errp)
> >
> >   int64_t qmp_guest_get_time(Error **errp)
> >   {
> > -   int ret;
> > -   qemu_timeval tq;
> > -
> > -   ret = qemu_gettimeofday();
> > -   if (ret < 0) {
> > -   error_setg_errno(errp, errno, "Failed to get time");
> > -   return -1;
> > -   }
> > -
> > -   return tq.tv_sec * 10LL + tq.tv_usec * 1000;
> > +return g_get_real_time();
>
> now you return microseconds, before it was nanoseconds.
>
>
my bad, will fix it in v2, thanks for noticing!


> qga/qapi-schema.json:
>
> ##
> # @guest-get-time:
> #
> # Get the information about guest's System Time relative to
> # the Epoch of 1970-01-01 in UTC.
> #
> # Returns: Time in nanoseconds.
> #
> # Since: 1.5
> ##
> { 'command': 'guest-get-time',
>'returns': 'int' }
>
> Except this problem:
>
> Reviewed-by: Laurent Vivier 
>
>

-- 
Marc-André Lureau

[PATCH v2 11/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets

[Daniel P . Berrangé]

This validates that connections to an NBD server running on a UNIX
socket can use TLS, and require a TLS hostname override to pass
certificate validation.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 tests/qemu-iotests/233 | 24 
 tests/qemu-iotests/233.out | 15 +++
 2 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index c24d877be8..442fd1378c 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -167,6 +167,30 @@ $QEMU_IMG info --image-opts \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
 2>&1 | _filter_nbd
 
+nbd_server_stop
+
+nbd_server_start_unix_socket \
+--object 
tls-creds-x509,dir=${tls_dir}/server1,endpoint=server,id=tls0,verify-peer=on \
+--tls-creds tls0 \
+-f $IMGFMT "$TEST_IMG" 2>> "$TEST_DIR/server.log"
+
+echo
+echo "== check TLS fail over UNIX with no hostname =="
+obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj1 \
+driver=nbd,path=$nbd_unix_socket,tls-creds=tls0 2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -k $nbd_unix_socket --object $obj1 --tls-creds=tls0 \
+2>&1 | _filter_qemu_nbd_exports
+
+echo
+echo "== check TLS works over UNIX with hostname override =="
+obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj1 \
+driver=nbd,path=$nbd_unix_socket,tls-creds=tls0,tls-hostname=127.0.0.1 \
+2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -k $nbd_unix_socket --object $obj1 \
+--tls-creds=tls0 --tls-hostname=127.0.0.1  2>&1 | _filter_qemu_nbd_exports
+
 echo
 echo "== final server log =="
 cat "$TEST_DIR/server.log" | _filter_authz_check_tls
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index d42611bf74..d79a9ed346 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -68,6 +68,19 @@ read 1048576/1048576 bytes at offset 1048576
 qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': 
Failed to read option reply: Cannot read from TLS channel: Software caused 
connection abort
 qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': 
Failed to read option reply: Cannot read from TLS channel: Software caused 
connection abort
 
+== check TLS fail over UNIX with no hostname ==
+qemu-img: Could not open 
'driver=nbd,path=SOCK_DIR/qemu-nbd.sock,tls-creds=tls0': No hostname for 
certificate validation
+qemu-nbd: No hostname for certificate validation
+
+== check TLS works over UNIX with hostname override ==
+image: nbd+unix://?socket=SOCK_DIR/qemu-nbd.sock
+file format: nbd
+virtual size: 64 MiB (67108864 bytes)
+disk size: unavailable
+exports available: 1
+  size:  67108864
+  min block: 1
+
 == final server log ==
 qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
 qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
@@ -75,4 +88,6 @@ qemu-nbd: option negotiation failed: Verify failed: No 
certificate was found.
 qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
 qemu-nbd: option negotiation failed: TLS x509 authz check for 
DISTINGUISHED-NAME is denied
 qemu-nbd: option negotiation failed: TLS x509 authz check for 
DISTINGUISHED-NAME is denied
+qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
+qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
 *** done
-- 
2.34.1

[PATCH v2 02/12] block: pass desired TLS hostname through from block driver client

[Daniel P . Berrangé]

In

  commit a71d597b989fd701b923f09b3c20ac4fcaa55e81
  Author: Vladimir Sementsov-Ogievskiy 
  Date:   Thu Jun 10 13:08:00 2021 +0300

block/nbd: reuse nbd_co_do_establish_connection() in nbd_open()

the use of the 'hostname' field from the BDRVNBDState struct was
lost, and 'nbd_connect' just hardcoded it to match the IP socket
address. This was a harmless bug at the time since we block use
with anything other than IP sockets.

Shortly though, we want to allow the caller to override the hostname
used in the TLS certificate checks. This is to allow for TLS
when doing port forwarding or tunneling. Thus we need to reinstate
the passing along of the 'hostname'.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 block/nbd.c |  7 ---
 include/block/nbd.h |  3 ++-
 nbd/client-connection.c | 12 +---
 3 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/block/nbd.c b/block/nbd.c
index 5853d85d60..dd43929207 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -92,7 +92,7 @@ typedef struct BDRVNBDState {
 SocketAddress *saddr;
 char *export, *tlscredsid;
 QCryptoTLSCreds *tlscreds;
-const char *hostname;
+const char *tlshostname;
 char *x_dirty_bitmap;
 bool alloc_depth;
 
@@ -1835,7 +1835,7 @@ static int nbd_process_options(BlockDriverState *bs, 
QDict *options,
 error_setg(errp, "TLS only supported over IP sockets");
 goto error;
 }
-s->hostname = s->saddr->u.inet.host;
+s->tlshostname = s->saddr->u.inet.host;
 }
 
 s->x_dirty_bitmap = g_strdup(qemu_opt_get(opts, "x-dirty-bitmap"));
@@ -1875,7 +1875,8 @@ static int nbd_open(BlockDriverState *bs, QDict *options, 
int flags,
 }
 
 s->conn = nbd_client_connection_new(s->saddr, true, s->export,
-s->x_dirty_bitmap, s->tlscreds);
+s->x_dirty_bitmap, s->tlscreds,
+s->tlshostname);
 
 if (s->open_timeout) {
 nbd_client_connection_enable_retry(s->conn);
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 78d101b774..a98eb665da 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -415,7 +415,8 @@ NBDClientConnection *nbd_client_connection_new(const 
SocketAddress *saddr,
bool do_negotiation,
const char *export_name,
const char *x_dirty_bitmap,
-   QCryptoTLSCreds *tlscreds);
+   QCryptoTLSCreds *tlscreds,
+   const char *tlshostname);
 void nbd_client_connection_release(NBDClientConnection *conn);
 
 QIOChannel *coroutine_fn
diff --git a/nbd/client-connection.c b/nbd/client-connection.c
index 2bda42641d..2a632931c3 100644
--- a/nbd/client-connection.c
+++ b/nbd/client-connection.c
@@ -33,6 +33,7 @@ struct NBDClientConnection {
 /* Initialization constants, never change */
 SocketAddress *saddr; /* address to connect to */
 QCryptoTLSCreds *tlscreds;
+char *tlshostname;
 NBDExportInfo initial_info;
 bool do_negotiation;
 bool do_retry;
@@ -77,7 +78,8 @@ NBDClientConnection *nbd_client_connection_new(const 
SocketAddress *saddr,
bool do_negotiation,
const char *export_name,
const char *x_dirty_bitmap,
-   QCryptoTLSCreds *tlscreds)
+   QCryptoTLSCreds *tlscreds,
+   const char *tlshostname)
 {
 NBDClientConnection *conn = g_new(NBDClientConnection, 1);
 
@@ -85,6 +87,7 @@ NBDClientConnection *nbd_client_connection_new(const 
SocketAddress *saddr,
 *conn = (NBDClientConnection) {
 .saddr = QAPI_CLONE(SocketAddress, saddr),
 .tlscreds = tlscreds,
+.tlshostname = g_strdup(tlshostname),
 .do_negotiation = do_negotiation,
 
 .initial_info.request_sizes = true,
@@ -107,6 +110,7 @@ static void 
nbd_client_connection_do_free(NBDClientConnection *conn)
 }
 error_free(conn->err);
 qapi_free_SocketAddress(conn->saddr);
+g_free(conn->tlshostname);
 object_unref(OBJECT(conn->tlscreds));
 g_free(conn->initial_info.x_dirty_bitmap);
 g_free(conn->initial_info.name);
@@ -120,6 +124,7 @@ static void 
nbd_client_connection_do_free(NBDClientConnection *conn)
  */
 static int nbd_connect(QIOChannelSocket *sioc, SocketAddress *addr,
NBDExportInfo *info, QCryptoTLSCreds *tlscreds,
+   const char *tlshostname,
QIOChannel **outioc, Error **errp)
 {
 int ret;
@@ -140,7 +145,7 @@ static int 

[PATCH v2 10/12] tests/qemu-iotests: validate NBD TLS with hostname mismatch

[Daniel P . Berrangé]

This validates that connections to an NBD server where the certificate
hostname does not match will fail. It further validates that using the
new 'tls-hostname' override option can solve the failure.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 tests/qemu-iotests/233| 18 ++
 tests/qemu-iotests/233.out| 16 
 tests/qemu-iotests/common.tls |  7 ---
 3 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index 050267298d..c24d877be8 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -106,6 +106,24 @@ $QEMU_IMG info --image-opts --object $obj2 \
 $QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj1 \
 --tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
 
+echo
+echo "== check TLS fail over TCP with mismatched hostname =="
+obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj1 \
+driver=nbd,host=localhost,port=$nbd_tcp_port,tls-creds=tls0 \
+2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -b localhost -p $nbd_tcp_port --object $obj1 \
+--tls-creds=tls0 | _filter_qemu_nbd_exports
+
+echo
+echo "== check TLS works over TCP with mismatched hostname and override =="
+obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj1 \
+
driver=nbd,host=localhost,port=$nbd_tcp_port,tls-creds=tls0,tls-hostname=127.0.0.1
 \
+2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -b localhost -p $nbd_tcp_port --object $obj1 \
+--tls-creds=tls0 --tls-hostname=127.0.0.1 | _filter_qemu_nbd_exports
+
 echo
 echo "== check TLS with different CA fails =="
 obj=tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 67a027d879..d42611bf74 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -38,6 +38,20 @@ exports available: 1
   size:  67108864
   min block: 1
 
+== check TLS fail over TCP with mismatched hostname ==
+qemu-img: Could not open 'driver=nbd,host=localhost,port=PORT,tls-creds=tls0': 
Certificate does not match the hostname localhost
+qemu-nbd: Certificate does not match the hostname localhost
+
+== check TLS works over TCP with mismatched hostname and override ==
+image: nbd://localhost:PORT
+file format: nbd
+virtual size: 64 MiB (67108864 bytes)
+disk size: unavailable
+exports available: 1
+ export: ''
+  size:  67108864
+  min block: 1
+
 == check TLS with different CA fails ==
 qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': 
The certificate hasn't got a known issuer
 qemu-nbd: The certificate hasn't got a known issuer
@@ -55,6 +69,8 @@ qemu-img: Could not open 
'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': F
 qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': 
Failed to read option reply: Cannot read from TLS channel: Software caused 
connection abort
 
 == final server log ==
+qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
+qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
 qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
 qemu-nbd: option negotiation failed: Verify failed: No certificate was found.
 qemu-nbd: option negotiation failed: TLS x509 authz check for 
DISTINGUISHED-NAME is denied
diff --git a/tests/qemu-iotests/common.tls b/tests/qemu-iotests/common.tls
index 6ba28a78d3..4a5760949d 100644
--- a/tests/qemu-iotests/common.tls
+++ b/tests/qemu-iotests/common.tls
@@ -118,12 +118,13 @@ tls_x509_create_server()
 caname=$1
 name=$2
 
+# We don't include 'localhost' in the cert, as
+# we want to keep it unlisted to let tests
+# validate hostname override
 mkdir -p "${tls_dir}/$name"
 cat > "${tls_dir}/cert.info" <

[PATCH v2 07/12] tests/qemu-iotests: expand _filter_nbd rules

[Daniel P . Berrangé]

Some tests will want to use 'localhost' instead of '127.0.0.1', and
some will use the image options syntax rather than the classic URI
syntax.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 tests/qemu-iotests/common.filter | 4 
 1 file changed, 4 insertions(+)

diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 75cc241580..25d1d22929 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -300,6 +300,10 @@ _filter_nbd()
 # Filter out the TCP port number since this changes between runs.
 $SED -e '/nbd\/.*\.c:/d' \
 -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \
+-e 's#localhost:[0-9]*#localhost:PORT#g' \
+-e 's#host=127\.0\.0\.1,port=[0-9]*#host=127.0.0.1,port=PORT#g' \
+-e 's#host=localhost,port=[0-9]*#host=localhost,port=PORT#g' \
+-e "s#path=$SOCK_DIR#path=SOCK_DIR#g" \
 -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \
 -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#'
 }
-- 
2.34.1

[PATCH v2 3/5] iotests: Remove explicit checks for qemu_img() == 0

[John Snow]

qemu_img() returning zero ought to be the rule, not the
exception. Remove all explicit checks against the condition in
preparation for making non-zero returns an Exception.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/163 |  9 +++--
 tests/qemu-iotests/216 |  6 +++---
 tests/qemu-iotests/218 |  2 +-
 tests/qemu-iotests/224 | 11 +--
 tests/qemu-iotests/228 | 12 ++--
 tests/qemu-iotests/257 |  3 +--
 tests/qemu-iotests/258 |  4 ++--
 tests/qemu-iotests/310 | 14 +++---
 tests/qemu-iotests/tests/block-status-cache|  3 +--
 tests/qemu-iotests/tests/graph-changes-while-io|  7 +++
 tests/qemu-iotests/tests/image-fleecing|  4 ++--
 tests/qemu-iotests/tests/mirror-ready-cancel-error |  6 ++
 tests/qemu-iotests/tests/mirror-top-perms  |  3 +--
 .../qemu-iotests/tests/remove-bitmap-from-backing  |  8 
 tests/qemu-iotests/tests/stream-error-on-reset |  4 ++--
 15 files changed, 43 insertions(+), 53 deletions(-)

diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163
index b8bfc95358..e4cd4b230f 100755
--- a/tests/qemu-iotests/163
+++ b/tests/qemu-iotests/163
@@ -107,8 +107,7 @@ class ShrinkBaseClass(iotests.QMPTestCase):
 
 if iotests.imgfmt == 'raw':
 return
-self.assertEqual(qemu_img('check', test_img), 0,
- "Verifying image corruption")
+qemu_img('check', test_img)
 
 def test_empty_image(self):
 qemu_img('resize',  '-f', iotests.imgfmt, '--shrink', test_img,
@@ -130,8 +129,7 @@ class ShrinkBaseClass(iotests.QMPTestCase):
 qemu_img('resize',  '-f', iotests.imgfmt, '--shrink', test_img,
  self.shrink_size)
 
-self.assertEqual(qemu_img("compare", test_img, check_img), 0,
- "Verifying image content")
+qemu_img("compare", test_img, check_img)
 
 self.image_verify()
 
@@ -146,8 +144,7 @@ class ShrinkBaseClass(iotests.QMPTestCase):
 qemu_img('resize',  '-f', iotests.imgfmt, '--shrink', test_img,
  self.shrink_size)
 
-self.assertEqual(qemu_img("compare", test_img, check_img), 0,
- "Verifying image content")
+qemu_img("compare", test_img, check_img)
 
 self.image_verify()
 
diff --git a/tests/qemu-iotests/216 b/tests/qemu-iotests/216
index c02f8d2880..88b385afa3 100755
--- a/tests/qemu-iotests/216
+++ b/tests/qemu-iotests/216
@@ -51,10 +51,10 @@ with iotests.FilePath('base.img') as base_img_path, \
 log('--- Setting up images ---')
 log('')
 
-assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
+qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M')
 assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0
-assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
-'-F', iotests.imgfmt, top_img_path) == 0
+qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
+ '-F', iotests.imgfmt, top_img_path)
 assert qemu_io_silent(top_img_path,  '-c', 'write -P 2 1M 1M') == 0
 
 log('Done')
diff --git a/tests/qemu-iotests/218 b/tests/qemu-iotests/218
index 4922b4d3b6..853ed52b34 100755
--- a/tests/qemu-iotests/218
+++ b/tests/qemu-iotests/218
@@ -145,7 +145,7 @@ log('')
 with iotests.VM() as vm, \
  iotests.FilePath('src.img') as src_img_path:
 
-assert qemu_img('create', '-f', iotests.imgfmt, src_img_path, '64M') == 0
+qemu_img('create', '-f', iotests.imgfmt, src_img_path, '64M')
 assert qemu_io_silent('-f', iotests.imgfmt, src_img_path,
   '-c', 'write -P 42 0M 64M') == 0
 
diff --git a/tests/qemu-iotests/224 b/tests/qemu-iotests/224
index 38dd153625..c31c55b49d 100755
--- a/tests/qemu-iotests/224
+++ b/tests/qemu-iotests/224
@@ -47,12 +47,11 @@ for filter_node_name in False, True:
  iotests.FilePath('top.img') as top_img_path, \
  iotests.VM() as vm:
 
-assert qemu_img('create', '-f', iotests.imgfmt,
-base_img_path, '64M') == 0
-assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
-'-F', iotests.imgfmt, mid_img_path) == 0
-assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path,
-'-F', iotests.imgfmt, top_img_path) == 0
+qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M')
+qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
+ '-F', iotests.imgfmt, mid_img_path)
+qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path,
+ '-F', iotests.imgfmt, top_img_path)
 
 # Something to commit
 assert 

Re: [PATCH v2 3/9] util: Unify implementations of qemu_memalign()

[Philippe Mathieu-Daudé]

On 4/3/22 12:21, Peter Maydell wrote:

We implement qemu_memalign() in both oslib-posix.c and oslib-win32.c,
but the two versions are essentially the same: they call
qemu_try_memalign(), and abort() after printing an error message if
it fails.  The only difference is that the win32 version prints the
GetLastError() value whereas the POSIX version prints
strerror(errno).  However, this is a bug in the win32 version: in
commit dfbd0b873a85021 in 2020 we changed the implementation of
qemu_try_memalign() from using VirtualAlloc() (which sets the
GetLastError() value) to using _aligned_malloc() (which sets errno),
but didn't update the error message to match.

Replace the two separate functions with a single version in a
new memalign.c file, which drops the unnecessary extra qemu_oom_check()
function and instead prints a more useful message including the
requested size and alignment as well as the errno string.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-id: 20220226180723.1706285-4-peter.mayd...@linaro.org
---
  util/memalign.c| 39 +++
  util/oslib-posix.c | 14 --
  util/oslib-win32.c | 14 --
  util/meson.build   |  1 +
  4 files changed, 40 insertions(+), 28 deletions(-)
  create mode 100644 util/memalign.c


Reviewed-by: Philippe Mathieu-Daudé 

[PATCH 5/5] iotests/296: Accept early failure

[Hanna Reitz]

As of "crypto: perform permission checks under BQL", permission failures
occur early in the blockdev-amend job to update a LUKS volume's keys.
Expect them in x-blockdev-amend's QMP reply instead of waiting for the
actual job to fail later.

(Note that the job will still be created, so we need to wait for it to
disappear even when the QMP command failed.  Otherwise, the job ID
"job0" will remain in use and we cannot launch another job with the same
ID.)

Signed-off-by: Hanna Reitz 
---
 tests/qemu-iotests/296 |  8 ++--
 tests/qemu-iotests/296.out | 17 +
 2 files changed, 11 insertions(+), 14 deletions(-)

diff --git a/tests/qemu-iotests/296 b/tests/qemu-iotests/296
index 099a3eeaa5..f80ef3434a 100755
--- a/tests/qemu-iotests/296
+++ b/tests/qemu-iotests/296
@@ -174,8 +174,12 @@ class EncryptionSetupTestCase(iotests.QMPTestCase):
 }
 
 result = vm.qmp('x-blockdev-amend', **args)
-assert result['return'] == {}
-vm.run_job('job0')
+iotests.log(result)
+# Run the job only if it was created
+event = ('JOB_STATUS_CHANGE',
+ {'data': {'id': 'job0', 'status': 'created'}})
+if vm.events_wait([event], timeout=0.0) is not None:
+vm.run_job('job0')
 
 # test that when the image opened by two qemu processes,
 # neither of them can update the encryption keys
diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
index 42205cc981..609826eaa0 100644
--- a/tests/qemu-iotests/296.out
+++ b/tests/qemu-iotests/296.out
@@ -1,11 +1,9 @@
 
-{"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
-Job failed: Failed to get shared "consistent read" lock
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
-Job failed: Failed to get shared "consistent read" lock
-{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"error": {"class": "GenericError", "desc": "Failed to get shared \"consistent 
read\" lock"}}
+{"error": {"class": "GenericError", "desc": "Failed to get shared \"consistent 
read\" lock"}}
 {"return": {}}
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
@@ -13,14 +11,9 @@ qemu-img: Failed to get shared "consistent read" lock
 Is another process using the image [TEST_DIR/test.img]?
 
 .
-Job failed: Block node is read-only
-{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-{"return": {}}
-Job failed: Failed to get shared "consistent read" lock
-{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-{"return": {}}
-Job failed: Failed to get shared "consistent read" lock
-{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"error": {"class": "GenericError", "desc": "Block node is read-only"}}
+{"error": {"class": "GenericError", "desc": "Failed to get shared \"consistent 
read\" lock"}}
+{"error": {"class": "GenericError", "desc": "Failed to get shared \"consistent 
read\" lock"}}
 {"return": {}}
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
-- 
2.34.1

[PATCH 0/5] block/amend: Fix failures seen in iotest 296

[Hanna Reitz]

Hi,

I’ve tried basing my block branch on Kevin’s and noticed that after
“crypto: perform permission checks under BQL”, iotest 296 was failing.
I/We have debugged those failures and here are fixes for it.

Hence, this series is based on Kevin’s block branch
(efa33ed9b298d39e2b8c19c5f4bdd80a3b632260 at the time of writing this
cover letter).  I’ve pushed it here:

  https://gitlab.com/hreitz/qemu/-/commits/amend-job-fixes-v1

Patch 1 adds clean-up of the amend job in an error path that said commit
adds to qmp_x_blockdev_amend().

Patch 2 changes the type of a JobDriver callback added in that commit;
together with patch 3, this is kind of a matter of style, but it can
also replace patch 3 and fix the bug that it fixes in another way.

Patch 3 fixes a permission bug: When changing the permissions fails
before amend, block/crypto will still keep updating_keys to be true.
Without patch 2, that will remains so indefinitely and then
block_crypto_child_perms() will continue to unshare the CONSISTENT_READ
permission, which is wrong.  (Patch 2 fixes this problem, too,
specifically because with it, block_crypto_amend_cleanup() will always
be called when the job is dismissed, and so updating_keys will be reset
at least then.)

Patch 4 fixes an issue that’s not related to “crypto: perform permission
checks under BQL”, but it became appearent only while debugging the
other issues here, so it’s part of this series, too.

Patch 5 fixes the test itself.  It expects permission-related errors to
occur when the job is already running, not as an immediate result of the
QMP x-blockdev-amend command.  “crypto: perform permission checks under
BQL” has changed this, so the test needs to take that into account.


Ideally, I believe the following patches should be squashed into
“crypto: perform permission checks under BQL” lest bisect breaks:
- Patch 1
- Patch 2 or 3 (or both)
- Patch 5

But if that isn’t feasible, we can just take the whole series on top.


Hanna Reitz (5):
  block/amend: Clean up job on early failure
  block/amend: Always call .bdrv_amend_clean()
  block/crypto: Reset updating_keys on perm failure
  block/amend: Keep strong reference to BDS
  iotests/296: Accept early failure

 block/amend.c  |  8 ++--
 block/crypto.c |  8 +++-
 tests/qemu-iotests/296 |  8 ++--
 tests/qemu-iotests/296.out | 17 +
 4 files changed, 24 insertions(+), 17 deletions(-)

-- 
2.34.1

[PULL 04/50] block: rename bdrv_invalidate_cache_all, blk_invalidate_cache and test_sync_op_invalidate_cache

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Following the bdrv_activate renaming, change also the name
of the respective callers.

bdrv_invalidate_cache_all -> bdrv_activate_all
blk_invalidate_cache -> blk_activate
test_sync_op_invalidate_cache -> test_sync_op_activate

No functional change intended.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Juan Quintela 
Reviewed-by: Hanna Reitz 
Message-Id: <20220209105452.1694545-5-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block.h|  2 +-
 include/sysemu/block-backend.h   |  2 +-
 block.c  |  2 +-
 block/block-backend.c|  2 +-
 hw/block/pflash_cfi01.c  |  2 +-
 hw/nvram/spapr_nvram.c   |  2 +-
 migration/block.c|  2 +-
 migration/migration.c| 14 +++---
 migration/savevm.c   |  6 +++---
 monitor/qmp-cmds.c   |  2 +-
 tests/unit/test-block-iothread.c |  6 +++---
 11 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 18cd336911..d27e616d29 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -493,8 +493,8 @@ int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf);
 /* Invalidate any cached metadata used by image formats */
 int generated_co_wrapper bdrv_invalidate_cache(BlockDriverState *bs,
Error **errp);
-void bdrv_invalidate_cache_all(Error **errp);
 int bdrv_activate(BlockDriverState *bs, Error **errp);
+void bdrv_activate_all(Error **errp);
 int bdrv_inactivate_all(void);
 
 /* Ensure contents are flushed to disk.  */
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index e5e1524f06..6c5104cd4a 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -206,7 +206,7 @@ bool blk_is_writable(BlockBackend *blk);
 bool blk_is_sg(BlockBackend *blk);
 bool blk_enable_write_cache(BlockBackend *blk);
 void blk_set_enable_write_cache(BlockBackend *blk, bool wce);
-void blk_invalidate_cache(BlockBackend *blk, Error **errp);
+void blk_activate(BlockBackend *blk, Error **errp);
 bool blk_is_inserted(BlockBackend *blk);
 bool blk_is_available(BlockBackend *blk);
 void blk_lock_medium(BlockBackend *blk, bool locked);
diff --git a/block.c b/block.c
index 7746727a47..5e65f134f8 100644
--- a/block.c
+++ b/block.c
@@ -6473,7 +6473,7 @@ int coroutine_fn 
bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 return 0;
 }
 
-void bdrv_invalidate_cache_all(Error **errp)
+void bdrv_activate_all(Error **errp)
 {
 BlockDriverState *bs;
 BdrvNextIterator it;
diff --git a/block/block-backend.c b/block/block-backend.c
index c516113a36..98bfcd5cf2 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1880,7 +1880,7 @@ void blk_set_enable_write_cache(BlockBackend *blk, bool 
wce)
 blk->enable_write_cache = wce;
 }
 
-void blk_invalidate_cache(BlockBackend *blk, Error **errp)
+void blk_activate(BlockBackend *blk, Error **errp)
 {
 BlockDriverState *bs = blk_bs(blk);
 
diff --git a/hw/block/pflash_cfi01.c b/hw/block/pflash_cfi01.c
index 81f9f971d8..74c7190302 100644
--- a/hw/block/pflash_cfi01.c
+++ b/hw/block/pflash_cfi01.c
@@ -1023,7 +1023,7 @@ static void postload_update_cb(void *opaque, bool 
running, RunState state)
 {
 PFlashCFI01 *pfl = opaque;
 
-/* This is called after bdrv_invalidate_cache_all.  */
+/* This is called after bdrv_activate_all.  */
 qemu_del_vm_change_state_handler(pfl->vmstate);
 pfl->vmstate = NULL;
 
diff --git a/hw/nvram/spapr_nvram.c b/hw/nvram/spapr_nvram.c
index fbfdf47e26..18b43be7f6 100644
--- a/hw/nvram/spapr_nvram.c
+++ b/hw/nvram/spapr_nvram.c
@@ -219,7 +219,7 @@ static void postload_update_cb(void *opaque, bool running, 
RunState state)
 {
 SpaprNvram *nvram = opaque;
 
-/* This is called after bdrv_invalidate_cache_all.  */
+/* This is called after bdrv_activate_all.  */
 
 qemu_del_vm_change_state_handler(nvram->vmstate);
 nvram->vmstate = NULL;
diff --git a/migration/block.c b/migration/block.c
index a950977855..077a413325 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -932,7 +932,7 @@ static int block_load(QEMUFile *f, void *opaque, int 
version_id)
 return -EINVAL;
 }
 
-blk_invalidate_cache(blk, _err);
+blk_activate(blk, _err);
 if (local_err) {
 error_report_err(local_err);
 return -EINVAL;
diff --git a/migration/migration.c b/migration/migration.c
index 9cc344514b..695f0f2900 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -503,9 +503,9 @@ static void process_incoming_migration_bh(void *opaque)
 if (!migrate_late_block_activate() ||
  (autostart && (!global_state_received() ||
 global_state_get_runstate() == RUN_STATE_RUNNING))) {
-/* Make sure all file formats 

[PULL 12/50] qsd: Add --daemonize

[Kevin Wolf]

From: Hanna Reitz 

To implement this, we reuse the existing daemonizing functions from the
system emulator, which mainly do the following:
- Fork off a child process, and set up a pipe between parent and child
- The parent process waits until the child sends a status byte over the
  pipe (0 means that the child was set up successfully; anything else
  (including errors or EOF) means that the child was not set up
  successfully), and then exits with an appropriate exit status
- The child process enters a new session (forking off again), changes
  the umask, and will ignore terminal signals from then on
- Once set-up is complete, the child will chdir to /, redirect all
  standard I/O streams to /dev/null, and tell the parent that set-up has
  been completed successfully

In contrast to qemu-nbd's --fork implementation, during the set up
phase, error messages are not piped through the parent process.
qemu-nbd mainly does this to detect errors, though (while os_daemonize()
has the child explicitly signal success after set up); because we do not
redirect stderr after forking, error messages continue to appear on
whatever the parent's stderr was (until set up is complete).

Signed-off-by: Hanna Reitz 
Message-Id: <20220303164814.284974-4-hre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 docs/tools/qemu-storage-daemon.rst   |  7 +++
 storage-daemon/qemu-storage-daemon.c | 15 +++
 2 files changed, 22 insertions(+)

diff --git a/docs/tools/qemu-storage-daemon.rst 
b/docs/tools/qemu-storage-daemon.rst
index 878e6a5c5c..8b97592663 100644
--- a/docs/tools/qemu-storage-daemon.rst
+++ b/docs/tools/qemu-storage-daemon.rst
@@ -154,6 +154,13 @@ Standard options:
   created but before accepting connections. The daemon has started successfully
   when the pid file is written and clients may begin connecting.
 
+.. option:: --daemonize
+
+  Daemonize the process. The parent process will exit once startup is complete
+  (i.e., after the pid file has been or would have been written) or failure
+  occurs. Its exit code reflects whether the child has started up successfully
+  or failed to do so.
+
 Examples
 
 Launch the daemon with QMP monitor socket ``qmp.sock`` so clients can execute
diff --git a/storage-daemon/qemu-storage-daemon.c 
b/storage-daemon/qemu-storage-daemon.c
index b798954edb..dd18b2cde8 100644
--- a/storage-daemon/qemu-storage-daemon.c
+++ b/storage-daemon/qemu-storage-daemon.c
@@ -93,6 +93,9 @@ static void help(void)
 "  --chardev configure a character device backend\n"
 " (see the qemu(1) man page for possible options)\n"
 "\n"
+"  --daemonizedaemonize the process, and have the parent exit\n"
+" once startup is complete\n"
+"\n"
 "  --export [type=]nbd,id=,node-name=[,name=]\n"
 "   [,writable=on|off][,bitmap=]\n"
 " export the specified block node over NBD\n"
@@ -144,6 +147,7 @@ QEMU_HELP_BOTTOM "\n",
 enum {
 OPTION_BLOCKDEV = 256,
 OPTION_CHARDEV,
+OPTION_DAEMONIZE,
 OPTION_EXPORT,
 OPTION_MONITOR,
 OPTION_NBD_SERVER,
@@ -200,6 +204,7 @@ static void process_options(int argc, char *argv[], bool 
pre_init_pass)
 static const struct option long_options[] = {
 {"blockdev", required_argument, NULL, OPTION_BLOCKDEV},
 {"chardev", required_argument, NULL, OPTION_CHARDEV},
+{"daemonize", no_argument, NULL, OPTION_DAEMONIZE},
 {"export", required_argument, NULL, OPTION_EXPORT},
 {"help", no_argument, NULL, 'h'},
 {"monitor", required_argument, NULL, OPTION_MONITOR},
@@ -225,6 +230,7 @@ static void process_options(int argc, char *argv[], bool 
pre_init_pass)
 c == '?' ||
 c == 'h' ||
 c == 'V' ||
+c == OPTION_DAEMONIZE ||
 c == OPTION_PIDFILE;
 
 /* Process every option only in its respective pass */
@@ -277,6 +283,12 @@ static void process_options(int argc, char *argv[], bool 
pre_init_pass)
 qemu_opts_del(opts);
 break;
 }
+case OPTION_DAEMONIZE:
+if (os_set_daemonize(true) < 0) {
+error_report("--daemonize not supported in this build");
+exit(EXIT_FAILURE);
+}
+break;
 case OPTION_EXPORT:
 {
 Visitor *v;
@@ -367,6 +379,8 @@ int main(int argc, char *argv[])
 
 process_options(argc, argv, true);
 
+os_daemonize();
+
 module_call_init(MODULE_INIT_QOM);
 module_call_init(MODULE_INIT_TRACE);
 qemu_add_opts(_trace_opts);
@@ -389,6 +403,7 @@ int main(int argc, char *argv[])
  * it.
  */
 pid_file_init();
+os_setup_post();
 
 while (!exit_requested) {
 main_loop_wait(false);
-- 
2.35.1

[PULL 01/50] crypto: perform permission checks under BQL

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Move the permission API calls into driver-specific callbacks
that always run under BQL. In this case, bdrv_crypto_luks
needs to perform permission checks before and after
qcrypto_block_amend_options(). The problem is that the caller,
block_crypto_amend_options_generic_luks(), can also run in I/O
from .bdrv_co_amend(). This does not comply with Global State-I/O API split,
as permissions API must always run under BQL.

Firstly, introduce .bdrv_amend_pre_run() and .bdrv_amend_clean()
callbacks. These two callbacks are guaranteed to be invoked under
BQL, respectively before and after .bdrv_co_amend().
They take care of performing the permission checks
in the same way as they are currently done before and after
qcrypto_block_amend_options().
These callbacks are in preparation for next patch, where we
delete the original permission check. Right now they just add redundant
control.

Then, call .bdrv_amend_pre_run() before job_start in
qmp_x_blockdev_amend(), so that it will be run before the job coroutine
is created and stay in the main loop.
As a cleanup, use JobDriver's .clean() callback to call
.bdrv_amend_clean(), and run amend-specific cleanup callbacks under BQL.

After this patch, permission failures occur early in the blockdev-amend
job to update a LUKS volume's keys.  iotest 296 must now expect them in
x-blockdev-amend's QMP reply instead of waiting for the actual job to
fail later.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220209105452.1694545-2-eespo...@redhat.com>
Signed-off-by: Hanna Reitz 
Message-Id: <20220304153729.711387-6-hre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h  | 14 ++
 block/amend.c  | 25 +
 block/crypto.c | 33 +
 tests/qemu-iotests/296 |  8 ++--
 tests/qemu-iotests/296.out | 17 +
 5 files changed, 83 insertions(+), 14 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 27008cfb22..31bd788919 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -124,6 +124,20 @@ struct BlockDriver {
  * on those children.
  */
 bool is_format;
+
+/*
+ * This function is invoked under BQL before .bdrv_co_amend()
+ * (which in contrast does not necessarily run under the BQL)
+ * to allow driver-specific initialization code that requires
+ * the BQL, like setting up specific permission flags.
+ */
+int (*bdrv_amend_pre_run)(BlockDriverState *bs, Error **errp);
+/*
+ * This function is invoked under BQL after .bdrv_co_amend()
+ * to allow cleaning up what was done in .bdrv_amend_pre_run().
+ */
+void (*bdrv_amend_clean)(BlockDriverState *bs);
+
 /*
  * Return true if @to_replace can be replaced by a BDS with the
  * same data as @bs without it affecting @bs's behavior (that is,
diff --git a/block/amend.c b/block/amend.c
index 392df9ef83..f465738665 100644
--- a/block/amend.c
+++ b/block/amend.c
@@ -53,10 +53,29 @@ static int coroutine_fn blockdev_amend_run(Job *job, Error 
**errp)
 return ret;
 }
 
+static int blockdev_amend_pre_run(BlockdevAmendJob *s, Error **errp)
+{
+if (s->bs->drv->bdrv_amend_pre_run) {
+return s->bs->drv->bdrv_amend_pre_run(s->bs, errp);
+}
+
+return 0;
+}
+
+static void blockdev_amend_clean(Job *job)
+{
+BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
+
+if (s->bs->drv->bdrv_amend_clean) {
+s->bs->drv->bdrv_amend_clean(s->bs);
+}
+}
+
 static const JobDriver blockdev_amend_job_driver = {
 .instance_size = sizeof(BlockdevAmendJob),
 .job_type  = JOB_TYPE_AMEND,
 .run   = blockdev_amend_run,
+.clean = blockdev_amend_clean,
 };
 
 void qmp_x_blockdev_amend(const char *job_id,
@@ -113,5 +132,11 @@ void qmp_x_blockdev_amend(const char *job_id,
 s->bs = bs,
 s->opts = QAPI_CLONE(BlockdevAmendOptions, options),
 s->force = has_force ? force : false;
+
+if (blockdev_amend_pre_run(s, errp)) {
+job_early_fail(>common);
+return;
+}
+
 job_start(>common);
 }
diff --git a/block/crypto.c b/block/crypto.c
index c8ba4681e2..70b2f07351 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -777,6 +777,37 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, 
Error **errp)
 return spec_info;
 }
 
+static int
+block_crypto_amend_prepare(BlockDriverState *bs, Error **errp)
+{
+BlockCrypto *crypto = bs->opaque;
+int ret;
+
+/* apply for exclusive read/write permissions to the underlying file */
+crypto->updating_keys = true;
+ret = bdrv_child_refresh_perms(bs, bs->file, errp);
+if (ret < 0) {
+/* Well, in this case we will not be updating any keys */
+crypto->updating_keys = false;
+}
+return ret;
+}
+
+static void
+block_crypto_amend_cleanup(BlockDriverState *bs)
+{
+BlockCrypto *crypto 

[PULL 05/50] block: move BQL logic of bdrv_co_invalidate_cache in bdrv_activate

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Split bdrv_co_invalidate cache in two: the Global State (under BQL)
code that takes care of permissions and running GS callbacks,
and leave only the I/O code (->bdrv_co_invalidate_cache) running in
the I/O coroutine.

The only side effect is that bdrv_co_invalidate_cache is not
recursive anymore, and so is every direct call to
bdrv_invalidate_cache().

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220209105452.1694545-6-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 37 +++--
 1 file changed, 23 insertions(+), 14 deletions(-)

diff --git a/block.c b/block.c
index 5e65f134f8..df353d55e8 100644
--- a/block.c
+++ b/block.c
@@ -6394,11 +6394,6 @@ void bdrv_init_with_whitelist(void)
 }
 
 int bdrv_activate(BlockDriverState *bs, Error **errp)
-{
-return bdrv_invalidate_cache(bs, errp);
-}
-
-int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 {
 BdrvChild *child, *parent;
 Error *local_err = NULL;
@@ -6410,7 +6405,7 @@ int coroutine_fn 
bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 }
 
 QLIST_FOREACH(child, >children, next) {
-bdrv_co_invalidate_cache(child->bs, _err);
+bdrv_activate(child->bs, _err);
 if (local_err) {
 error_propagate(errp, local_err);
 return -EINVAL;
@@ -6423,7 +6418,7 @@ int coroutine_fn 
bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
  * Note that the required permissions of inactive images are always a
  * subset of the permissions required after activating the image. This
  * allows us to just get the permissions upfront without restricting
- * drv->bdrv_invalidate_cache().
+ * bdrv_co_invalidate_cache().
  *
  * It also means that in error cases, we don't have to try and revert to
  * the old permissions (which is an operation that could fail, too). We can
@@ -6438,13 +6433,10 @@ int coroutine_fn 
bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 return ret;
 }
 
-if (bs->drv->bdrv_co_invalidate_cache) {
-bs->drv->bdrv_co_invalidate_cache(bs, _err);
-if (local_err) {
-bs->open_flags |= BDRV_O_INACTIVE;
-error_propagate(errp, local_err);
-return -EINVAL;
-}
+ret = bdrv_invalidate_cache(bs, errp);
+if (ret < 0) {
+bs->open_flags |= BDRV_O_INACTIVE;
+return ret;
 }
 
 FOR_EACH_DIRTY_BITMAP(bs, bm) {
@@ -6473,6 +6465,23 @@ int coroutine_fn 
bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 return 0;
 }
 
+int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
+{
+Error *local_err = NULL;
+
+assert(!(bs->open_flags & BDRV_O_INACTIVE));
+
+if (bs->drv->bdrv_co_invalidate_cache) {
+bs->drv->bdrv_co_invalidate_cache(bs, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return -EINVAL;
+}
+}
+
+return 0;
+}
+
 void bdrv_activate_all(Error **errp)
 {
 BlockDriverState *bs;
-- 
2.35.1

[PULL 09/50] cpus: use coroutine TLS macros for iothread_locked

[Kevin Wolf]

From: Stefan Hajnoczi 

qemu_mutex_iothread_locked() may be used from coroutines. Standard
__thread variables cannot be used by coroutines. Use the coroutine TLS
macros instead.

Signed-off-by: Stefan Hajnoczi 
Message-Id: <20220222140150.27240-5-stefa...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 softmmu/cpus.c | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 035395ae13..d1ff3cfea1 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -25,6 +25,7 @@
 #include "qemu/osdep.h"
 #include "qemu-common.h"
 #include "monitor/monitor.h"
+#include "qemu/coroutine-tls.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-machine.h"
 #include "qapi/qapi-commands-misc.h"
@@ -473,11 +474,11 @@ bool qemu_in_vcpu_thread(void)
 return current_cpu && qemu_cpu_is_self(current_cpu);
 }
 
-static __thread bool iothread_locked = false;
+QEMU_DEFINE_STATIC_CO_TLS(bool, iothread_locked)
 
 bool qemu_mutex_iothread_locked(void)
 {
-return iothread_locked;
+return get_iothread_locked();
 }
 
 /*
@@ -490,13 +491,13 @@ void qemu_mutex_lock_iothread_impl(const char *file, int 
line)
 
 g_assert(!qemu_mutex_iothread_locked());
 bql_lock(_global_mutex, file, line);
-iothread_locked = true;
+set_iothread_locked(true);
 }
 
 void qemu_mutex_unlock_iothread(void)
 {
 g_assert(qemu_mutex_iothread_locked());
-iothread_locked = false;
+set_iothread_locked(false);
 qemu_mutex_unlock(_global_mutex);
 }
 
-- 
2.35.1

Re: [PULL 0/6] hw/nvme updates

[Peter Maydell]

On Fri, 4 Mar 2022 at 06:24, Klaus Jensen  wrote:
>
> From: Klaus Jensen 
>
> Hi Peter,
>
> Last round of hw/nvme updates for v7.0.
>
> The following changes since commit 64ada298b98a51eb2512607f6e6180cb330c47b1:
>
>   Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220302' into 
> staging (2022-03-02 12:38:46 +)
>
> are available in the Git repository at:
>
>   git://git.infradead.org/qemu-nvme.git tags/nvme-next-pull-request
>
> for you to fetch changes up to 44219b6029fc52d5e967a963be91a9cf33f9f185:
>
>   hw/nvme: 64-bit pi support (2022-03-03 09:30:21 +0100)
>
> 
> hw/nvme updates
>
> - add enhanced protection information (64-bit guard)
>
> 



Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0
for any user-visible changes.

-- PMM

[PULL 02/50] crypto: distinguish between main loop and I/O in block_crypto_amend_options_generic_luks

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

block_crypto_amend_options_generic_luks uses the block layer
permission API, therefore it should be called with the BQL held.

However, the same function is being called by two BlockDriver
callbacks: bdrv_amend_options (under BQL) and bdrv_co_amend (I/O).

The latter is I/O because it is invoked by block/amend.c's
blockdev_amend_run(), a .run callback of the amend JobDriver.

Therefore we want to change this function to still perform
the permission check, but making sure it is done under BQL regardless
of the caller context.

Remove the permission check in block_crypto_amend_options_generic_luks()
and:
- in block_crypto_amend_options_luks() (BQL case, called by
  .bdrv_amend_options()), reuse helper functions
  block_crypto_amend_{prepare/cleanup} that take care of checking
  permissions.

- for block_crypto_co_amend_luks() (I/O case, called by
  .bdrv_co_amend()), don't check for permissions but delegate
  .bdrv_amend_pre_run() and .bdrv_amend_clean() to do it,
  performing these checks before and after the job runs in its aiocontext.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220209105452.1694545-3-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/crypto.c | 35 +++
 1 file changed, 15 insertions(+), 20 deletions(-)

diff --git a/block/crypto.c b/block/crypto.c
index 70b2f07351..9d5fecbef8 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -815,30 +815,17 @@ block_crypto_amend_options_generic_luks(BlockDriverState 
*bs,
 Error **errp)
 {
 BlockCrypto *crypto = bs->opaque;
-int ret;
 
 assert(crypto);
 assert(crypto->block);
 
-/* apply for exclusive read/write permissions to the underlying file*/
-crypto->updating_keys = true;
-ret = bdrv_child_refresh_perms(bs, bs->file, errp);
-if (ret) {
-goto cleanup;
-}
-
-ret = qcrypto_block_amend_options(crypto->block,
-  block_crypto_read_func,
-  block_crypto_write_func,
-  bs,
-  amend_options,
-  force,
-  errp);
-cleanup:
-/* release exclusive read/write permissions to the underlying file*/
-crypto->updating_keys = false;
-bdrv_child_refresh_perms(bs, bs->file, errp);
-return ret;
+return qcrypto_block_amend_options(crypto->block,
+   block_crypto_read_func,
+   block_crypto_write_func,
+   bs,
+   amend_options,
+   force,
+   errp);
 }
 
 static int
@@ -864,8 +851,16 @@ block_crypto_amend_options_luks(BlockDriverState *bs,
 if (!amend_options) {
 goto cleanup;
 }
+
+ret = block_crypto_amend_prepare(bs, errp);
+if (ret) {
+goto perm_cleanup;
+}
 ret = block_crypto_amend_options_generic_luks(bs, amend_options,
   force, errp);
+
+perm_cleanup:
+block_crypto_amend_cleanup(bs);
 cleanup:
 qapi_free_QCryptoBlockAmendOptions(amend_options);
 return ret;
-- 
2.35.1

[PULL 03/50] block: introduce bdrv_activate

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

This function is currently just a wrapper for bdrv_invalidate_cache(),
but in future will contain the code of bdrv_co_invalidate_cache() that
has to always be protected by BQL, and leave the rest in the I/O
coroutine.

Replace all bdrv_invalidate_cache() invokations with bdrv_activate().

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Hanna Reitz 
Message-Id: <20220209105452.1694545-4-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block.h| 1 +
 block.c  | 7 ++-
 block/block-backend.c| 2 +-
 block/export/export.c| 2 +-
 block/parallels.c| 2 +-
 tests/unit/test-block-iothread.c | 2 +-
 6 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index e1713ee306..18cd336911 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -494,6 +494,7 @@ int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf);
 int generated_co_wrapper bdrv_invalidate_cache(BlockDriverState *bs,
Error **errp);
 void bdrv_invalidate_cache_all(Error **errp);
+int bdrv_activate(BlockDriverState *bs, Error **errp);
 int bdrv_inactivate_all(void);
 
 /* Ensure contents are flushed to disk.  */
diff --git a/block.c b/block.c
index b54d59d1fa..7746727a47 100644
--- a/block.c
+++ b/block.c
@@ -6393,6 +6393,11 @@ void bdrv_init_with_whitelist(void)
 bdrv_init();
 }
 
+int bdrv_activate(BlockDriverState *bs, Error **errp)
+{
+return bdrv_invalidate_cache(bs, errp);
+}
+
 int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 {
 BdrvChild *child, *parent;
@@ -6478,7 +6483,7 @@ void bdrv_invalidate_cache_all(Error **errp)
 int ret;
 
 aio_context_acquire(aio_context);
-ret = bdrv_invalidate_cache(bs, errp);
+ret = bdrv_activate(bs, errp);
 aio_context_release(aio_context);
 if (ret < 0) {
 bdrv_next_cleanup();
diff --git a/block/block-backend.c b/block/block-backend.c
index 4ff6b4d785..c516113a36 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1889,7 +1889,7 @@ void blk_invalidate_cache(BlockBackend *blk, Error **errp)
 return;
 }
 
-bdrv_invalidate_cache(bs, errp);
+bdrv_activate(bs, errp);
 }
 
 bool blk_is_inserted(BlockBackend *blk)
diff --git a/block/export/export.c b/block/export/export.c
index 6d3b9964c8..7253af3bc3 100644
--- a/block/export/export.c
+++ b/block/export/export.c
@@ -139,7 +139,7 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error 
**errp)
  * access since the export could be available before migration handover.
  * ctx was acquired in the caller.
  */
-bdrv_invalidate_cache(bs, NULL);
+bdrv_activate(bs, NULL);
 
 perm = BLK_PERM_CONSISTENT_READ;
 if (export->writable) {
diff --git a/block/parallels.c b/block/parallels.c
index 6ebad2a2bb..e58c828422 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -873,7 +873,7 @@ static int parallels_open(BlockDriverState *bs, QDict 
*options, int flags,
 s->bat_dirty_bmap =
 bitmap_new(DIV_ROUND_UP(s->header_size, s->bat_dirty_block));
 
-/* Disable migration until bdrv_invalidate_cache method is added */
+/* Disable migration until bdrv_activate method is added */
 error_setg(>migration_blocker, "The Parallels format used by node '%s' "
"does not support live migration",
bdrv_get_device_or_node_name(bs));
diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c
index aea660aeed..378a7b7869 100644
--- a/tests/unit/test-block-iothread.c
+++ b/tests/unit/test-block-iothread.c
@@ -282,7 +282,7 @@ static void test_sync_op_check(BdrvChild *c)
 static void test_sync_op_invalidate_cache(BdrvChild *c)
 {
 /* Early success: Image is not inactive */
-bdrv_invalidate_cache(c->bs, NULL);
+bdrv_activate(c->bs, NULL);
 }
 
 
-- 
2.35.1

[PULL 34/50] assertions for blockdev.h global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-22-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/block-backend.c |  3 +++
 blockdev.c| 16 
 2 files changed, 19 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index 2ab1274dfe..bf77c4a8fa 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -810,6 +810,7 @@ bool bdrv_is_root_node(BlockDriverState *bs)
  */
 DriveInfo *blk_legacy_dinfo(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 return blk->legacy_dinfo;
 }
 
@@ -821,6 +822,7 @@ DriveInfo *blk_legacy_dinfo(BlockBackend *blk)
 DriveInfo *blk_set_legacy_dinfo(BlockBackend *blk, DriveInfo *dinfo)
 {
 assert(!blk->legacy_dinfo);
+GLOBAL_STATE_CODE();
 return blk->legacy_dinfo = dinfo;
 }
 
@@ -831,6 +833,7 @@ DriveInfo *blk_set_legacy_dinfo(BlockBackend *blk, 
DriveInfo *dinfo)
 BlockBackend *blk_by_legacy_dinfo(DriveInfo *dinfo)
 {
 BlockBackend *blk = NULL;
+GLOBAL_STATE_CODE();
 
 while ((blk = blk_next(blk)) != NULL) {
 if (blk->legacy_dinfo == dinfo) {
diff --git a/blockdev.c b/blockdev.c
index 12a317f149..e46e831212 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -113,6 +113,8 @@ void override_max_devs(BlockInterfaceType type, int 
max_devs)
 BlockBackend *blk;
 DriveInfo *dinfo;
 
+GLOBAL_STATE_CODE();
+
 if (max_devs <= 0) {
 return;
 }
@@ -142,6 +144,8 @@ void blockdev_mark_auto_del(BlockBackend *blk)
 DriveInfo *dinfo = blk_legacy_dinfo(blk);
 BlockJob *job;
 
+GLOBAL_STATE_CODE();
+
 if (!dinfo) {
 return;
 }
@@ -163,6 +167,7 @@ void blockdev_mark_auto_del(BlockBackend *blk)
 void blockdev_auto_del(BlockBackend *blk)
 {
 DriveInfo *dinfo = blk_legacy_dinfo(blk);
+GLOBAL_STATE_CODE();
 
 if (dinfo && dinfo->auto_del) {
 monitor_remove_blk(blk);
@@ -187,6 +192,8 @@ QemuOpts *drive_add(BlockInterfaceType type, int index, 
const char *file,
 {
 QemuOpts *opts;
 
+GLOBAL_STATE_CODE();
+
 opts = qemu_opts_parse_noisily(qemu_find_opts("drive"), optstr, false);
 if (!opts) {
 return NULL;
@@ -207,6 +214,8 @@ DriveInfo *drive_get(BlockInterfaceType type, int bus, int 
unit)
 BlockBackend *blk;
 DriveInfo *dinfo;
 
+GLOBAL_STATE_CODE();
+
 for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
 dinfo = blk_legacy_dinfo(blk);
 if (dinfo && dinfo->type == type
@@ -229,6 +238,8 @@ void drive_check_orphaned(void)
 Location loc;
 bool orphans = false;
 
+GLOBAL_STATE_CODE();
+
 for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
 dinfo = blk_legacy_dinfo(blk);
 /*
@@ -262,6 +273,7 @@ void drive_check_orphaned(void)
 
 DriveInfo *drive_get_by_index(BlockInterfaceType type, int index)
 {
+GLOBAL_STATE_CODE();
 return drive_get(type,
  drive_index_to_bus_id(type, index),
  drive_index_to_unit_id(type, index));
@@ -273,6 +285,8 @@ int drive_get_max_bus(BlockInterfaceType type)
 BlockBackend *blk;
 DriveInfo *dinfo;
 
+GLOBAL_STATE_CODE();
+
 max_bus = -1;
 for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
 dinfo = blk_legacy_dinfo(blk);
@@ -759,6 +773,8 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType 
block_default_type,
 const char *filename;
 int i;
 
+GLOBAL_STATE_CODE();
+
 /* Change legacy command line options into QMP ones */
 static const struct {
 const char *from;
-- 
2.35.1

[PATCH v2 5/5] iotests: fortify compare_images() against crashes

[John Snow]

Fortify compare_images() to be more discerning about the status codes it
receives. If qemu_img() returns an exit code that implies it didn't
actually perform the comparison, treat that as an exceptional
circumstance and force the caller to be aware of the peril.

If a negative test is desired (perhaps to test how qemu_img compare
behaves on malformed images, for instance), it is still possible to
catch the exception in the test and deal with that circumstance
manually.

Signed-off-by: John Snow 
Reviewed-by: Eric Blake 
---
 tests/qemu-iotests/iotests.py | 21 -
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index f97eeb5f91..cc10ad47c6 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -504,11 +504,22 @@ def qemu_nbd_popen(*args):
 p.kill()
 p.wait()
 
-def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt):
-'''Return True if two image files are identical'''
-res = qemu_img('compare', '-f', fmt1,
-   '-F', fmt2, img1, img2, check=False)
-return res.returncode == 0
+def compare_images(img1: str, img2: str,
+   fmt1: str = imgfmt, fmt2: str = imgfmt) -> bool:
+"""
+Compare two images with QEMU_IMG; return True if they are identical.
+
+:raise CalledProcessError:
+when qemu-img crashes or returns a status code of anything other
+than 0 (identical) or 1 (different).
+"""
+try:
+qemu_img('compare', '-f', fmt1, '-F', fmt2, img1, img2)
+return True
+except subprocess.CalledProcessError as exc:
+if exc.returncode == 1:
+return False
+raise
 
 def create_image(name, size):
 '''Create a fully-allocated raw image with sector markers'''
-- 
2.34.1

Re: [PATCH 4/4] oslib: drop qemu_gettimeofday()

[Laurent Vivier]

Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :

From: Marc-André Lureau 

No longer used after the previous patches.

Signed-off-by: Marc-André Lureau 
---
  include/sysemu/os-posix.h |  3 ---
  include/sysemu/os-win32.h |  6 --
  util/oslib-win32.c| 20 
  3 files changed, 29 deletions(-)



Reviewed-by: Laurent Vivier 

Re: [PATCH 07/12] tests/qemu-iotests: expand _filter_nbd rules

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:25PM +, Daniel P. Berrangé wrote:
> Some tests will want to use 'localhost' instead of '127.0.0.1', and
> some will use the image options syntax rather than the classic URI
> syntax.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  tests/qemu-iotests/common.filter | 4 
>  1 file changed, 4 insertions(+)

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[PULL 27/50] block: introduce assert_bdrv_graph_writable

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

We want to be sure that the functions that write the child and
parent list of a bs are under BQL and drain.

BQL prevents from concurrent writings from the GS API, while
drains protect from I/O.

TODO: drains are missing in some functions using this assert.
Therefore a proper assertion will fail. Because adding drains
requires additional discussions, they will be added in future
series.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-15-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int-global-state.h | 17 +
 block.c|  4 
 2 files changed, 21 insertions(+)

diff --git a/include/block/block_int-global-state.h 
b/include/block/block_int-global-state.h
index 5078d6a6ea..0f21b0570b 100644
--- a/include/block/block_int-global-state.h
+++ b/include/block/block_int-global-state.h
@@ -309,4 +309,21 @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs,
  */
 void bdrv_drain_all_end_quiesce(BlockDriverState *bs);
 
+/**
+ * Make sure that the function is running under both drain and BQL.
+ * The latter protects from concurrent writings
+ * from the GS API, while the former prevents concurrent reads
+ * from I/O.
+ */
+static inline void assert_bdrv_graph_writable(BlockDriverState *bs)
+{
+/*
+ * TODO: this function is incomplete. Because the users of this
+ * assert lack the necessary drains, check only for BQL.
+ * Once the necessary drains are added,
+ * assert also for qatomic_read(>quiesce_counter) > 0
+ */
+assert(qemu_in_main_thread());
+}
+
 #endif /* BLOCK_INT_GLOBAL_STATE */
diff --git a/block.c b/block.c
index f2a7350548..fe05df471a 100644
--- a/block.c
+++ b/block.c
@@ -1420,6 +1420,7 @@ static void bdrv_child_cb_attach(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
 
+assert_bdrv_graph_writable(bs);
 QLIST_INSERT_HEAD(>children, child, next);
 
 if (child->role & BDRV_CHILD_COW) {
@@ -1439,6 +1440,7 @@ static void bdrv_child_cb_detach(BdrvChild *child)
 
 bdrv_unapply_subtree_drain(child, bs);
 
+assert_bdrv_graph_writable(bs);
 QLIST_REMOVE(child, next);
 }
 
@@ -2829,6 +2831,7 @@ static void bdrv_replace_child_noperm(BdrvChild **childp,
 if (child->klass->detach) {
 child->klass->detach(child);
 }
+assert_bdrv_graph_writable(old_bs);
 QLIST_REMOVE(child, next_parent);
 }
 
@@ -2838,6 +2841,7 @@ static void bdrv_replace_child_noperm(BdrvChild **childp,
 }
 
 if (new_bs) {
+assert_bdrv_graph_writable(new_bs);
 QLIST_INSERT_HEAD(_bs->parents, child, next_parent);
 
 /*
-- 
2.35.1

[PULL 44/50] job.h: assertions in the callers of JobDriver function pointers

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-32-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 job.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/job.c b/job.c
index 54db80df66..075c6f3a20 100644
--- a/job.c
+++ b/job.c
@@ -381,6 +381,8 @@ void job_ref(Job *job)
 
 void job_unref(Job *job)
 {
+GLOBAL_STATE_CODE();
+
 if (--job->refcnt == 0) {
 assert(job->status == JOB_STATUS_NULL);
 assert(!timer_pending(>sleep_timer));
@@ -602,6 +604,7 @@ bool job_user_paused(Job *job)
 void job_user_resume(Job *job, Error **errp)
 {
 assert(job);
+GLOBAL_STATE_CODE();
 if (!job->user_paused || job->pause_count <= 0) {
 error_setg(errp, "Can't resume a job that was not paused");
 return;
@@ -672,6 +675,7 @@ static void job_update_rc(Job *job)
 static void job_commit(Job *job)
 {
 assert(!job->ret);
+GLOBAL_STATE_CODE();
 if (job->driver->commit) {
 job->driver->commit(job);
 }
@@ -680,6 +684,7 @@ static void job_commit(Job *job)
 static void job_abort(Job *job)
 {
 assert(job->ret);
+GLOBAL_STATE_CODE();
 if (job->driver->abort) {
 job->driver->abort(job);
 }
@@ -687,6 +692,7 @@ static void job_abort(Job *job)
 
 static void job_clean(Job *job)
 {
+GLOBAL_STATE_CODE();
 if (job->driver->clean) {
 job->driver->clean(job);
 }
@@ -726,6 +732,7 @@ static int job_finalize_single(Job *job)
 
 static void job_cancel_async(Job *job, bool force)
 {
+GLOBAL_STATE_CODE();
 if (job->driver->cancel) {
 force = job->driver->cancel(job, force);
 } else {
@@ -825,6 +832,7 @@ static void job_completed_txn_abort(Job *job)
 
 static int job_prepare(Job *job)
 {
+GLOBAL_STATE_CODE();
 if (job->ret == 0 && job->driver->prepare) {
 job->ret = job->driver->prepare(job);
 job_update_rc(job);
@@ -952,6 +960,7 @@ static void coroutine_fn job_co_entry(void *opaque)
 Job *job = opaque;
 
 assert(job && job->driver && job->driver->run);
+assert(job->aio_context == qemu_get_current_aio_context());
 job_pause_point(job);
 job->ret = job->driver->run(job, >err);
 job->deferred_to_main_loop = true;
@@ -1054,6 +1063,7 @@ void job_complete(Job *job, Error **errp)
 {
 /* Should not be reachable via external interface for internal jobs */
 assert(job->id);
+GLOBAL_STATE_CODE();
 if (job_apply_verb(job, JOB_VERB_COMPLETE, errp)) {
 return;
 }
-- 
2.35.1

[PULL 43/50] job.h: split function pointers in JobDriver

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

The job API will be handled separately in another serie.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-31-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/qemu/job.h | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/include/qemu/job.h b/include/qemu/job.h
index 6e67b6977f..c105b31076 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -169,6 +169,12 @@ typedef struct Job {
  * Callbacks and other information about a Job driver.
  */
 struct JobDriver {
+
+/*
+ * These fields are initialized when this object is created,
+ * and are never changed afterwards
+ */
+
 /** Derived Job struct size */
 size_t instance_size;
 
@@ -184,9 +190,18 @@ struct JobDriver {
  * aborted. If it returns zero, the job moves into the WAITING state. If it
  * is the last job to complete in its transaction, all jobs in the
  * transaction move from WAITING to PENDING.
+ *
+ * This callback must be run in the job's context.
  */
 int coroutine_fn (*run)(Job *job, Error **errp);
 
+/*
+ * Functions run without regard to the BQL that may run in any
+ * arbitrary thread. These functions do not need to be thread-safe
+ * because the caller ensures that they are invoked from one
+ * thread at time.
+ */
+
 /**
  * If the callback is not NULL, it will be invoked when the job transitions
  * into the paused state.  Paused jobs must not perform any asynchronous
@@ -201,6 +216,13 @@ struct JobDriver {
  */
 void coroutine_fn (*resume)(Job *job);
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 /**
  * Called when the job is resumed by the user (i.e. user_paused becomes
  * false). .user_resume is called before .resume.
-- 
2.35.1

[PULL 40/50] block_int-common.h: split function pointers in BdrvChildClass

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-28-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int-common.h | 81 ++--
 1 file changed, 47 insertions(+), 34 deletions(-)

diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index f05ebb0da3..5a04c778e4 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -830,19 +830,16 @@ struct BdrvChildClass {
  */
 bool parent_is_bds;
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
 void (*inherit_options)(BdrvChildRole role, bool parent_is_format,
 int *child_flags, QDict *child_options,
 int parent_flags, QDict *parent_options);
-
 void (*change_media)(BdrvChild *child, bool load);
-void (*resize)(BdrvChild *child);
-
-/*
- * Returns a name that is supposedly more useful for human users than the
- * node name for identifying the node in question (in particular, a BB
- * name), or NULL if the parent can't provide a better name.
- */
-const char *(*get_name)(BdrvChild *child);
 
 /*
  * Returns a malloced string that describes the parent of the child for a
@@ -852,6 +849,47 @@ struct BdrvChildClass {
  */
 char *(*get_parent_desc)(BdrvChild *child);
 
+/*
+ * Notifies the parent that the child has been activated/inactivated (e.g.
+ * when migration is completing) and it can start/stop requesting
+ * permissions and doing I/O on it.
+ */
+void (*activate)(BdrvChild *child, Error **errp);
+int (*inactivate)(BdrvChild *child);
+
+void (*attach)(BdrvChild *child);
+void (*detach)(BdrvChild *child);
+
+/*
+ * Notifies the parent that the filename of its child has changed (e.g.
+ * because the direct child was removed from the backing chain), so that it
+ * can update its reference.
+ */
+int (*update_filename)(BdrvChild *child, BlockDriverState *new_base,
+   const char *filename, Error **errp);
+
+bool (*can_set_aio_ctx)(BdrvChild *child, AioContext *ctx,
+GSList **ignore, Error **errp);
+void (*set_aio_ctx)(BdrvChild *child, AioContext *ctx, GSList **ignore);
+
+AioContext *(*get_parent_aio_context)(BdrvChild *child);
+
+/*
+ * I/O API functions. These functions are thread-safe.
+ *
+ * See include/block/block-io.h for more information about
+ * the I/O API.
+ */
+
+void (*resize)(BdrvChild *child);
+
+/*
+ * Returns a name that is supposedly more useful for human users than the
+ * node name for identifying the node in question (in particular, a BB
+ * name), or NULL if the parent can't provide a better name.
+ */
+const char *(*get_name)(BdrvChild *child);
+
 /*
  * If this pair of functions is implemented, the parent doesn't issue new
  * requests after returning from .drained_begin() until .drained_end() is
@@ -876,31 +914,6 @@ struct BdrvChildClass {
  * activity on the child has stopped.
  */
 bool (*drained_poll)(BdrvChild *child);
-
-/*
- * Notifies the parent that the child has been activated/inactivated (e.g.
- * when migration is completing) and it can start/stop requesting
- * permissions and doing I/O on it.
- */
-void (*activate)(BdrvChild *child, Error **errp);
-int (*inactivate)(BdrvChild *child);
-
-void (*attach)(BdrvChild *child);
-void (*detach)(BdrvChild *child);
-
-/*
- * Notifies the parent that the filename of its child has changed (e.g.
- * because the direct child was removed from the backing chain), so that it
- * can update its reference.
- */
-int (*update_filename)(BdrvChild *child, BlockDriverState *new_base,
-   const char *filename, Error **errp);
-
-bool (*can_set_aio_ctx)(BdrvChild *child, AioContext *ctx,
-GSList **ignore, Error **errp);
-void (*set_aio_ctx)(BdrvChild *child, AioContext *ctx, GSList **ignore);
-
-AioContext *(*get_parent_aio_context)(BdrvChild *child);
 };
 
 extern const BdrvChildClass child_of_bds;
-- 
2.35.1

[PULL 33/50] include/sysemu/blockdev.h: global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

blockdev functions run always under the BQL lock.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-21-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/sysemu/blockdev.h | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index f9fb54d437..3211b16513 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -13,9 +13,6 @@
 #include "block/block.h"
 #include "qemu/queue.h"
 
-void blockdev_mark_auto_del(BlockBackend *blk);
-void blockdev_auto_del(BlockBackend *blk);
-
 typedef enum {
 IF_DEFAULT = -1,/* for use with drive_add() only */
 /*
@@ -38,6 +35,16 @@ struct DriveInfo {
 QTAILQ_ENTRY(DriveInfo) next;
 };
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
+void blockdev_mark_auto_del(BlockBackend *blk);
+void blockdev_auto_del(BlockBackend *blk);
+
 DriveInfo *blk_legacy_dinfo(BlockBackend *blk);
 DriveInfo *blk_set_legacy_dinfo(BlockBackend *blk, DriveInfo *dinfo);
 BlockBackend *blk_by_legacy_dinfo(DriveInfo *dinfo);
-- 
2.35.1

[PULL 42/50] block-backend-common.h: split function pointers in BlockDevOps

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Assertions in the callers of the function pointrs are already
added by previous patches.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Stefan Hajnoczi 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20220303151616.325444-30-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/sysemu/block-backend-common.h | 28 ++-
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/include/sysemu/block-backend-common.h 
b/include/sysemu/block-backend-common.h
index 6963bbf45a..2391679c56 100644
--- a/include/sysemu/block-backend-common.h
+++ b/include/sysemu/block-backend-common.h
@@ -27,6 +27,14 @@
 
 /* Callbacks for block device models */
 typedef struct BlockDevOps {
+
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 /*
  * Runs when virtual media changed (monitor commands eject, change)
  * Argument load is true on load and false on eject.
@@ -44,16 +52,26 @@ typedef struct BlockDevOps {
  * true, even if they do not support eject requests.
  */
 void (*eject_request_cb)(void *opaque, bool force);
-/*
- * Is the virtual tray open?
- * Device models implement this only when the device has a tray.
- */
-bool (*is_tray_open)(void *opaque);
+
 /*
  * Is the virtual medium locked into the device?
  * Device models implement this only when device has such a lock.
  */
 bool (*is_medium_locked)(void *opaque);
+
+/*
+ * I/O API functions. These functions are thread-safe.
+ *
+ * See include/block/block-io.h for more information about
+ * the I/O API.
+ */
+
+/*
+ * Is the virtual tray open?
+ * Device models implement this only when the device has a tray.
+ */
+bool (*is_tray_open)(void *opaque);
+
 /*
  * Runs when the size changed (e.g. monitor command block_resize)
  */
-- 
2.35.1

[PATCH v2 4/5] iotests: make qemu_img raise on non-zero rc by default

[John Snow]

re-write qemu_img() as a function that will by default raise a
VerboseProcessException (extended from CalledProcessException) on
non-zero return codes. This will produce a stack trace that will show
the command line arguments and return code from the failed process run.

Users that want something more flexible (there appears to be only one)
can use check=False and manage the return themselves. However, when the
return code is negative, the Exception will be raised no matter what.
This is done under the belief that there's no legitimate reason, even in
negative tests, to see a crash from qemu-img.

Signed-off-by: John Snow 
Reviewed-by: Eric Blake 
---
 tests/qemu-iotests/257|  8 +++--
 tests/qemu-iotests/iotests.py | 56 ++-
 2 files changed, 54 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index fb5359c581..e7e7a2317e 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -241,11 +241,13 @@ def compare_images(image, reference, baseimg=None, 
expected_match=True):
 expected_ret = 0 if expected_match else 1
 if baseimg:
 qemu_img("rebase", "-u", "-b", baseimg, '-F', iotests.imgfmt, image)
-ret = qemu_img("compare", image, reference)
+
+sub = qemu_img("compare", image, reference, check=False)
+
 log('qemu_img compare "{:s}" "{:s}" ==> {:s}, {:s}'.format(
 image, reference,
-"Identical" if ret == 0 else "Mismatch",
-"OK!" if ret == expected_ret else "ERROR!"),
+"Identical" if sub.returncode == 0 else "Mismatch",
+"OK!" if sub.returncode == expected_ret else "ERROR!"),
 filters=[iotests.filter_testfiles])
 
 def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 6027780180..f97eeb5f91 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -37,9 +37,10 @@
 
 from contextlib import contextmanager
 
+from qemu.aqmp.legacy import QEMUMonitorProtocol
 from qemu.machine import qtest
 from qemu.qmp import QMPMessage
-from qemu.aqmp.legacy import QEMUMonitorProtocol
+from qemu.utils import VerboseProcessError
 
 # Use this logger for logging messages directly from the iotests module
 logger = logging.getLogger('qemu.iotests')
@@ -216,9 +217,49 @@ def qemu_img_pipe_and_status(*args: str) -> Tuple[str, 
int]:
 return qemu_tool_pipe_and_status('qemu-img', full_args,
  drop_successful_output=is_create)
 
-def qemu_img(*args: str) -> int:
-'''Run qemu-img and return the exit code'''
-return qemu_img_pipe_and_status(*args)[1]
+def qemu_img(*args: str, check: bool = True, combine_stdio: bool = True
+ ) -> subprocess.CompletedProcess[str]:
+"""
+Run qemu_img and return the status code and console output.
+
+This function always prepends QEMU_IMG_OPTIONS and may further alter
+the args for 'create' commands.
+
+:param args: command-line arguments to qemu-img.
+:param check: Enforce a return code of zero.
+:param combine_stdio: set to False to keep stdout/stderr separated.
+
+:raise VerboseProcessError:
+When the return code is negative, or on any non-zero exit code
+when 'check=True' was provided (the default). This exception has
+'stdout', 'stderr', and 'returncode' properties that may be
+inspected to show greater detail. If this exception is not
+handled, the command-line, return code, and all console output
+will be included at the bottom of the stack trace.
+
+:return: a CompletedProcess. This object has args, returncode, and
+stdout properties. If streams are not combined, it will also
+have a stderr property.
+"""
+full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
+
+subp = subprocess.run(
+full_args,
+stdout=subprocess.PIPE,
+stderr=subprocess.STDOUT if combine_stdio else subprocess.PIPE,
+universal_newlines=True,
+check=False
+)
+
+if check and subp.returncode or (subp.returncode < 0):
+raise VerboseProcessError(
+subp.returncode, full_args,
+output=subp.stdout,
+stderr=subp.stderr,
+)
+
+return subp
+
 
 def ordered_qmp(qmsg, conv_keys=True):
 # Dictionaries are not ordered prior to 3.6, therefore:
@@ -233,7 +274,7 @@ def ordered_qmp(qmsg, conv_keys=True):
 return od
 return qmsg
 
-def qemu_img_create(*args):
+def qemu_img_create(*args: str) -> subprocess.CompletedProcess[str]:
 return qemu_img('create', *args)
 
 def qemu_img_measure(*args):
@@ -465,8 +506,9 @@ def qemu_nbd_popen(*args):
 
 def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt):
 '''Return True if two image files are identical'''
-return qemu_img('compare', '-f', fmt1,
-'-F', fmt2, img1, img2) == 0
+res = 

[PATCH 2/5] block/amend: Always call .bdrv_amend_clean()

[Hanna Reitz]

.bdrv_amend_clean() says block drivers can use it to clean up what was
done in .bdrv_amend_pre_run().  Therefore, it should always be called
after .bdrv_amend_pre_run(), which means we need it to call it in the
JobDriver.free() callback, not in JobDriver.clean().

Signed-off-by: Hanna Reitz 
---
 block/amend.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/amend.c b/block/amend.c
index f465738665..553890ded9 100644
--- a/block/amend.c
+++ b/block/amend.c
@@ -62,7 +62,7 @@ static int blockdev_amend_pre_run(BlockdevAmendJob *s, Error 
**errp)
 return 0;
 }
 
-static void blockdev_amend_clean(Job *job)
+static void blockdev_amend_free(Job *job)
 {
 BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
 
@@ -75,7 +75,7 @@ static const JobDriver blockdev_amend_job_driver = {
 .instance_size = sizeof(BlockdevAmendJob),
 .job_type  = JOB_TYPE_AMEND,
 .run   = blockdev_amend_run,
-.clean = blockdev_amend_clean,
+.free  = blockdev_amend_free,
 };
 
 void qmp_x_blockdev_amend(const char *job_id,
-- 
2.34.1

[PATCH 3/5] block/crypto: Reset updating_keys on perm failure

[Hanna Reitz]

When the permissions could not be changed as would be necessary for
updating the keys, reset updating_keys to false so
block_crypto_child_perms() will not continue to try claiming these
permissions.

Signed-off-by: Hanna Reitz 
---
 block/crypto.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/block/crypto.c b/block/crypto.c
index 5225a68a54..9d5fecbef8 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -781,10 +781,16 @@ static int
 block_crypto_amend_prepare(BlockDriverState *bs, Error **errp)
 {
 BlockCrypto *crypto = bs->opaque;
+int ret;
 
 /* apply for exclusive read/write permissions to the underlying file */
 crypto->updating_keys = true;
-return bdrv_child_refresh_perms(bs, bs->file, errp);
+ret = bdrv_child_refresh_perms(bs, bs->file, errp);
+if (ret < 0) {
+/* Well, in this case we will not be updating any keys */
+crypto->updating_keys = false;
+}
+return ret;
 }
 
 static void
-- 
2.34.1

[PATCH 4/5] block/amend: Keep strong reference to BDS

[Hanna Reitz]

Otherwise, the BDS might be freed while the job is running, which would
cause a use-after-free.

Signed-off-by: Hanna Reitz 
---
 block/amend.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/amend.c b/block/amend.c
index 553890ded9..f696a006e3 100644
--- a/block/amend.c
+++ b/block/amend.c
@@ -69,6 +69,8 @@ static void blockdev_amend_free(Job *job)
 if (s->bs->drv->bdrv_amend_clean) {
 s->bs->drv->bdrv_amend_clean(s->bs);
 }
+
+bdrv_unref(s->bs);
 }
 
 static const JobDriver blockdev_amend_job_driver = {
@@ -129,6 +131,7 @@ void qmp_x_blockdev_amend(const char *job_id,
 return;
 }
 
+bdrv_ref(bs);
 s->bs = bs,
 s->opts = QAPI_CLONE(BlockdevAmendOptions, options),
 s->force = has_force ? force : false;
-- 
2.34.1

Re: [PATCH 1/4] m68k/nios2-semi: fix gettimeofday() result check

[Laurent Vivier]

Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :

From: Marc-André Lureau 

gettimeofday() returns 0 for success.

Signed-off-by: Marc-André Lureau 
---
  target/m68k/m68k-semi.c   | 2 +-
  target/nios2/nios2-semi.c | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/m68k/m68k-semi.c b/target/m68k/m68k-semi.c
index 44ec7e4612c6..c5c164e096c8 100644
--- a/target/m68k/m68k-semi.c
+++ b/target/m68k/m68k-semi.c
@@ -381,7 +381,7 @@ void do_m68k_semihosting(CPUM68KState *env, int nr)
  qemu_timeval tv;
  struct gdb_timeval *p;
  result = qemu_gettimeofday();
-if (result != 0) {
+if (result == 0) {
  if (!(p = lock_user(VERIFY_WRITE,
  arg0, sizeof(struct gdb_timeval), 0))) {
  /* FIXME - check error code? */
diff --git a/target/nios2/nios2-semi.c b/target/nios2/nios2-semi.c
index fe5598bae4d7..5a7ad0c7108d 100644
--- a/target/nios2/nios2-semi.c
+++ b/target/nios2/nios2-semi.c
@@ -403,7 +403,7 @@ void do_nios2_semihosting(CPUNios2State *env)
  qemu_timeval tv;
  struct gdb_timeval *p;
  result = qemu_gettimeofday();
-if (result != 0) {
+if (result == 0) {
  p = lock_user(VERIFY_WRITE, arg0, sizeof(struct gdb_timeval),
0);
  if (!p) {


It seems to have never worked correctly...

Reviewed-by: Laurent Vivier 

[PULL 08/50] rcu: use coroutine TLS macros

[Kevin Wolf]

From: Stefan Hajnoczi 

RCU may be used from coroutines. Standard __thread variables cannot be
used by coroutines. Use the coroutine TLS macros instead.

Signed-off-by: Stefan Hajnoczi 
Message-Id: <20220222140150.27240-4-stefa...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/qemu/rcu.h |  7 ---
 tests/unit/rcutorture.c| 10 +-
 tests/unit/test-rcu-list.c |  4 ++--
 util/rcu.c | 10 +-
 4 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h
index e69efbd47f..b063c6fde8 100644
--- a/include/qemu/rcu.h
+++ b/include/qemu/rcu.h
@@ -29,6 +29,7 @@
 #include "qemu/atomic.h"
 #include "qemu/notify.h"
 #include "qemu/sys_membarrier.h"
+#include "qemu/coroutine-tls.h"
 
 #ifdef __cplusplus
 extern "C" {
@@ -76,11 +77,11 @@ struct rcu_reader_data {
 NotifierList force_rcu;
 };
 
-extern __thread struct rcu_reader_data rcu_reader;
+QEMU_DECLARE_CO_TLS(struct rcu_reader_data, rcu_reader)
 
 static inline void rcu_read_lock(void)
 {
-struct rcu_reader_data *p_rcu_reader = _reader;
+struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader();
 unsigned ctr;
 
 if (p_rcu_reader->depth++ > 0) {
@@ -96,7 +97,7 @@ static inline void rcu_read_lock(void)
 
 static inline void rcu_read_unlock(void)
 {
-struct rcu_reader_data *p_rcu_reader = _reader;
+struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader();
 
 assert(p_rcu_reader->depth != 0);
 if (--p_rcu_reader->depth > 0) {
diff --git a/tests/unit/rcutorture.c b/tests/unit/rcutorture.c
index de6f649058..495a4e6f42 100644
--- a/tests/unit/rcutorture.c
+++ b/tests/unit/rcutorture.c
@@ -122,7 +122,7 @@ static void *rcu_read_perf_test(void *arg)
 
 rcu_register_thread();
 
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 qatomic_inc();
 while (goflag == GOFLAG_INIT) {
 g_usleep(1000);
@@ -148,7 +148,7 @@ static void *rcu_update_perf_test(void *arg)
 
 rcu_register_thread();
 
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 qatomic_inc();
 while (goflag == GOFLAG_INIT) {
 g_usleep(1000);
@@ -253,7 +253,7 @@ static void *rcu_read_stress_test(void *arg)
 
 rcu_register_thread();
 
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 while (goflag == GOFLAG_INIT) {
 g_usleep(1000);
 }
@@ -304,7 +304,7 @@ static void *rcu_update_stress_test(void *arg)
 struct rcu_stress *cp = qatomic_read(_stress_current);
 
 rcu_register_thread();
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 
 while (goflag == GOFLAG_INIT) {
 g_usleep(1000);
@@ -347,7 +347,7 @@ static void *rcu_fake_update_stress_test(void *arg)
 {
 rcu_register_thread();
 
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 while (goflag == GOFLAG_INIT) {
 g_usleep(1000);
 }
diff --git a/tests/unit/test-rcu-list.c b/tests/unit/test-rcu-list.c
index 49641e1936..64b81ae058 100644
--- a/tests/unit/test-rcu-list.c
+++ b/tests/unit/test-rcu-list.c
@@ -171,7 +171,7 @@ static void *rcu_q_reader(void *arg)
 
 rcu_register_thread();
 
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 qatomic_inc();
 while (qatomic_read() == GOFLAG_INIT) {
 g_usleep(1000);
@@ -206,7 +206,7 @@ static void *rcu_q_updater(void *arg)
 long long n_removed_local = 0;
 struct list_element *el, *prev_el;
 
-*(struct rcu_reader_data **)arg = _reader;
+*(struct rcu_reader_data **)arg = get_ptr_rcu_reader();
 qatomic_inc();
 while (qatomic_read() == GOFLAG_INIT) {
 g_usleep(1000);
diff --git a/util/rcu.c b/util/rcu.c
index c91da9f137..b6d6c71cff 100644
--- a/util/rcu.c
+++ b/util/rcu.c
@@ -65,7 +65,7 @@ static inline int rcu_gp_ongoing(unsigned long *ctr)
 /* Written to only by each individual reader. Read by both the reader and the
  * writers.
  */
-__thread struct rcu_reader_data rcu_reader;
+QEMU_DEFINE_CO_TLS(struct rcu_reader_data, rcu_reader)
 
 /* Protected by rcu_registry_lock.  */
 typedef QLIST_HEAD(, rcu_reader_data) ThreadList;
@@ -355,23 +355,23 @@ void drain_call_rcu(void)
 
 void rcu_register_thread(void)
 {
-assert(rcu_reader.ctr == 0);
+assert(get_ptr_rcu_reader()->ctr == 0);
 qemu_mutex_lock(_registry_lock);
-QLIST_INSERT_HEAD(, _reader, node);
+QLIST_INSERT_HEAD(, get_ptr_rcu_reader(), node);
 qemu_mutex_unlock(_registry_lock);
 }
 
 void rcu_unregister_thread(void)
 {
 qemu_mutex_lock(_registry_lock);
-QLIST_REMOVE(_reader, node);
+QLIST_REMOVE(get_ptr_rcu_reader(), node);
 qemu_mutex_unlock(_registry_lock);
 }
 
 void rcu_add_force_rcu_notifier(Notifier *n)
 {
   

[PULL 21/50] block/block-backend.c: assertions for block-backend

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

All the global state (GS) API functions will check that
qemu_in_main_thread() returns true. If not, it means
that the safety of BQL cannot be guaranteed, and
they need to be moved to I/O.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-9-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/block-backend.c  | 78 ++
 softmmu/qdev-monitor.c |  2 ++
 2 files changed, 80 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index 462e18facf..4476b61b8b 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -239,6 +239,7 @@ static void blk_root_activate(BdrvChild *child, Error 
**errp)
 
 void blk_set_force_allow_inactivate(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 blk->force_allow_inactivate = true;
 }
 
@@ -357,6 +358,8 @@ BlockBackend *blk_new(AioContext *ctx, uint64_t perm, 
uint64_t shared_perm)
 {
 BlockBackend *blk;
 
+GLOBAL_STATE_CODE();
+
 blk = g_new0(BlockBackend, 1);
 blk->refcnt = 1;
 blk->ctx = ctx;
@@ -394,6 +397,8 @@ BlockBackend *blk_new_with_bs(BlockDriverState *bs, 
uint64_t perm,
 {
 BlockBackend *blk = blk_new(bdrv_get_aio_context(bs), perm, shared_perm);
 
+GLOBAL_STATE_CODE();
+
 if (blk_insert_bs(blk, bs, errp) < 0) {
 blk_unref(blk);
 return NULL;
@@ -422,6 +427,8 @@ BlockBackend *blk_new_open(const char *filename, const char 
*reference,
 uint64_t perm = 0;
 uint64_t shared = BLK_PERM_ALL;
 
+GLOBAL_STATE_CODE();
+
 /*
  * blk_new_open() is mainly used in .bdrv_create implementations and the
  * tools where sharing isn't a major concern because the BDS stays private
@@ -499,6 +506,7 @@ static void drive_info_del(DriveInfo *dinfo)
 
 int blk_get_refcnt(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 return blk ? blk->refcnt : 0;
 }
 
@@ -509,6 +517,7 @@ int blk_get_refcnt(BlockBackend *blk)
 void blk_ref(BlockBackend *blk)
 {
 assert(blk->refcnt > 0);
+GLOBAL_STATE_CODE();
 blk->refcnt++;
 }
 
@@ -519,6 +528,7 @@ void blk_ref(BlockBackend *blk)
  */
 void blk_unref(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 if (blk) {
 assert(blk->refcnt > 0);
 if (blk->refcnt > 1) {
@@ -539,6 +549,7 @@ void blk_unref(BlockBackend *blk)
  */
 BlockBackend *blk_all_next(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 return blk ? QTAILQ_NEXT(blk, link)
: QTAILQ_FIRST(_backends);
 }
@@ -547,6 +558,8 @@ void blk_remove_all_bs(void)
 {
 BlockBackend *blk = NULL;
 
+GLOBAL_STATE_CODE();
+
 while ((blk = blk_all_next(blk)) != NULL) {
 AioContext *ctx = blk_get_aio_context(blk);
 
@@ -570,6 +583,7 @@ void blk_remove_all_bs(void)
  */
 BlockBackend *blk_next(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 return blk ? QTAILQ_NEXT(blk, monitor_link)
: QTAILQ_FIRST(_block_backends);
 }
@@ -636,6 +650,7 @@ static void bdrv_next_reset(BdrvNextIterator *it)
 
 BlockDriverState *bdrv_first(BdrvNextIterator *it)
 {
+GLOBAL_STATE_CODE();
 bdrv_next_reset(it);
 return bdrv_next(it);
 }
@@ -673,6 +688,7 @@ bool monitor_add_blk(BlockBackend *blk, const char *name, 
Error **errp)
 {
 assert(!blk->name);
 assert(name && name[0]);
+GLOBAL_STATE_CODE();
 
 if (!id_wellformed(name)) {
 error_setg(errp, "Invalid device name");
@@ -700,6 +716,8 @@ bool monitor_add_blk(BlockBackend *blk, const char *name, 
Error **errp)
  */
 void monitor_remove_blk(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
+
 if (!blk->name) {
 return;
 }
@@ -726,6 +744,7 @@ BlockBackend *blk_by_name(const char *name)
 {
 BlockBackend *blk = NULL;
 
+GLOBAL_STATE_CODE();
 assert(name);
 while ((blk = blk_next(blk)) != NULL) {
 if (!strcmp(name, blk->name)) {
@@ -760,6 +779,7 @@ static BlockBackend *bdrv_first_blk(BlockDriverState *bs)
  */
 bool bdrv_has_blk(BlockDriverState *bs)
 {
+GLOBAL_STATE_CODE();
 return bdrv_first_blk(bs) != NULL;
 }
 
@@ -770,6 +790,7 @@ bool bdrv_is_root_node(BlockDriverState *bs)
 {
 BdrvChild *c;
 
+GLOBAL_STATE_CODE();
 QLIST_FOREACH(c, >parents, next_parent) {
 if (c->klass != _root) {
 return false;
@@ -819,6 +840,7 @@ BlockBackend *blk_by_legacy_dinfo(DriveInfo *dinfo)
  */
 BlockBackendPublic *blk_get_public(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 return >public;
 }
 
@@ -827,6 +849,7 @@ BlockBackendPublic *blk_get_public(BlockBackend *blk)
  */
 BlockBackend *blk_by_public(BlockBackendPublic *public)
 {
+GLOBAL_STATE_CODE();
 return container_of(public, BlockBackend, public);
 }
 
@@ -838,6 +861,8 @@ void blk_remove_bs(BlockBackend *blk)
 ThrottleGroupMember *tgm = >public.throttle_group_member;
 BdrvChild *root;
 
+GLOBAL_STATE_CODE();
+
 notifier_list_notify(>remove_bs_notifiers, blk);
 if (tgm->throttle_state) {
 BlockDriverState 

[PULL 46/50] iotests: Allow using QMP with the QSD

[Kevin Wolf]

From: Hanna Reitz 

Add a parameter to optionally open a QMP connection when creating a
QemuStorageDaemon instance.

Signed-off-by: Hanna Reitz 
Message-Id: <20220216105355.30729-3-hre...@redhat.com>
Reviewed-by: Eric Blake 
Reviewed-by: Stefan Hajnoczi 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/iotests.py | 32 +++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 6ba65eb1ff..6027780180 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -39,6 +39,7 @@
 
 from qemu.machine import qtest
 from qemu.qmp import QMPMessage
+from qemu.aqmp.legacy import QEMUMonitorProtocol
 
 # Use this logger for logging messages directly from the iotests module
 logger = logging.getLogger('qemu.iotests')
@@ -348,14 +349,30 @@ def cmd(self, cmd):
 
 
 class QemuStorageDaemon:
-def __init__(self, *args: str, instance_id: str = 'a'):
+_qmp: Optional[QEMUMonitorProtocol] = None
+_qmpsock: Optional[str] = None
+# Python < 3.8 would complain if this type were not a string literal
+# (importing `annotations` from `__future__` would work; but not on <= 3.6)
+_p: 'Optional[subprocess.Popen[bytes]]' = None
+
+def __init__(self, *args: str, instance_id: str = 'a', qmp: bool = False):
 assert '--pidfile' not in args
 self.pidfile = os.path.join(test_dir, f'qsd-{instance_id}-pid')
 all_args = [qsd_prog] + list(args) + ['--pidfile', self.pidfile]
 
+if qmp:
+self._qmpsock = os.path.join(sock_dir, f'qsd-{instance_id}.sock')
+all_args += ['--chardev',
+ f'socket,id=qmp-sock,path={self._qmpsock}',
+ '--monitor', 'qmp-sock']
+
+self._qmp = QEMUMonitorProtocol(self._qmpsock, server=True)
+
 # Cannot use with here, we want the subprocess to stay around
 # pylint: disable=consider-using-with
 self._p = subprocess.Popen(all_args)
+if self._qmp is not None:
+self._qmp.accept()
 while not os.path.exists(self.pidfile):
 if self._p.poll() is not None:
 cmd = ' '.join(all_args)
@@ -370,11 +387,24 @@ def __init__(self, *args: str, instance_id: str = 'a'):
 
 assert self._pid == self._p.pid
 
+def qmp(self, cmd: str, args: Optional[Dict[str, object]] = None) \
+-> QMPMessage:
+assert self._qmp is not None
+return self._qmp.cmd(cmd, args)
+
 def stop(self, kill_signal=15):
 self._p.send_signal(kill_signal)
 self._p.wait()
 self._p = None
 
+if self._qmp:
+self._qmp.close()
+
+if self._qmpsock is not None:
+try:
+os.remove(self._qmpsock)
+except OSError:
+pass
 try:
 os.remove(self.pidfile)
 except OSError:
-- 
2.35.1

[PULL 47/50] iotests/graph-changes-while-io: New test

[Kevin Wolf]

From: Hanna Reitz 

Test the following scenario:
1. Some block node (null-co) attached to a user (here: NBD server) that
   performs I/O and keeps the node in an I/O thread
2. Repeatedly run blockdev-add/blockdev-del to add/remove an overlay
   to/from that node

Each blockdev-add triggers bdrv_refresh_limits(), and because
blockdev-add runs in the main thread, it does not stop the I/O requests.
I/O can thus happen while the limits are refreshed, and when such a
request sees a temporarily invalid block limit (e.g. alignment is 0),
this may easily crash qemu (or the storage daemon in this case).

The block layer needs to ensure that I/O requests to a node are paused
while that node's BlockLimits are refreshed.

Signed-off-by: Hanna Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20220216105355.30729-4-hre...@redhat.com>
Reviewed-by: Stefan Hajnoczi 
Signed-off-by: Kevin Wolf 
---
 .../qemu-iotests/tests/graph-changes-while-io | 91 +++
 .../tests/graph-changes-while-io.out  |  5 +
 2 files changed, 96 insertions(+)
 create mode 100755 tests/qemu-iotests/tests/graph-changes-while-io
 create mode 100644 tests/qemu-iotests/tests/graph-changes-while-io.out

diff --git a/tests/qemu-iotests/tests/graph-changes-while-io 
b/tests/qemu-iotests/tests/graph-changes-while-io
new file mode 100755
index 00..567e8cf21e
--- /dev/null
+++ b/tests/qemu-iotests/tests/graph-changes-while-io
@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+# group: rw
+#
+# Test graph changes while I/O is happening
+#
+# Copyright (C) 2022 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+from threading import Thread
+import iotests
+from iotests import imgfmt, qemu_img, qemu_img_create, QMPTestCase, \
+QemuStorageDaemon
+
+
+top = os.path.join(iotests.test_dir, 'top.img')
+nbd_sock = os.path.join(iotests.sock_dir, 'nbd.sock')
+
+
+def do_qemu_img_bench() -> None:
+"""
+Do some I/O requests on `nbd_sock`.
+"""
+assert qemu_img('bench', '-f', 'raw', '-c', '200',
+f'nbd+unix:///node0?socket={nbd_sock}') == 0
+
+
+class TestGraphChangesWhileIO(QMPTestCase):
+def setUp(self) -> None:
+# Create an overlay that can be added at runtime on top of the
+# null-co block node that will receive I/O
+assert qemu_img_create('-f', imgfmt, '-F', 'raw', '-b', 'null-co://',
+   top) == 0
+
+# QSD instance with a null-co block node in an I/O thread,
+# exported over NBD (on `nbd_sock`, export name "node0")
+self.qsd = QemuStorageDaemon(
+'--object', 'iothread,id=iothread0',
+'--blockdev', 'null-co,node-name=node0,read-zeroes=true',
+'--nbd-server', f'addr.type=unix,addr.path={nbd_sock}',
+'--export', 'nbd,id=exp0,node-name=node0,iothread=iothread0,' +
+'fixed-iothread=true,writable=true',
+qmp=True
+)
+
+def tearDown(self) -> None:
+self.qsd.stop()
+
+def test_blockdev_add_while_io(self) -> None:
+# Run qemu-img bench in the background
+bench_thr = Thread(target=do_qemu_img_bench)
+bench_thr.start()
+
+# While qemu-img bench is running, repeatedly add and remove an
+# overlay to/from node0
+while bench_thr.is_alive():
+result = self.qsd.qmp('blockdev-add', {
+'driver': imgfmt,
+'node-name': 'overlay',
+'backing': 'node0',
+'file': {
+'driver': 'file',
+'filename': top
+}
+})
+self.assert_qmp(result, 'return', {})
+
+result = self.qsd.qmp('blockdev-del', {
+'node-name': 'overlay'
+})
+self.assert_qmp(result, 'return', {})
+
+bench_thr.join()
+
+if __name__ == '__main__':
+# Format must support raw backing files
+iotests.main(supported_fmts=['qcow', 'qcow2', 'qed'],
+ supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/graph-changes-while-io.out 
b/tests/qemu-iotests/tests/graph-changes-while-io.out
new file mode 100644
index 00..ae1213e6f8
--- /dev/null
+++ b/tests/qemu-iotests/tests/graph-changes-while-io.out
@@ -0,0 +1,5 @@
+.

[PULL 45/50] block: Make bdrv_refresh_limits() non-recursive

[Kevin Wolf]

From: Hanna Reitz 

bdrv_refresh_limits() recurses down to the node's children.  That does
not seem necessary: When we refresh limits on some node, and then
recurse down and were to change one of its children's BlockLimits, then
that would mean we noticed the changed limits by pure chance.  The fact
that we refresh the parent's limits has nothing to do with it, so the
reason for the change probably happened before this point in time, and
we should have refreshed the limits then.

Consequently, we should actually propagate block limits changes upwards,
not downwards.  That is a separate and pre-existing issue, though, and
so will not be addressed in this patch.

The problem with recursing is that bdrv_refresh_limits() is not atomic.
It begins with zeroing BDS.bl, and only then sets proper, valid limits.
If we do not drain all nodes whose limits are refreshed, then concurrent
I/O requests can encounter invalid request_alignment values and crash
qemu.  Therefore, a recursing bdrv_refresh_limits() requires the whole
subtree to be drained, which is currently not ensured by most callers.

A non-recursive bdrv_refresh_limits() only requires the node in question
to not receive I/O requests, and this is done by most callers in some
way or another:
- bdrv_open_driver() deals with a new node with no parents yet
- bdrv_set_file_or_backing_noperm() acts on a drained node
- bdrv_reopen_commit() acts only on drained nodes
- bdrv_append() should in theory require the node to be drained; in
  practice most callers just lock the AioContext, which should at least
  be enough to prevent concurrent I/O requests from accessing invalid
  limits

So we can resolve the bug by making bdrv_refresh_limits() non-recursive.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1879437
Signed-off-by: Hanna Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20220216105355.30729-2-hre...@redhat.com>
Reviewed-by: Stefan Hajnoczi 
Signed-off-by: Kevin Wolf 
---
 block/io.c | 4 
 1 file changed, 4 deletions(-)

diff --git a/block/io.c b/block/io.c
index 4b1d97c7da..efc011ce65 100644
--- a/block/io.c
+++ b/block/io.c
@@ -193,10 +193,6 @@ void bdrv_refresh_limits(BlockDriverState *bs, Transaction 
*tran, Error **errp)
 QLIST_FOREACH(c, >children, next) {
 if (c->role & (BDRV_CHILD_DATA | BDRV_CHILD_FILTERED | BDRV_CHILD_COW))
 {
-bdrv_refresh_limits(c->bs, tran, errp);
-if (*errp) {
-return;
-}
 bdrv_merge_limits(>bl, >bs->bl);
 have_limits = true;
 }
-- 
2.35.1

[PULL 32/50] assertions for blockjob.h global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-20-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 blockjob.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/blockjob.c b/blockjob.c
index d79a52d204..4868453d74 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -62,6 +62,7 @@ static bool is_block_job(Job *job)
 BlockJob *block_job_next(BlockJob *bjob)
 {
 Job *job = bjob ? >job : NULL;
+GLOBAL_STATE_CODE();
 
 do {
 job = job_next(job);
@@ -73,6 +74,7 @@ BlockJob *block_job_next(BlockJob *bjob)
 BlockJob *block_job_get(const char *id)
 {
 Job *job = job_get(id);
+GLOBAL_STATE_CODE();
 
 if (job && is_block_job(job)) {
 return container_of(job, BlockJob, job);
@@ -184,6 +186,7 @@ static const BdrvChildClass child_job = {
 
 void block_job_remove_all_bdrv(BlockJob *job)
 {
+GLOBAL_STATE_CODE();
 /*
  * bdrv_root_unref_child() may reach child_job_[can_]set_aio_ctx(),
  * which will also traverse job->nodes, so consume the list one by
@@ -206,6 +209,7 @@ void block_job_remove_all_bdrv(BlockJob *job)
 bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs)
 {
 GSList *el;
+GLOBAL_STATE_CODE();
 
 for (el = job->nodes; el; el = el->next) {
 BdrvChild *c = el->data;
@@ -222,6 +226,7 @@ int block_job_add_bdrv(BlockJob *job, const char *name, 
BlockDriverState *bs,
 {
 BdrvChild *c;
 bool need_context_ops;
+GLOBAL_STATE_CODE();
 
 bdrv_ref(bs);
 
@@ -271,6 +276,8 @@ bool block_job_set_speed(BlockJob *job, int64_t speed, 
Error **errp)
 const BlockJobDriver *drv = block_job_driver(job);
 int64_t old_speed = job->speed;
 
+GLOBAL_STATE_CODE();
+
 if (job_apply_verb(>job, JOB_VERB_SET_SPEED, errp) < 0) {
 return false;
 }
@@ -309,6 +316,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp)
 BlockJobInfo *info;
 uint64_t progress_current, progress_total;
 
+GLOBAL_STATE_CODE();
+
 if (block_job_is_internal(job)) {
 error_setg(errp, "Cannot query QEMU internal jobs");
 return NULL;
@@ -491,6 +500,7 @@ fail:
 
 void block_job_iostatus_reset(BlockJob *job)
 {
+GLOBAL_STATE_CODE();
 if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) {
 return;
 }
@@ -548,5 +558,6 @@ BlockErrorAction block_job_error_action(BlockJob *job, 
BlockdevOnError on_err,
 
 AioContext *block_job_get_aio_context(BlockJob *job)
 {
+GLOBAL_STATE_CODE();
 return job->job.aio_context;
 }
-- 
2.35.1

[PULL 50/50] block/amend: Keep strong reference to BDS

[Kevin Wolf]

From: Hanna Reitz 

Otherwise, the BDS might be freed while the job is running, which would
cause a use-after-free.

Signed-off-by: Hanna Reitz 
Message-Id: <20220304153729.711387-5-hre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/amend.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/amend.c b/block/amend.c
index 553890ded9..f696a006e3 100644
--- a/block/amend.c
+++ b/block/amend.c
@@ -69,6 +69,8 @@ static void blockdev_amend_free(Job *job)
 if (s->bs->drv->bdrv_amend_clean) {
 s->bs->drv->bdrv_amend_clean(s->bs);
 }
+
+bdrv_unref(s->bs);
 }
 
 static const JobDriver blockdev_amend_job_driver = {
@@ -129,6 +131,7 @@ void qmp_x_blockdev_amend(const char *job_id,
 return;
 }
 
+bdrv_ref(bs);
 s->bs = bs,
 s->opts = QAPI_CLONE(BlockdevAmendOptions, options),
 s->force = has_force ? force : false;
-- 
2.35.1

Re: [PATCH 02/12] block: pass desired TLS hostname through from block driver client

[Daniel P . Berrangé]

On Thu, Mar 03, 2022 at 02:14:34PM -0600, Eric Blake wrote:
> On Thu, Mar 03, 2022 at 04:03:20PM +, Daniel P. Berrangé wrote:
> > In
> > 
> >   commit a71d597b989fd701b923f09b3c20ac4fcaa55e81
> >   Author: Vladimir Sementsov-Ogievskiy 
> >   Date:   Thu Jun 10 13:08:00 2021 +0300
> > 
> > block/nbd: reuse nbd_co_do_establish_connection() in nbd_open()
> > 
> > the use of the 'hostname' field from the BDRVNBDState struct was
> > lost, and 'nbd_connect' just hardcoded it to match the IP socket
> > address. This was a harmless bug at the time since we block use
> > with anything other than IP sockets.
> > 
> > Shortly though, We want to allow the caller to override the hostname
> 
> s/We/we/
> 
> > used in the TLS certificate checks. This is to allow for TLS
> > when doing port forwarding or tunneling. Thus we need to reinstate
> > the passing along of the 'hostname'.
> > 
> > Signed-off-by: Daniel P. Berrangé 
> > ---
> >  block/nbd.c |  7 ---
> >  include/block/nbd.h |  3 ++-
> >  nbd/client-connection.c | 12 +---
> >  3 files changed, 15 insertions(+), 7 deletions(-)
> 
> Nice - this a great step towards fixing a longstanding annoyance of
> mine that libnbd and nbdkit support TLS over Unix sockets, but qemu
> didn't.


> > diff --git a/include/block/nbd.h b/include/block/nbd.h
> > index 78d101b774..a98eb665da 100644
> > --- a/include/block/nbd.h
> > +++ b/include/block/nbd.h
> > @@ -415,7 +415,8 @@ NBDClientConnection *nbd_client_connection_new(const 
> > SocketAddress *saddr,
> > bool do_negotiation,
> > const char *export_name,
> > const char *x_dirty_bitmap,
> > -   QCryptoTLSCreds *tlscreds);
> > +   QCryptoTLSCreds *tlscreds,
> > +   const char *tlshostname);
> 
> We already have a lot of parameters; does it make sense to bundle
> tlshostname into the QCryptoTLSCreds struct at all?  But that would
> change the QAPI (or maybe you do it later in the series), it is not a
> show-stopper to this patch.

The credentials object is something that can be used for multiple
connections. The TLS hostname override meanwhile is specific to
a single connection. Thus it would not be appropriate to store the
TLS hostname in the credentials struct.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[PATCH v2 03/12] block/nbd: support override of hostname for TLS certificate validation

[Daniel P . Berrangé]

When connecting to an NBD server with TLS and x509 credentials,
the client must validate the hostname it uses for the connection,
against that published in the server's certificate. If the client
is tunnelling its connection over some other channel, however, the
hostname it uses may not match the info reported in the server's
certificate. In such a case, the user needs to explicitly set an
override for the hostname to use for certificate validation.

This is achieved by adding a 'tls-hostname' property to the NBD
block driver.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 block/nbd.c  | 18 +++---
 qapi/block-core.json |  3 +++
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/block/nbd.c b/block/nbd.c
index dd43929207..113aa5d3af 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -90,9 +90,10 @@ typedef struct BDRVNBDState {
 uint32_t reconnect_delay;
 uint32_t open_timeout;
 SocketAddress *saddr;
-char *export, *tlscredsid;
+char *export;
+char *tlscredsid;
 QCryptoTLSCreds *tlscreds;
-const char *tlshostname;
+char *tlshostname;
 char *x_dirty_bitmap;
 bool alloc_depth;
 
@@ -121,6 +122,8 @@ static void nbd_clear_bdrvstate(BlockDriverState *bs)
 s->export = NULL;
 g_free(s->tlscredsid);
 s->tlscredsid = NULL;
+g_free(s->tlshostname);
+s->tlshostname = NULL;
 g_free(s->x_dirty_bitmap);
 s->x_dirty_bitmap = NULL;
 }
@@ -1764,6 +1767,11 @@ static QemuOptsList nbd_runtime_opts = {
 .type = QEMU_OPT_STRING,
 .help = "ID of the TLS credentials to use",
 },
+{
+.name = "tls-hostname",
+.type = QEMU_OPT_STRING,
+.help = "Override hostname for validating TLS x509 certificate",
+},
 {
 .name = "x-dirty-bitmap",
 .type = QEMU_OPT_STRING,
@@ -1835,7 +1843,10 @@ static int nbd_process_options(BlockDriverState *bs, 
QDict *options,
 error_setg(errp, "TLS only supported over IP sockets");
 goto error;
 }
-s->tlshostname = s->saddr->u.inet.host;
+s->tlshostname = g_strdup(qemu_opt_get(opts, "tls-hostname"));
+if (!s->tlshostname) {
+s->tlshostname = g_strdup(s->saddr->u.inet.host);
+}
 }
 
 s->x_dirty_bitmap = g_strdup(qemu_opt_get(opts, "x-dirty-bitmap"));
@@ -2037,6 +2048,7 @@ static const char *const nbd_strong_runtime_opts[] = {
 "port",
 "export",
 "tls-creds",
+"tls-hostname",
 "server.",
 
 NULL
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 9a5a3641d0..1c730c6f2a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4078,6 +4078,8 @@
 #
 # @tls-creds: TLS credentials ID
 #
+# @tls-hostname: TLS hostname override for certificate validation (Since 7.0)
+#
 # @x-dirty-bitmap: A metadata context name such as "qemu:dirty-bitmap:NAME"
 #  or "qemu:allocation-depth" to query in place of the
 #  traditional "base:allocation" block status (see
@@ -4108,6 +4110,7 @@
   'data': { 'server': 'SocketAddress',
 '*export': 'str',
 '*tls-creds': 'str',
+'*tls-hostname': 'str',
 '*x-dirty-bitmap': { 'type': 'str', 'features': [ 'unstable' ] },
 '*reconnect-delay': 'uint32',
 '*open-timeout': 'uint32' } }
-- 
2.34.1

[PATCH v2 09/12] tests/qemu-iotests: convert NBD TLS test to use standard filters

[Daniel P . Berrangé]

Using standard filters is more future proof than rolling our own.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 tests/qemu-iotests/233 | 29 -
 tests/qemu-iotests/233.out |  8 
 2 files changed, 16 insertions(+), 21 deletions(-)

diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index 9ca7b68f42..050267298d 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -65,7 +65,7 @@ tls_x509_create_client "ca1" "client3"
 echo
 echo "== preparing image =="
 _make_test_img 64M
-$QEMU_IO -c 'w -P 0x11 1m 1m' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c 'w -P 0x11 1m 1m' "$TEST_IMG" 2>&1 | _filter_qemu_io
 
 echo
 echo "== check TLS client to plain server fails =="
@@ -74,9 +74,9 @@ nbd_server_start_tcp_socket -f $IMGFMT "$TEST_IMG" 2> 
"$TEST_DIR/server.log"
 obj=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
 $QEMU_IMG info --image-opts --object $obj \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
-2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+2>&1 | _filter_nbd
 $QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
---tls-creds=tls0
+--tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
 
 nbd_server_stop
 
@@ -88,8 +88,10 @@ nbd_server_start_tcp_socket \
 --tls-creds tls0 \
 -f $IMGFMT "$TEST_IMG" 2>> "$TEST_DIR/server.log"
 
-$QEMU_IMG info nbd://localhost:$nbd_tcp_port 2>&1 | sed 
"s/$nbd_tcp_port/PORT/g"
-$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port
+$QEMU_IMG info nbd://localhost:$nbd_tcp_port \
+2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port \
+2>&1 | _filter_qemu_nbd_exports
 
 echo
 echo "== check TLS works =="
@@ -97,21 +99,21 @@ 
obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
 obj2=tls-creds-x509,dir=${tls_dir}/client3,endpoint=client,id=tls0
 $QEMU_IMG info --image-opts --object $obj1 \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
-2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+2>&1 | _filter_nbd
 $QEMU_IMG info --image-opts --object $obj2 \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
-2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+2>&1 | _filter_nbd
 $QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj1 \
---tls-creds=tls0
+--tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
 
 echo
 echo "== check TLS with different CA fails =="
 obj=tls-creds-x509,dir=${tls_dir}/client2,endpoint=client,id=tls0
 $QEMU_IMG info --image-opts --object $obj \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
-2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+2>&1 | _filter_nbd
 $QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj \
---tls-creds=tls0
+--tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
 
 echo
 echo "== perform I/O over TLS =="
@@ -121,7 +123,8 @@ $QEMU_IO -c 'r -P 0x11 1m 1m' -c 'w -P 0x22 1m 1m' 
--image-opts \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
 2>&1 | _filter_qemu_io
 
-$QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" \
+2>&1 | _filter_qemu_io
 
 echo
 echo "== check TLS with authorization =="
@@ -139,12 +142,12 @@ nbd_server_start_tcp_socket \
 $QEMU_IMG info --image-opts \
 --object tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0 \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
-2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+2>&1 | _filter_nbd
 
 $QEMU_IMG info --image-opts \
 --object tls-creds-x509,dir=${tls_dir}/client3,endpoint=client,id=tls0 \
 driver=nbd,host=$nbd_tcp_addr,port=$nbd_tcp_port,tls-creds=tls0 \
-2>&1 | sed "s/$nbd_tcp_port/PORT/g"
+2>&1 | _filter_nbd
 
 echo
 echo "== final server log =="
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index 4b1f6a0e15..67a027d879 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -17,15 +17,12 @@ wrote 1048576/1048576 bytes at offset 1048576
 qemu-img: Could not open 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': 
Denied by server for option 5 (starttls)
 server reported: TLS not configured
 qemu-nbd: Denied by server for option 5 (starttls)
-server reported: TLS not configured
 
 == check plain client to TLS server fails ==
 qemu-img: Could not open 'nbd://localhost:PORT': TLS negotiation required 
before option 7 (go)
 Did you forget a valid tls-creds?
 server reported: Option 0x7 not permitted before TLS
 qemu-nbd: TLS negotiation required before option 3 (list)
-Did you forget a valid tls-creds?
-server reported: Option 0x3 not permitted before TLS
 
 == check TLS works ==
 image: nbd://127.0.0.1:PORT
@@ -39,12 +36,7 @@ disk size: unavailable
 exports available: 1
  export: ''
   size:  67108864
-  flags: 0xced ( flush fua trim zeroes df cache fast-zero )
   min block: 1
-  opt block: 4096
-  max block: 

Re: [PATCH v2 6/9] util: Share qemu_try_memalign() implementation between POSIX and Windows

[Philippe Mathieu-Daudé]

On 4/3/22 12:21, Peter Maydell wrote:

The qemu_try_memalign() functions for POSIX and Windows used to be
significantly different, but these days they are identical except for
the actual allocation function called, and the POSIX version already
has to have ifdeffery for different allocation functions.

Move to a single implementation in memalign.c, which uses the Windows
_aligned_malloc if we detect that function in meson.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-id: 20220226180723.1706285-7-peter.mayd...@linaro.org
---
  meson.build|  1 +
  util/memalign.c| 39 +++
  util/oslib-posix.c | 29 -
  util/oslib-win32.c | 17 -
  4 files changed, 40 insertions(+), 46 deletions(-)


Nice.

Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v2 1/3] block: Make bdrv_refresh_limits() non-recursive

[Hanna Reitz]

On 04.03.22 15:14, Kevin Wolf wrote:

Am 04.03.2022 um 13:44 hat Hanna Reitz geschrieben:

On 03.03.22 17:56, Kevin Wolf wrote:

Am 16.02.2022 um 11:53 hat Hanna Reitz geschrieben:

bdrv_refresh_limits() recurses down to the node's children.  That does
not seem necessary: When we refresh limits on some node, and then
recurse down and were to change one of its children's BlockLimits, then
that would mean we noticed the changed limits by pure chance.  The fact
that we refresh the parent's limits has nothing to do with it, so the
reason for the change probably happened before this point in time, and
we should have refreshed the limits then.

On the other hand, we do not have infrastructure for noticing that block
limits change after they have been initialized for the first time (this
would require propagating the change upwards to the respective node's
parents), and so evidently we consider this case impossible.

I like your optimistic approach, but my interpretation would have been
that this is simply a bug. ;-)

blockdev-reopen allows changing options that affect the block limits
(most importantly probably request_alignment), so this should be
propagated to the parents. I think we'll actually not see failures if we
forget to do this, but parents can either advertise excessive alignment
requirements or they may run into RMW when accessing the child, so this
would only affect performance. This is probably why nobody reported it
yet.

Ah, right, I forgot this for parents of parents...  I thought the
block limits of a node might change if its children list changes, and
so we should bdrv_refresh_limits() when that children list changes,
but forgot that we really do need to propagate this up, right.

I mean the case that you mention is true as well. A few places do call
bdrv_refresh_limits() after changing the graph, but I don't know if it
covers all cases.


If this case is impossible, then we will not need to recurse down in
bdrv_refresh_limits().  Every node's limits are initialized in
bdrv_open_driver(), and are refreshed whenever its children change.
We want to use the childrens' limits to get some initial default, but
we can just take them, we do not need to refresh them.

I think even if we need to propagate to the parents, we still don't need
to propagate to the children because the children have already been
refreshed by whatever changed their options (like bdrv_reopen_commit()).
And parent limits don't influence the child limits at all.

So this patch looks good to me, just not the reasoning.

OK, so, uh, can we just drop these two paragraphs?  (“On the other hand...”
and “If this case is impossible…”)

Or we could replace them with a note hinting at the potential bug that would
need to be fixed, e.g.

“
Consequently, we should actually propagate block limits changes upwards,
not downwards.  That is a separate and pre-existing issue, though, and
so will not be addressed in this patch.
”

Ok, I'm replacing this in my tree.


Question is, if we at some point do propagate this upwards, won’t this cause
exactly the same problem that this patch is trying to get around, i.e. that
we might call bdrv_refresh_limits() on non-drained parent nodes?

Drain also propagates upwards, so at least those callers that drain the
node itself won't have the problem. And the other cases from the commit
messages look like they shouldn't have any parents.


Finally some good news today :)

[PATCH 1/5] block/amend: Clean up job on early failure

[Hanna Reitz]

Signed-off-by: Hanna Reitz 
---
 block/amend.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/amend.c b/block/amend.c
index 329bca53dc..f465738665 100644
--- a/block/amend.c
+++ b/block/amend.c
@@ -134,6 +134,7 @@ void qmp_x_blockdev_amend(const char *job_id,
 s->force = has_force ? force : false;
 
 if (blockdev_amend_pre_run(s, errp)) {
+job_early_fail(>common);
 return;
 }
 
-- 
2.34.1

Re: [PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

[Laurent Vivier]

Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :

From: Marc-André Lureau 

GLib g_get_real_time() is an alternative to gettimeofday().

For semihosting, a few bits are lost on POSIX host, but this shouldn't
be a big concern.

Signed-off-by: Marc-André Lureau 
---
  blockdev.c |  8 
  hw/rtc/m41t80.c|  6 +++---
  hw/virtio/virtio-balloon.c |  9 +
  qapi/qmp-event.c   | 12 +---
  qemu-img.c |  8 
  qga/commands-posix.c   | 11 +--
  target/m68k/m68k-semi.c| 22 ++
  target/nios2/nios2-semi.c  | 24 +++-
  8 files changed, 39 insertions(+), 61 deletions(-)


...

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 75dbaab68ea9..3311a4ca0167 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -138,16 +138,7 @@ void qmp_guest_shutdown(bool has_mode, const char *mode, 
Error **errp)
  
  int64_t qmp_guest_get_time(Error **errp)

  {
-   int ret;
-   qemu_timeval tq;
-
-   ret = qemu_gettimeofday();
-   if (ret < 0) {
-   error_setg_errno(errp, errno, "Failed to get time");
-   return -1;
-   }
-
-   return tq.tv_sec * 10LL + tq.tv_usec * 1000;
+return g_get_real_time();


now you return microseconds, before it was nanoseconds.

qga/qapi-schema.json:

##
# @guest-get-time:
#
# Get the information about guest's System Time relative to
# the Epoch of 1970-01-01 in UTC.
#
# Returns: Time in nanoseconds.
#
# Since: 1.5
##
{ 'command': 'guest-get-time',
  'returns': 'int' }

Except this problem:

Reviewed-by: Laurent Vivier 

Re: [PATCH 0/5] block/amend: Fix failures seen in iotest 296

[Kevin Wolf]

Am 04.03.2022 um 16:37 hat Hanna Reitz geschrieben:
> Hi,
> 
> I’ve tried basing my block branch on Kevin’s and noticed that after
> “crypto: perform permission checks under BQL”, iotest 296 was failing.
> I/We have debugged those failures and here are fixes for it.
> 
> Hence, this series is based on Kevin’s block branch
> (efa33ed9b298d39e2b8c19c5f4bdd80a3b632260 at the time of writing this
> cover letter).  I’ve pushed it here:
> 
>   https://gitlab.com/hreitz/qemu/-/commits/amend-job-fixes-v1
> 
> Patch 1 adds clean-up of the amend job in an error path that said commit
> adds to qmp_x_blockdev_amend().
> 
> Patch 2 changes the type of a JobDriver callback added in that commit;
> together with patch 3, this is kind of a matter of style, but it can
> also replace patch 3 and fix the bug that it fixes in another way.
> 
> Patch 3 fixes a permission bug: When changing the permissions fails
> before amend, block/crypto will still keep updating_keys to be true.
> Without patch 2, that will remains so indefinitely and then
> block_crypto_child_perms() will continue to unshare the CONSISTENT_READ
> permission, which is wrong.  (Patch 2 fixes this problem, too,
> specifically because with it, block_crypto_amend_cleanup() will always
> be called when the job is dismissed, and so updating_keys will be reset
> at least then.)
> 
> Patch 4 fixes an issue that’s not related to “crypto: perform permission
> checks under BQL”, but it became appearent only while debugging the
> other issues here, so it’s part of this series, too.
> 
> Patch 5 fixes the test itself.  It expects permission-related errors to
> occur when the job is already running, not as an immediate result of the
> QMP x-blockdev-amend command.  “crypto: perform permission checks under
> BQL” has changed this, so the test needs to take that into account.
> 
> 
> Ideally, I believe the following patches should be squashed into
> “crypto: perform permission checks under BQL” lest bisect breaks:
> - Patch 1
> - Patch 2 or 3 (or both)
> - Patch 5
> 
> But if that isn’t feasible, we can just take the whole series on top.

Thanks, I'm squashing in 1, 3 and 5 and taking 2 and 4 on top.

Kevin

Re: [PATCH 06/12] tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:24PM +, Daniel P. Berrangé wrote:
> When developing an I/O test it is typical to add some logic to the
> test script, run it to view the output diff, and then apply the
> output diff to the reference file. This can be drastically simplified
> by letting the test runner update the reference file in place.
> 
> By setting 'QEMU_IOTESTS_REGEN=1', the test runner will report the
> failure and show the diff, but at the same time update the reference
> file. So next time the I/O test is run it will succeed.
> 
> Continuing to display the diff when updating the reference gives the
> developer a chance to review what was changed.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  tests/qemu-iotests/testrunner.py | 6 ++
>  1 file changed, 6 insertions(+)

I like it - I've wished for similar when writing a test.

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[PULL 00/50] Block layer patches

[Kevin Wolf]

The following changes since commit 6629bf78aac7e53f83fd0bcbdbe322e2302dfd1f:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20220302' 
into staging (2022-03-03 14:46:48 +)

are available in the Git repository at:

  https://gitlab.com/kmwolf/qemu.git tags/for-upstream

for you to fetch changes up to a5df6d1db7db37636c8624bf4a9df9da645853aa:

  block/amend: Keep strong reference to BDS (2022-03-04 17:15:33 +0100)


Block layer patches

- qemu-storage-daemon: Add --daemonize
- Fix x-blockdev-amend and block node activation code which incorrectly
  executed code in the iothread that must run in the main thread.
- Add macros for coroutine-safe TLS variables (required for correctness
  with LTO)
- Fix crashes with concurrent I/O and bdrv_refresh_limits()
- Split block APIs in global state and I/O
- iotests: Don't refuse to run at all without GNU sed, just skip tests
  that need it


Emanuele Giuseppe Esposito (36):
  crypto: perform permission checks under BQL
  crypto: distinguish between main loop and I/O in 
block_crypto_amend_options_generic_luks
  block: introduce bdrv_activate
  block: rename bdrv_invalidate_cache_all, blk_invalidate_cache and 
test_sync_op_invalidate_cache
  block: move BQL logic of bdrv_co_invalidate_cache in bdrv_activate
  main-loop.h: introduce qemu_in_main_thread()
  main loop: macros to mark GS and I/O functions
  include/block/block: split header into I/O and global state API
  assertions for block global state API
  IO_CODE and IO_OR_GS_CODE for block I/O API
  block/export/fuse.c: allow writable exports to take RESIZE permission
  include/sysemu/block-backend: split header into I/O and global state (GS) 
API
  block/block-backend.c: assertions for block-backend
  IO_CODE and IO_OR_GS_CODE for block-backend I/O API
  block.c: assertions to the block layer permissions API
  include/block/block_int: split header into I/O and global state API
  assertions for block_int global state API
  IO_CODE and IO_OR_GS_CODE for block_int I/O API
  block: introduce assert_bdrv_graph_writable
  include/block/blockjob_int.h: split header into I/O and GS API
  GS and IO CODE macros for blockjob_int.h
  block.c: add assertions to static functions
  include/block/blockjob.h: global state API
  assertions for blockjob.h global state API
  include/sysemu/blockdev.h: global state API
  assertions for blockdev.h global state API
  include/block/snapshot: global state API + assertions
  block/copy-before-write.h: global state API + assertions
  block/coroutines: I/O and "I/O or GS" API
  block_int-common.h: split function pointers in BlockDriver
  block_int-common.h: assertions in the callers of BlockDriver function 
pointers
  block_int-common.h: split function pointers in BdrvChildClass
  block_int-common.h: assertions in the callers of BdrvChildClass function 
pointers
  block-backend-common.h: split function pointers in BlockDevOps
  job.h: split function pointers in JobDriver
  job.h: assertions in the callers of JobDriver function pointers

Hanna Reitz (9):
  os-posix: Add os_set_daemonize()
  qsd: Add pre-init argument parsing pass
  qsd: Add --daemonize
  iotests/185: Add post-READY quit tests
  block: Make bdrv_refresh_limits() non-recursive
  iotests: Allow using QMP with the QSD
  iotests/graph-changes-while-io: New test
  block/amend: Always call .bdrv_amend_clean()
  block/amend: Keep strong reference to BDS

Stefan Hajnoczi (4):
  tls: add macros for coroutine-safe TLS variables
  util/async: replace __thread with QEMU TLS macros
  rcu: use coroutine TLS macros
  cpus: use coroutine TLS macros for iothread_locked

Thomas Huth (1):
  tests/qemu-iotests: Rework the checks and spots using GNU sed

 docs/tools/qemu-storage-daemon.rst |7 +
 block/copy-before-write.h  |7 +
 block/coroutines.h |   81 +-
 include/block/block-common.h   |  418 ++
 include/block/block-global-state.h |  253 
 include/block/block-io.h   |  368 +
 include/block/block.h  |  878 +---
 include/block/block_int-common.h   | 1222 
 include/block/block_int-global-state.h |  329 +
 include/block/block_int-io.h   |  185 +++
 include/block/block_int.h  | 1475 +---
 include/block/blockjob.h   |   29 +-
 include/block/blockjob_int.h   |   28 +
 include/block/snapshot.h   |   13 +-
 include/qemu/coroutine-tls.h

[PULL 18/50] IO_CODE and IO_OR_GS_CODE for block I/O API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Mark all I/O functions with IO_CODE, and all "I/O OR GS" with
IO_OR_GS_CODE.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-6-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block-io.h |  1 +
 block.c  | 37 +-
 block/dirty-bitmap.c |  1 +
 block/io.c   | 43 ++--
 4 files changed, 79 insertions(+), 3 deletions(-)

diff --git a/include/block/block-io.h b/include/block/block-io.h
index 2cbdfb92c9..5e3f346806 100644
--- a/include/block/block-io.h
+++ b/include/block/block-io.h
@@ -264,6 +264,7 @@ void bdrv_drained_end_no_poll(BlockDriverState *bs, int 
*drained_end_counter);
 
 #define BDRV_POLL_WHILE(bs, cond) ({   \
 BlockDriverState *bs_ = (bs);  \
+IO_OR_GS_CODE();   \
 AIO_WAIT_WHILE(bdrv_get_aio_context(bs_),  \
cond); })
 
diff --git a/block.c b/block.c
index ed1026e4d8..fd3f565a16 100644
--- a/block.c
+++ b/block.c
@@ -137,6 +137,7 @@ size_t bdrv_opt_mem_align(BlockDriverState *bs)
 /* page size or 4k (hdd sector size) should be on the safe side */
 return MAX(4096, qemu_real_host_page_size);
 }
+IO_CODE();
 
 return bs->bl.opt_mem_alignment;
 }
@@ -147,6 +148,7 @@ size_t bdrv_min_mem_align(BlockDriverState *bs)
 /* page size or 4k (hdd sector size) should be on the safe side */
 return MAX(4096, qemu_real_host_page_size);
 }
+IO_CODE();
 
 return bs->bl.min_mem_alignment;
 }
@@ -272,12 +274,15 @@ void bdrv_parse_filename_strip_prefix(const char 
*filename, const char *prefix,
  * image is inactivated. */
 bool bdrv_is_read_only(BlockDriverState *bs)
 {
+IO_CODE();
 return !(bs->open_flags & BDRV_O_RDWR);
 }
 
 int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only,
bool ignore_allow_rdw, Error **errp)
 {
+IO_CODE();
+
 /* Do not set read_only if copy_on_read is enabled */
 if (bs->copy_on_read && read_only) {
 error_setg(errp, "Can't set node '%s' to r/o with copy-on-read 
enabled",
@@ -311,6 +316,7 @@ int bdrv_apply_auto_read_only(BlockDriverState *bs, const 
char *errmsg,
   Error **errp)
 {
 int ret = 0;
+IO_CODE();
 
 if (!(bs->open_flags & BDRV_O_RDWR)) {
 return 0;
@@ -757,6 +763,7 @@ int coroutine_fn bdrv_co_delete_file(BlockDriverState *bs, 
Error **errp)
 Error *local_err = NULL;
 int ret;
 
+IO_CODE();
 assert(bs != NULL);
 
 if (!bs->drv) {
@@ -782,6 +789,7 @@ void coroutine_fn 
bdrv_co_delete_file_noerr(BlockDriverState *bs)
 {
 Error *local_err = NULL;
 int ret;
+IO_CODE();
 
 if (!bs) {
 return;
@@ -1442,6 +1450,7 @@ static int bdrv_child_cb_update_filename(BdrvChild *c, 
BlockDriverState *base,
 AioContext *child_of_bds_get_parent_aio_context(BdrvChild *c)
 {
 BlockDriverState *bs = c->opaque;
+IO_CODE();
 
 return bdrv_get_aio_context(bs);
 }
@@ -1464,6 +1473,7 @@ const BdrvChildClass child_of_bds = {
 
 AioContext *bdrv_child_get_parent_aio_context(BdrvChild *c)
 {
+IO_CODE();
 return c->klass->get_parent_aio_context(c);
 }
 
@@ -2077,6 +2087,7 @@ static bool 
bdrv_is_writable_after_reopen(BlockDriverState *bs,
  */
 bool bdrv_is_writable(BlockDriverState *bs)
 {
+IO_CODE();
 return bdrv_is_writable_after_reopen(bs, NULL);
 }
 
@@ -5706,6 +5717,8 @@ static int64_t 
bdrv_sum_allocated_file_size(BlockDriverState *bs)
 int64_t bdrv_get_allocated_file_size(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
+IO_CODE();
+
 if (!drv) {
 return -ENOMEDIUM;
 }
@@ -5755,6 +5768,7 @@ int64_t bdrv_get_allocated_file_size(BlockDriverState *bs)
 BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts *opts,
BlockDriverState *in_bs, Error **errp)
 {
+IO_CODE();
 if (!drv->bdrv_measure) {
 error_setg(errp, "Block driver '%s' does not support size measurement",
drv->format_name);
@@ -5770,6 +5784,7 @@ BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts 
*opts,
 int64_t bdrv_nb_sectors(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
+IO_CODE();
 
 if (!drv)
 return -ENOMEDIUM;
@@ -5790,6 +5805,7 @@ int64_t bdrv_nb_sectors(BlockDriverState *bs)
 int64_t bdrv_getlength(BlockDriverState *bs)
 {
 int64_t ret = bdrv_nb_sectors(bs);
+IO_CODE();
 
 if (ret < 0) {
 return ret;
@@ -5804,12 +5820,14 @@ int64_t bdrv_getlength(BlockDriverState *bs)
 void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr)
 {
 int64_t nb_sectors = bdrv_nb_sectors(bs);
+IO_CODE();
 
 *nb_sectors_ptr = nb_sectors < 0 ? 0 : nb_sectors;
 }
 
 bool bdrv_is_sg(BlockDriverState *bs)
 {
+IO_CODE();
 return bs->sg;
 }
 

[PULL 17/50] assertions for block global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

All the global state (GS) API functions will check that
qemu_in_main_thread() returns true. If not, it means
that the safety of BQL cannot be guaranteed, and
they need to be moved to I/O.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-5-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c| 125 -
 block/commit.c |   2 +
 block/io.c |  11 +
 blockdev.c |   1 +
 4 files changed, 137 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 7483dfaddc..ed1026e4d8 100644
--- a/block.c
+++ b/block.c
@@ -387,12 +387,14 @@ static char *bdrv_make_absolute_filename(BlockDriverState 
*relative_to,
 
 char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
 {
+GLOBAL_STATE_CODE();
 return bdrv_make_absolute_filename(bs, bs->backing_file, errp);
 }
 
 void bdrv_register(BlockDriver *bdrv)
 {
 assert(bdrv->format_name);
+GLOBAL_STATE_CODE();
 QLIST_INSERT_HEAD(_drivers, bdrv, list);
 }
 
@@ -401,6 +403,8 @@ BlockDriverState *bdrv_new(void)
 BlockDriverState *bs;
 int i;
 
+GLOBAL_STATE_CODE();
+
 bs = g_new0(BlockDriverState, 1);
 QLIST_INIT(>dirty_bitmaps);
 for (i = 0; i < BLOCK_OP_TYPE_MAX; i++) {
@@ -443,6 +447,8 @@ BlockDriver *bdrv_find_format(const char *format_name)
 BlockDriver *drv1;
 int i;
 
+GLOBAL_STATE_CODE();
+
 drv1 = bdrv_do_find_format(format_name);
 if (drv1) {
 return drv1;
@@ -492,6 +498,7 @@ static int bdrv_format_is_whitelisted(const char 
*format_name, bool read_only)
 
 int bdrv_is_whitelisted(BlockDriver *drv, bool read_only)
 {
+GLOBAL_STATE_CODE();
 return bdrv_format_is_whitelisted(drv->format_name, read_only);
 }
 
@@ -527,6 +534,8 @@ int bdrv_create(BlockDriver *drv, const char* filename,
 {
 int ret;
 
+GLOBAL_STATE_CODE();
+
 Coroutine *co;
 CreateCo cco = {
 .drv = drv,
@@ -702,6 +711,8 @@ int bdrv_create_file(const char *filename, QemuOpts *opts, 
Error **errp)
 QDict *qdict;
 int ret;
 
+GLOBAL_STATE_CODE();
+
 drv = bdrv_find_protocol(filename, true, errp);
 if (drv == NULL) {
 return -ENOENT;
@@ -799,6 +810,7 @@ int bdrv_probe_blocksizes(BlockDriverState *bs, BlockSizes 
*bsz)
 {
 BlockDriver *drv = bs->drv;
 BlockDriverState *filtered = bdrv_filter_bs(bs);
+GLOBAL_STATE_CODE();
 
 if (drv && drv->bdrv_probe_blocksizes) {
 return drv->bdrv_probe_blocksizes(bs, bsz);
@@ -819,6 +831,7 @@ int bdrv_probe_geometry(BlockDriverState *bs, HDGeometry 
*geo)
 {
 BlockDriver *drv = bs->drv;
 BlockDriverState *filtered = bdrv_filter_bs(bs);
+GLOBAL_STATE_CODE();
 
 if (drv && drv->bdrv_probe_geometry) {
 return drv->bdrv_probe_geometry(bs, geo);
@@ -910,6 +923,7 @@ BlockDriver *bdrv_find_protocol(const char *filename,
 const char *p;
 int i;
 
+GLOBAL_STATE_CODE();
 /* TODO Drivers without bdrv_file_open must be specified explicitly */
 
 /*
@@ -1634,6 +1648,8 @@ BlockDriverState *bdrv_new_open_driver_opts(BlockDriver 
*drv,
 BlockDriverState *bs;
 int ret;
 
+GLOBAL_STATE_CODE();
+
 bs = bdrv_new();
 bs->open_flags = flags;
 bs->options = options ?: qdict_new();
@@ -1659,6 +1675,7 @@ BlockDriverState *bdrv_new_open_driver_opts(BlockDriver 
*drv,
 BlockDriverState *bdrv_new_open_driver(BlockDriver *drv, const char *node_name,
int flags, Error **errp)
 {
+GLOBAL_STATE_CODE();
 return bdrv_new_open_driver_opts(drv, node_name, NULL, flags, errp);
 }
 
@@ -3094,6 +3111,8 @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs,
 BdrvChild *child = NULL;
 Transaction *tran = tran_new();
 
+GLOBAL_STATE_CODE();
+
 ret = bdrv_attach_child_noperm(parent_bs, child_bs, child_name, 
child_class,
child_role, , tran, errp);
 if (ret < 0) {
@@ -3120,6 +3139,8 @@ void bdrv_root_unref_child(BdrvChild *child)
 {
 BlockDriverState *child_bs;
 
+GLOBAL_STATE_CODE();
+
 child_bs = child->bs;
 bdrv_detach_child();
 bdrv_unref(child_bs);
@@ -3194,6 +3215,7 @@ static void bdrv_unset_inherits_from(BlockDriverState 
*root, BdrvChild *child,
 /* Callers must ensure that child->frozen is false. */
 void bdrv_unref_child(BlockDriverState *parent, BdrvChild *child)
 {
+GLOBAL_STATE_CODE();
 if (child == NULL) {
 return;
 }
@@ -3344,6 +3366,7 @@ int bdrv_set_backing_hd(BlockDriverState *bs, 
BlockDriverState *backing_hd,
 int ret;
 Transaction *tran = tran_new();
 
+GLOBAL_STATE_CODE();
 bdrv_drained_begin(bs);
 
 ret = bdrv_set_backing_noperm(bs, backing_hd, tran, errp);
@@ -3383,6 +3406,8 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 QDict *tmp_parent_options = NULL;
 Error *local_err = NULL;
 
+GLOBAL_STATE_CODE();
+
 if 

[PULL 20/50] include/sysemu/block-backend: split header into I/O and global state (GS) API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Similarly to the previous patches, split block-backend.h
in block-backend-io.h and block-backend-global-state.h

In addition, remove "block/block.h" include as it seems
it is not necessary anymore, together with "qemu/iov.h"

block-backend-common.h contains the structures shared between
the two headers, and the functions that can't be categorized as
I/O or global state.

Assertions are added in the next patch.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-8-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/sysemu/block-backend-common.h   |  84 ++
 include/sysemu/block-backend-global-state.h | 116 +
 include/sysemu/block-backend-io.h   | 159 
 include/sysemu/block-backend.h  | 269 +---
 block/block-backend.c   |   9 +-
 5 files changed, 368 insertions(+), 269 deletions(-)
 create mode 100644 include/sysemu/block-backend-common.h
 create mode 100644 include/sysemu/block-backend-global-state.h
 create mode 100644 include/sysemu/block-backend-io.h

diff --git a/include/sysemu/block-backend-common.h 
b/include/sysemu/block-backend-common.h
new file mode 100644
index 00..6963bbf45a
--- /dev/null
+++ b/include/sysemu/block-backend-common.h
@@ -0,0 +1,84 @@
+/*
+ * QEMU Block backends
+ *
+ * Copyright (C) 2014-2016 Red Hat, Inc.
+ *
+ * Authors:
+ *  Markus Armbruster ,
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1
+ * or later.  See the COPYING.LIB file in the top-level directory.
+ */
+
+#ifndef BLOCK_BACKEND_COMMON_H
+#define BLOCK_BACKEND_COMMON_H
+
+#include "qemu/iov.h"
+#include "block/throttle-groups.h"
+
+/*
+ * TODO Have to include block/block.h for a bunch of block layer
+ * types.  Unfortunately, this pulls in the whole BlockDriverState
+ * API, which we don't want used by many BlockBackend users.  Some of
+ * the types belong here, and the rest should be split into a common
+ * header and one for the BlockDriverState API.
+ */
+#include "block/block.h"
+
+/* Callbacks for block device models */
+typedef struct BlockDevOps {
+/*
+ * Runs when virtual media changed (monitor commands eject, change)
+ * Argument load is true on load and false on eject.
+ * Beware: doesn't run when a host device's physical media
+ * changes.  Sure would be useful if it did.
+ * Device models with removable media must implement this callback.
+ */
+void (*change_media_cb)(void *opaque, bool load, Error **errp);
+/*
+ * Runs when an eject request is issued from the monitor, the tray
+ * is closed, and the medium is locked.
+ * Device models that do not implement is_medium_locked will not need
+ * this callback.  Device models that can lock the medium or tray might
+ * want to implement the callback and unlock the tray when "force" is
+ * true, even if they do not support eject requests.
+ */
+void (*eject_request_cb)(void *opaque, bool force);
+/*
+ * Is the virtual tray open?
+ * Device models implement this only when the device has a tray.
+ */
+bool (*is_tray_open)(void *opaque);
+/*
+ * Is the virtual medium locked into the device?
+ * Device models implement this only when device has such a lock.
+ */
+bool (*is_medium_locked)(void *opaque);
+/*
+ * Runs when the size changed (e.g. monitor command block_resize)
+ */
+void (*resize_cb)(void *opaque);
+/*
+ * Runs when the backend receives a drain request.
+ */
+void (*drained_begin)(void *opaque);
+/*
+ * Runs when the backend's last drain request ends.
+ */
+void (*drained_end)(void *opaque);
+/*
+ * Is the device still busy?
+ */
+bool (*drained_poll)(void *opaque);
+} BlockDevOps;
+
+/*
+ * This struct is embedded in (the private) BlockBackend struct and contains
+ * fields that must be public. This is in particular for QLIST_ENTRY() and
+ * friends so that BlockBackends can be kept in lists outside block-backend.c
+ */
+typedef struct BlockBackendPublic {
+ThrottleGroupMember throttle_group_member;
+} BlockBackendPublic;
+
+#endif /* BLOCK_BACKEND_COMMON_H */
diff --git a/include/sysemu/block-backend-global-state.h 
b/include/sysemu/block-backend-global-state.h
new file mode 100644
index 00..2e93a74679
--- /dev/null
+++ b/include/sysemu/block-backend-global-state.h
@@ -0,0 +1,116 @@
+/*
+ * QEMU Block backends
+ *
+ * Copyright (C) 2014-2016 Red Hat, Inc.
+ *
+ * Authors:
+ *  Markus Armbruster ,
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1
+ * or later.  See the COPYING.LIB file in the top-level directory.
+ */
+
+#ifndef BLOCK_BACKEND_GS_H
+#define BLOCK_BACKEND_GS_H
+
+#include "block-backend-common.h"
+
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS 

[PULL 22/50] IO_CODE and IO_OR_GS_CODE for block-backend I/O API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Mark all I/O functions with IO_CODE, and all "I/O OR GS" with
IO_OR_GS_CODE.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-10-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/sysemu/block-backend-io.h |  2 ++
 block/block-backend.c | 58 +++
 2 files changed, 60 insertions(+)

diff --git a/include/sysemu/block-backend-io.h 
b/include/sysemu/block-backend-io.h
index 045fe97714..6517c39295 100644
--- a/include/sysemu/block-backend-io.h
+++ b/include/sysemu/block-backend-io.h
@@ -121,6 +121,7 @@ static inline int coroutine_fn blk_co_pread(BlockBackend 
*blk, int64_t offset,
 BdrvRequestFlags flags)
 {
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
+IO_OR_GS_CODE();
 
 assert(bytes <= SIZE_MAX);
 
@@ -132,6 +133,7 @@ static inline int coroutine_fn blk_co_pwrite(BlockBackend 
*blk, int64_t offset,
  BdrvRequestFlags flags)
 {
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
+IO_OR_GS_CODE();
 
 assert(bytes <= SIZE_MAX);
 
diff --git a/block/block-backend.c b/block/block-backend.c
index 4476b61b8b..be7adce246 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -733,6 +733,7 @@ void monitor_remove_blk(BlockBackend *blk)
  */
 const char *blk_name(const BlockBackend *blk)
 {
+IO_CODE();
 return blk->name ?: "";
 }
 
@@ -759,6 +760,7 @@ BlockBackend *blk_by_name(const char *name)
  */
 BlockDriverState *blk_bs(BlockBackend *blk)
 {
+IO_CODE();
 return blk->root ? blk->root->bs : NULL;
 }
 
@@ -1009,6 +1011,7 @@ DeviceState *blk_get_attached_dev(BlockBackend *blk)
 char *blk_get_attached_dev_id(BlockBackend *blk)
 {
 DeviceState *dev = blk->dev;
+IO_CODE();
 
 if (!dev) {
 return g_strdup("");
@@ -1171,6 +1174,7 @@ void blk_iostatus_enable(BlockBackend *blk)
  * enables it _and_ the VM is configured to stop on errors */
 bool blk_iostatus_is_enabled(const BlockBackend *blk)
 {
+IO_CODE();
 return (blk->iostatus_enabled &&
(blk->on_write_error == BLOCKDEV_ON_ERROR_ENOSPC ||
 blk->on_write_error == BLOCKDEV_ON_ERROR_STOP   ||
@@ -1199,6 +1203,7 @@ void blk_iostatus_reset(BlockBackend *blk)
 
 void blk_iostatus_set_err(BlockBackend *blk, int error)
 {
+IO_CODE();
 assert(blk_iostatus_is_enabled(blk));
 if (blk->iostatus == BLOCK_DEVICE_IO_STATUS_OK) {
 blk->iostatus = error == ENOSPC ? BLOCK_DEVICE_IO_STATUS_NOSPACE :
@@ -1208,16 +1213,19 @@ void blk_iostatus_set_err(BlockBackend *blk, int error)
 
 void blk_set_allow_write_beyond_eof(BlockBackend *blk, bool allow)
 {
+IO_CODE();
 blk->allow_write_beyond_eof = allow;
 }
 
 void blk_set_allow_aio_context_change(BlockBackend *blk, bool allow)
 {
+IO_CODE();
 blk->allow_aio_context_change = allow;
 }
 
 void blk_set_disable_request_queuing(BlockBackend *blk, bool disable)
 {
+IO_CODE();
 blk->disable_request_queuing = disable;
 }
 
@@ -1301,6 +1309,7 @@ int coroutine_fn blk_co_preadv(BlockBackend *blk, int64_t 
offset,
BdrvRequestFlags flags)
 {
 int ret;
+IO_OR_GS_CODE();
 
 blk_inc_in_flight(blk);
 ret = blk_co_do_preadv(blk, offset, bytes, qiov, flags);
@@ -1352,6 +1361,7 @@ int coroutine_fn blk_co_pwritev_part(BlockBackend *blk, 
int64_t offset,
  BdrvRequestFlags flags)
 {
 int ret;
+IO_OR_GS_CODE();
 
 blk_inc_in_flight(blk);
 ret = blk_co_do_pwritev_part(blk, offset, bytes, qiov, qiov_offset, flags);
@@ -1364,6 +1374,7 @@ int coroutine_fn blk_co_pwritev(BlockBackend *blk, 
int64_t offset,
 int64_t bytes, QEMUIOVector *qiov,
 BdrvRequestFlags flags)
 {
+IO_OR_GS_CODE();
 return blk_co_pwritev_part(blk, offset, bytes, qiov, 0, flags);
 }
 
@@ -1392,6 +1403,7 @@ typedef struct BlkRwCo {
 int blk_pwrite_zeroes(BlockBackend *blk, int64_t offset,
   int64_t bytes, BdrvRequestFlags flags)
 {
+IO_OR_GS_CODE();
 return blk_pwritev_part(blk, offset, bytes, NULL, 0,
 flags | BDRV_REQ_ZERO_WRITE);
 }
@@ -1404,11 +1416,13 @@ int blk_make_zero(BlockBackend *blk, BdrvRequestFlags 
flags)
 
 void blk_inc_in_flight(BlockBackend *blk)
 {
+IO_CODE();
 qatomic_inc(>in_flight);
 }
 
 void blk_dec_in_flight(BlockBackend *blk)
 {
+IO_CODE();
 qatomic_dec(>in_flight);
 aio_wait_kick();
 }
@@ -1427,6 +1441,7 @@ BlockAIOCB *blk_abort_aio_request(BlockBackend *blk,
   void *opaque, int ret)
 {
 struct BlockBackendAIOCB *acb;
+IO_CODE();
 
 blk_inc_in_flight(blk);
 acb = blk_aio_get(_backend_aiocb_info, blk, cb, opaque);
@@ -1534,6 +1549,7 @@ BlockAIOCB *blk_aio_pwrite_zeroes(BlockBackend *blk, 
int64_t offset,

[PULL 26/50] IO_CODE and IO_OR_GS_CODE for block_int I/O API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Mark all I/O functions with IO_CODE, and all "I/O OR GS" with
IO_OR_GS_CODE.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-14-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int-io.h |  6 ++
 block.c  | 14 +-
 block/block-backend.c|  2 ++
 block/dirty-bitmap.c |  3 +++
 block/io.c   | 13 +
 5 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h
index 677b8b49f4..3da5f01c42 100644
--- a/include/block/block_int-io.h
+++ b/include/block/block_int-io.h
@@ -50,6 +50,7 @@ static inline int coroutine_fn bdrv_co_pread(BdrvChild *child,
 int64_t offset, unsigned int bytes, void *buf, BdrvRequestFlags flags)
 {
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
+IO_CODE();
 
 return bdrv_co_preadv(child, offset, bytes, , flags);
 }
@@ -58,6 +59,7 @@ static inline int coroutine_fn bdrv_co_pwrite(BdrvChild 
*child,
 int64_t offset, unsigned int bytes, void *buf, BdrvRequestFlags flags)
 {
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
+IO_CODE();
 
 return bdrv_co_pwritev(child, offset, bytes, , flags);
 }
@@ -120,21 +122,25 @@ BlockDriverState 
*bdrv_backing_chain_next(BlockDriverState *bs);
 
 static inline BlockDriverState *bdrv_cow_bs(BlockDriverState *bs)
 {
+IO_CODE();
 return child_bs(bdrv_cow_child(bs));
 }
 
 static inline BlockDriverState *bdrv_filter_bs(BlockDriverState *bs)
 {
+IO_CODE();
 return child_bs(bdrv_filter_child(bs));
 }
 
 static inline BlockDriverState *bdrv_filter_or_cow_bs(BlockDriverState *bs)
 {
+IO_CODE();
 return child_bs(bdrv_filter_or_cow_child(bs));
 }
 
 static inline BlockDriverState *bdrv_primary_bs(BlockDriverState *bs)
 {
+IO_CODE();
 return child_bs(bdrv_primary_child(bs));
 }
 
diff --git a/block.c b/block.c
index 1751b14baf..f2a7350548 100644
--- a/block.c
+++ b/block.c
@@ -999,6 +999,7 @@ BlockDriver *bdrv_probe_all(const uint8_t *buf, int 
buf_size,
 {
 int score_max = 0, score;
 BlockDriver *drv = NULL, *d;
+IO_CODE();
 
 QLIST_FOREACH(d, _drivers, list) {
 if (d->bdrv_probe) {
@@ -1051,6 +1052,7 @@ static int find_image_format(BlockBackend *file, const 
char *filename,
 int refresh_total_sectors(BlockDriverState *bs, int64_t hint)
 {
 BlockDriver *drv = bs->drv;
+IO_CODE();
 
 if (!drv) {
 return -ENOMEDIUM;
@@ -6197,6 +6199,7 @@ const char *bdrv_get_parent_name(const BlockDriverState 
*bs)
 {
 BdrvChild *c;
 const char *name;
+IO_CODE();
 
 /* If multiple parents have a name, just pick the first one. */
 QLIST_FOREACH(c, >parents, next_parent) {
@@ -7933,6 +7936,8 @@ int bdrv_make_empty(BdrvChild *c, Error **errp)
  */
 BdrvChild *bdrv_cow_child(BlockDriverState *bs)
 {
+IO_CODE();
+
 if (!bs || !bs->drv) {
 return NULL;
 }
@@ -7956,6 +7961,7 @@ BdrvChild *bdrv_cow_child(BlockDriverState *bs)
 BdrvChild *bdrv_filter_child(BlockDriverState *bs)
 {
 BdrvChild *c;
+IO_CODE();
 
 if (!bs || !bs->drv) {
 return NULL;
@@ -7987,6 +7993,7 @@ BdrvChild *bdrv_filter_or_cow_child(BlockDriverState *bs)
 {
 BdrvChild *cow_child = bdrv_cow_child(bs);
 BdrvChild *filter_child = bdrv_filter_child(bs);
+IO_CODE();
 
 /* Filter nodes cannot have COW backing files */
 assert(!(cow_child && filter_child));
@@ -8007,6 +8014,7 @@ BdrvChild *bdrv_filter_or_cow_child(BlockDriverState *bs)
 BdrvChild *bdrv_primary_child(BlockDriverState *bs)
 {
 BdrvChild *c, *found = NULL;
+IO_CODE();
 
 QLIST_FOREACH(c, >children, next) {
 if (c->role & BDRV_CHILD_PRIMARY) {
@@ -8069,6 +8077,7 @@ BlockDriverState 
*bdrv_skip_implicit_filters(BlockDriverState *bs)
  */
 BlockDriverState *bdrv_skip_filters(BlockDriverState *bs)
 {
+IO_CODE();
 return bdrv_do_skip_filters(bs, false);
 }
 
@@ -8078,6 +8087,7 @@ BlockDriverState *bdrv_skip_filters(BlockDriverState *bs)
  */
 BlockDriverState *bdrv_backing_chain_next(BlockDriverState *bs)
 {
+IO_CODE();
 return bdrv_skip_filters(bdrv_cow_bs(bdrv_skip_filters(bs)));
 }
 
@@ -8113,8 +8123,8 @@ static bool 
bdrv_bsc_range_overlaps_locked(BlockDriverState *bs,
  */
 bool bdrv_bsc_is_data(BlockDriverState *bs, int64_t offset, int64_t *pnum)
 {
+IO_CODE();
 RCU_READ_LOCK_GUARD();
-
 return bdrv_bsc_range_overlaps_locked(bs, offset, 1, pnum);
 }
 
@@ -8124,6 +8134,7 @@ bool bdrv_bsc_is_data(BlockDriverState *bs, int64_t 
offset, int64_t *pnum)
 void bdrv_bsc_invalidate_range(BlockDriverState *bs,
int64_t offset, int64_t bytes)
 {
+IO_CODE();
 RCU_READ_LOCK_GUARD();
 
 if (bdrv_bsc_range_overlaps_locked(bs, offset, bytes, NULL)) {
@@ -8138,6 +8149,7 @@ void bdrv_bsc_fill(BlockDriverState *bs, int64_t offset, 
int64_t bytes)
 {
 

[PULL 28/50] include/block/blockjob_int.h: split header into I/O and GS API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Since the I/O functions are not many, keep a single file.
Also split the function pointers in BlockJobDriver.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Stefan Hajnoczi 
Message-Id: <20220303151616.325444-16-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/blockjob_int.h | 28 
 1 file changed, 28 insertions(+)

diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h
index 6633d83da2..6bd9ae2b20 100644
--- a/include/block/blockjob_int.h
+++ b/include/block/blockjob_int.h
@@ -38,6 +38,13 @@ struct BlockJobDriver {
 /** Generic JobDriver callbacks and settings */
 JobDriver job_driver;
 
+/*
+ * I/O API functions. These functions are thread-safe.
+ *
+ * See include/block/block-io.h for more information about
+ * the I/O API.
+ */
+
 /*
  * Returns whether the job has pending requests for the child or will
  * submit new requests before the next pause point. This callback is polled
@@ -46,6 +53,13 @@ struct BlockJobDriver {
  */
 bool (*drained_poll)(BlockJob *job);
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 /*
  * If the callback is not NULL, it will be invoked before the job is
  * resumed in a new AioContext.  This is the place to move any resources
@@ -56,6 +70,13 @@ struct BlockJobDriver {
 void (*set_speed)(BlockJob *job, int64_t speed);
 };
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 /**
  * block_job_create:
  * @job_id: The id of the newly-created job, or %NULL to have one
@@ -98,6 +119,13 @@ void block_job_free(Job *job);
  */
 void block_job_user_resume(Job *job);
 
+/*
+ * I/O API functions. These functions are thread-safe.
+ *
+ * See include/block/block-io.h for more information about
+ * the I/O API.
+ */
+
 /**
  * block_job_ratelimit_get_delay:
  *
-- 
2.35.1

Re: [PATCH 10/12] tests/qemu-iotests: validate NBD TLS with hostname mismatch

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:28PM +, Daniel P. Berrangé wrote:
> This validates that connections to an NBD server where the certificate
> hostname does not match will fail. It further validates that using the
> new 'tls-hostname' override option can solve the failure.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  tests/qemu-iotests/233| 18 ++
>  tests/qemu-iotests/233.out| 15 +++
>  tests/qemu-iotests/common.tls |  7 ---
>  3 files changed, 37 insertions(+), 3 deletions(-)

> 
> diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
> index 050267298d..09cfb7039b 100755
> --- a/tests/qemu-iotests/233
> +++ b/tests/qemu-iotests/233
> @@ -106,6 +106,24 @@ $QEMU_IMG info --image-opts --object $obj2 \
>  $QEMU_NBD_PROG -L -b $nbd_tcp_addr -p $nbd_tcp_port --object $obj1 \
>  --tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
>  
> +echo
> +echo "== check TLS fail over TCP with mismatch hostname =="

Maybe s/mismatch/mismatched/, with corresponding .out tweaks.

> +obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
> +$QEMU_IMG info --image-opts --object $obj1 \
> +driver=nbd,host=localhost,port=$nbd_tcp_port,tls-creds=tls0 \
> +2>&1 | _filter_nbd
> +$QEMU_NBD_PROG -L -b localhost -p $nbd_tcp_port --object $obj1 \
> +--tls-creds=tls0 | _filter_qemu_nbd_exports
> +
> +echo
> +echo "== check TLS works over TCP with mismatch hostname and override =="
> +obj1=tls-creds-x509,dir=${tls_dir}/client1,endpoint=client,id=tls0
> +$QEMU_IMG info --image-opts --object $obj1 \
> +
> driver=nbd,host=localhost,port=$nbd_tcp_port,tls-creds=tls0,tls-hostname=127.0.0.1
>  \

And hence your need to filter both 127.0.0.1 and localhost.  Cute way
to force hostname mismatch without needing two hosts.

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [PATCH 11/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:29PM +, Daniel P. Berrangé wrote:
> This validates that connections to an NBD server running on a UNIX
> socket can use TLS, and require a TLS hostname override to pass
> certificate validation.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  tests/qemu-iotests/233 | 24 
>  tests/qemu-iotests/233.out | 15 +++
>  2 files changed, 39 insertions(+)

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [PATCH 12/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:30PM +, Daniel P. Berrangé wrote:
> This validates that connections to an NBD server running on a UNIX
> socket can use TLS with pre-shared keys (PSK).
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  tests/qemu-iotests/233| 28 
>  tests/qemu-iotests/233.out| 17 +
>  tests/qemu-iotests/common.tls | 24 
>  3 files changed, 69 insertions(+)

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [PATCH] hw/block: m25p80: Add support for w25q01jvq

[Philippe Mathieu-Daudé]

On 4/3/22 19:09, Patrick Williams wrote:

The w25q01jvq is a 128MB part.  Support is being added to the kernel[1]
and the two have been tested together.

1. https://lore.kernel.org/lkml/2022022209.23108-1-potin@quantatw.com/

Signed-off-by: Patrick Williams 
Cc: Potin Lai 
---
  hw/block/m25p80.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/hw/block/m25p80.c b/hw/block/m25p80.c
index c6bf3c6bfa..7d3d8b12e0 100644
--- a/hw/block/m25p80.c
+++ b/hw/block/m25p80.c
@@ -340,6 +340,7 @@ static const FlashPartInfo known_devices[] = {
  { INFO("w25q80bl",0xef4014,  0,  64 << 10,  16, ER_4K) },
  { INFO("w25q256", 0xef4019,  0,  64 << 10, 512, ER_4K) },
  { INFO("w25q512jv",   0xef4020,  0,  64 << 10, 1024, ER_4K) },
+{ INFO("w25q01jvq",   0xef4021,  0,  64 << 10, 2048, ER_4K) },
  };
  
  typedef enum {


Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH 08/12] tests/qemu-iotests: introduce filter for qemu-nbd export list

[Daniel P . Berrangé]

On Fri, Mar 04, 2022 at 10:43:45AM -0600, Eric Blake wrote:
> On Thu, Mar 03, 2022 at 04:03:26PM +, Daniel P. Berrangé wrote:
> > Introduce a filter for the output of qemu-nbd export list so it can be
> > reused in multiple tests.
> > 
> > The filter is a bit more permissive that what test 241 currently uses,
> > as its allows printing of the export count, along with any possible
> > error messages that might be emitted.
> > 
> > Signed-off-by: Daniel P. Berrangé 
> > ---
> >  tests/qemu-iotests/241   | 6 +++---
> >  tests/qemu-iotests/241.out   | 3 +++
> >  tests/qemu-iotests/common.filter | 5 +
> >  3 files changed, 11 insertions(+), 3 deletions(-)
> >
> 
> Reviewed-by: Eric Blake 

I'm going to post a v2 with a slight tweak to expose one more
interesting piece of (stable) info in the output:

diff --git a/tests/qemu-iotests/241.out b/tests/qemu-iotests/241.out
index db2d71ab9d..88e8cfcd7e 100644
--- a/tests/qemu-iotests/241.out
+++ b/tests/qemu-iotests/241.out
@@ -3,6 +3,7 @@ QA output created by 241
 === Exporting unaligned raw image, natural alignment ===
 
 exports available: 1
+ export: ''
   size:  1024
   min block: 1
 [{ "start": 0, "length": 1000, "depth": 0, "present": true, "zero": false, 
"data": true, "offset": OFFSET},
@@ -12,6 +13,7 @@ exports available: 1
 === Exporting unaligned raw image, forced server sector alignment ===
 
 exports available: 1
+ export: ''
   size:  1024
   min block: 512
 [{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, 
"data": true, "offset": OFFSET}]
@@ -23,6 +25,7 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' 
and probing guessed
 === Exporting unaligned raw image, forced client sector alignment ===
 
 exports available: 1
+ export: ''
   size:  1024
   min block: 1
 [{ "start": 0, "length": 1000, "depth": 0, "present": true, "zero": false, 
"data": true, "offset": OFFSET},
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 940c9884bd..14b6f80dcb 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -310,7 +310,7 @@ _filter_nbd()
 
 _filter_qemu_nbd_exports()
 {
-grep '\(exports available\|size\|min block\|qemu-nbd\):'
+grep '\(exports available\|export\|size\|min block\|qemu-nbd\):'
 }
 
 _filter_qmp_empty_return()


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[PATCH v2 05/12] block/nbd: don't restrict TLS usage to IP sockets

[Daniel P . Berrangé]

The TLS usage for NBD was restricted to IP sockets because validating
x509 certificates requires knowledge of the hostname that the client
is connecting to.

TLS does not have to use x509 certificates though, as PSK (pre-shared
keys) provide an alternative credential option. These have no
requirement for a hostname and can thus be trivially used for UNIX
sockets.

Furthermore, with the ability to overide the default hostname for
TLS validation in the previous patch, it is now also valid to want
to use x509 certificates with FD passing and UNIX sockets.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 block/nbd.c| 8 ++--
 blockdev-nbd.c | 6 --
 qemu-nbd.c | 8 +++-
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/block/nbd.c b/block/nbd.c
index 113aa5d3af..3ede47dec9 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -1838,13 +1838,9 @@ static int nbd_process_options(BlockDriverState *bs, 
QDict *options,
 goto error;
 }
 
-/* TODO SOCKET_ADDRESS_KIND_FD where fd has AF_INET or AF_INET6 */
-if (s->saddr->type != SOCKET_ADDRESS_TYPE_INET) {
-error_setg(errp, "TLS only supported over IP sockets");
-goto error;
-}
 s->tlshostname = g_strdup(qemu_opt_get(opts, "tls-hostname"));
-if (!s->tlshostname) {
+if (!s->tlshostname &&
+s->saddr->type == SOCKET_ADDRESS_TYPE_INET) {
 s->tlshostname = g_strdup(s->saddr->u.inet.host);
 }
 }
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index bdfa7ed3a5..9840d25a82 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -148,12 +148,6 @@ void nbd_server_start(SocketAddress *addr, const char 
*tls_creds,
 if (!nbd_server->tlscreds) {
 goto error;
 }
-
-/* TODO SOCKET_ADDRESS_TYPE_FD where fd has AF_INET or AF_INET6 */
-if (addr->type != SOCKET_ADDRESS_TYPE_INET) {
-error_setg(errp, "TLS is only supported with IPv4/IPv6");
-goto error;
-}
 }
 
 nbd_server->tlsauthz = g_strdup(tls_authz);
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 18d281aba3..713e7557a9 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -808,7 +808,9 @@ int main(int argc, char **argv)
 
 socket_activation = check_socket_activation();
 if (socket_activation == 0) {
-setup_address_and_port(, );
+if (!sockpath) {
+setup_address_and_port(, );
+}
 } else {
 /* Using socket activation - check user didn't use -p etc. */
 const char *err_msg = socket_activation_validate_opts(device, sockpath,
@@ -829,10 +831,6 @@ int main(int argc, char **argv)
 }
 
 if (tlscredsid) {
-if (sockpath) {
-error_report("TLS is only supported with IPv4/IPv6");
-exit(EXIT_FAILURE);
-}
 if (device) {
 error_report("TLS is not supported with a host device");
 exit(EXIT_FAILURE);
-- 
2.34.1

[PATCH v2 00/12] nbd: enable use of TLS on non-TCP transports and other TLS improvements

[Daniel P . Berrangé]

This series was principally motivated by a desire to enabl use of TLS
on non-TCP transports. For x509 certificates this means we need a way
to set the hostname to use for validation. This also lets us override
the hostname when connecting on a TCP transport that is tunnelled or
port-forwarded. It also unlocks the ability to use PSK (pre-shared
keys) with UNIX sockets which would always have worked, had it not
been blocked by explicit checks in NBD code.

NB, the first patch in this series is common with my corresponding
migration series for TLS

  https://lists.gnu.org/archive/html/qemu-devel/2022-03/msg00556.html

In v2:

  - Tweak the filter for qemu-nbd export list to expose export name
  - Add "since" tag to QAPI addition
  - Minor docs fixes

Daniel P. Berrangé (12):
  crypto: mandate a hostname when checking x509 creds on a client
  block: pass desired TLS hostname through from block driver client
  block/nbd: support override of hostname for TLS certificate validation
  qemu-nbd: add --tls-hostname option for TLS certificate validation
  block/nbd: don't restrict TLS usage to IP sockets
  tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file
  tests/qemu-iotests: expand _filter_nbd rules
  tests/qemu-iotests: introduce filter for qemu-nbd export list
  tests/qemu-iotests: convert NBD TLS test to use standard filters
  tests/qemu-iotests: validate NBD TLS with hostname mismatch
  tests/qemu-iotests: validate NBD TLS with UNIX sockets
  tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

 block/nbd.c  | 25 +---
 blockdev-nbd.c   |  6 --
 crypto/tlssession.c  |  6 ++
 docs/tools/qemu-nbd.rst  | 13 +
 include/block/nbd.h  |  3 +-
 nbd/client-connection.c  | 12 +++-
 qapi/block-core.json |  3 +
 qemu-nbd.c   | 25 ++--
 tests/qemu-iotests/233   | 99 +++-
 tests/qemu-iotests/233.out   | 56 +++---
 tests/qemu-iotests/241   |  6 +-
 tests/qemu-iotests/241.out   |  6 ++
 tests/qemu-iotests/common.filter |  9 +++
 tests/qemu-iotests/common.tls| 31 +-
 tests/qemu-iotests/testrunner.py |  6 ++
 15 files changed, 255 insertions(+), 51 deletions(-)

-- 
2.34.1

[PATCH v2 12/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

[Daniel P . Berrangé]

This validates that connections to an NBD server running on a UNIX
socket can use TLS with pre-shared keys (PSK).

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 tests/qemu-iotests/233| 28 
 tests/qemu-iotests/233.out| 17 +
 tests/qemu-iotests/common.tls | 24 
 3 files changed, 69 insertions(+)

diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index 442fd1378c..55db5b3811 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -61,6 +61,8 @@ tls_x509_create_server "ca1" "server1"
 tls_x509_create_client "ca1" "client1"
 tls_x509_create_client "ca2" "client2"
 tls_x509_create_client "ca1" "client3"
+tls_psk_create_creds "psk1"
+tls_psk_create_creds "psk2"
 
 echo
 echo "== preparing image =="
@@ -191,6 +193,32 @@ $QEMU_IMG info --image-opts --object $obj1 \
 $QEMU_NBD_PROG -L -k $nbd_unix_socket --object $obj1 \
 --tls-creds=tls0 --tls-hostname=127.0.0.1  2>&1 | _filter_qemu_nbd_exports
 
+
+echo
+echo "== check TLS works over UNIX with PSK =="
+nbd_server_stop
+
+nbd_server_start_unix_socket \
+--object 
tls-creds-psk,dir=${tls_dir}/psk1,endpoint=server,id=tls0,verify-peer=on \
+--tls-creds tls0 \
+-f $IMGFMT "$TEST_IMG" 2>> "$TEST_DIR/server.log"
+
+obj1=tls-creds-psk,dir=${tls_dir}/psk1,username=psk1,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj1 \
+driver=nbd,path=$nbd_unix_socket,tls-creds=tls0 \
+2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -k $nbd_unix_socket --object $obj1 \
+--tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
+
+echo
+echo "== check TLS fails over UNIX with mismatch PSK =="
+obj1=tls-creds-psk,dir=${tls_dir}/psk2,username=psk2,endpoint=client,id=tls0
+$QEMU_IMG info --image-opts --object $obj1 \
+driver=nbd,path=$nbd_unix_socket,tls-creds=tls0 \
+2>&1 | _filter_nbd
+$QEMU_NBD_PROG -L -k $nbd_unix_socket --object $obj1 \
+--tls-creds=tls0 2>&1 | _filter_qemu_nbd_exports
+
 echo
 echo "== final server log =="
 cat "$TEST_DIR/server.log" | _filter_authz_check_tls
diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out
index d79a9ed346..3f160c687c 100644
--- a/tests/qemu-iotests/233.out
+++ b/tests/qemu-iotests/233.out
@@ -7,6 +7,8 @@ Generating a signed certificate...
 Generating a signed certificate...
 Generating a signed certificate...
 Generating a signed certificate...
+Generating a random key for user 'psk1'
+Generating a random key for user 'psk2'
 
 == preparing image ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
@@ -81,6 +83,19 @@ exports available: 1
   size:  67108864
   min block: 1
 
+== check TLS works over UNIX with PSK ==
+image: nbd+unix://?socket=SOCK_DIR/qemu-nbd.sock
+file format: nbd
+virtual size: 64 MiB (67108864 bytes)
+disk size: unavailable
+exports available: 1
+  size:  67108864
+  min block: 1
+
+== check TLS fails over UNIX with mismatch PSK ==
+qemu-img: Could not open 
'driver=nbd,path=SOCK_DIR/qemu-nbd.sock,tls-creds=tls0': TLS handshake failed: 
The TLS connection was non-properly terminated.
+qemu-nbd: TLS handshake failed: The TLS connection was non-properly terminated.
+
 == final server log ==
 qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
 qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
@@ -90,4 +105,6 @@ qemu-nbd: option negotiation failed: TLS x509 authz check 
for DISTINGUISHED-NAME
 qemu-nbd: option negotiation failed: TLS x509 authz check for 
DISTINGUISHED-NAME is denied
 qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
 qemu-nbd: option negotiation failed: Failed to read opts magic: Cannot read 
from TLS channel: Software caused connection abort
+qemu-nbd: option negotiation failed: TLS handshake failed: An illegal 
parameter has been received.
+qemu-nbd: option negotiation failed: TLS handshake failed: An illegal 
parameter has been received.
 *** done
diff --git a/tests/qemu-iotests/common.tls b/tests/qemu-iotests/common.tls
index 4a5760949d..b9c5462986 100644
--- a/tests/qemu-iotests/common.tls
+++ b/tests/qemu-iotests/common.tls
@@ -24,6 +24,7 @@ tls_x509_cleanup()
 {
 rm -f "${tls_dir}"/*.pem
 rm -f "${tls_dir}"/*/*.pem
+rm -f "${tls_dir}"/*/*.psk
 rmdir "${tls_dir}"/*
 rmdir "${tls_dir}"
 }
@@ -40,6 +41,18 @@ tls_certtool()
 rm -f "${tls_dir}"/certtool.log
 }
 
+tls_psktool()
+{
+psktool "$@" 1>"${tls_dir}"/psktool.log 2>&1
+if test "$?" = 0; then
+  head -1 "${tls_dir}"/psktool.log
+else
+  cat "${tls_dir}"/psktool.log
+fi
+rm -f "${tls_dir}"/psktool.log
+}
+
+
 tls_x509_init()
 {
 (certtool --help) >/dev/null 2>&1 || \
@@ -176,3 +189,14 @@ EOF
 
 rm -f "${tls_dir}/cert.info"
 }
+
+tls_psk_create_creds()
+{
+name=$1
+
+mkdir -p 

[PATCH 1/4] m68k/nios2-semi: fix gettimeofday() result check

[marcandre . lureau]

From: Marc-André Lureau 

gettimeofday() returns 0 for success.

Signed-off-by: Marc-André Lureau 
---
 target/m68k/m68k-semi.c   | 2 +-
 target/nios2/nios2-semi.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/m68k/m68k-semi.c b/target/m68k/m68k-semi.c
index 44ec7e4612c6..c5c164e096c8 100644
--- a/target/m68k/m68k-semi.c
+++ b/target/m68k/m68k-semi.c
@@ -381,7 +381,7 @@ void do_m68k_semihosting(CPUM68KState *env, int nr)
 qemu_timeval tv;
 struct gdb_timeval *p;
 result = qemu_gettimeofday();
-if (result != 0) {
+if (result == 0) {
 if (!(p = lock_user(VERIFY_WRITE,
 arg0, sizeof(struct gdb_timeval), 0))) {
 /* FIXME - check error code? */
diff --git a/target/nios2/nios2-semi.c b/target/nios2/nios2-semi.c
index fe5598bae4d7..5a7ad0c7108d 100644
--- a/target/nios2/nios2-semi.c
+++ b/target/nios2/nios2-semi.c
@@ -403,7 +403,7 @@ void do_nios2_semihosting(CPUNios2State *env)
 qemu_timeval tv;
 struct gdb_timeval *p;
 result = qemu_gettimeofday();
-if (result != 0) {
+if (result == 0) {
 p = lock_user(VERIFY_WRITE, arg0, sizeof(struct gdb_timeval),
   0);
 if (!p) {
-- 
2.35.1.273.ge6ebfd0e8cbb

[PATCH 4/4] oslib: drop qemu_gettimeofday()

[marcandre . lureau]

From: Marc-André Lureau 

No longer used after the previous patches.

Signed-off-by: Marc-André Lureau 
---
 include/sysemu/os-posix.h |  3 ---
 include/sysemu/os-win32.h |  6 --
 util/oslib-win32.c| 20 
 3 files changed, 29 deletions(-)

diff --git a/include/sysemu/os-posix.h b/include/sysemu/os-posix.h
index 2edf33658a44..c9b1d63fedda 100644
--- a/include/sysemu/os-posix.h
+++ b/include/sysemu/os-posix.h
@@ -52,9 +52,6 @@ int os_mlock(void);
 #define closesocket(s) close(s)
 #define ioctlsocket(s, r, v) ioctl(s, r, v)
 
-typedef struct timeval qemu_timeval;
-#define qemu_gettimeofday(tp) gettimeofday(tp, NULL)
-
 bool is_daemonized(void);
 
 /**
diff --git a/include/sysemu/os-win32.h b/include/sysemu/os-win32.h
index 43f569b5c216..4d4be826f48c 100644
--- a/include/sysemu/os-win32.h
+++ b/include/sysemu/os-win32.h
@@ -71,12 +71,6 @@ int getpagesize(void);
 # define EPROTONOSUPPORT EINVAL
 #endif
 
-typedef struct {
-long tv_sec;
-long tv_usec;
-} qemu_timeval;
-int qemu_gettimeofday(qemu_timeval *tp);
-
 static inline bool is_daemonized(void)
 {
 return false;
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index af559ef3398d..7faf59e9aaea 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -265,26 +265,6 @@ void qemu_set_cloexec(int fd)
 {
 }
 
-/* Offset between 1/1/1601 and 1/1/1970 in 100 nanosec units */
-#define _W32_FT_OFFSET (1164447360ULL)
-
-int qemu_gettimeofday(qemu_timeval *tp)
-{
-  union {
-unsigned long long ns100; /*time since 1 Jan 1601 in 100ns units */
-FILETIME ft;
-  }  _now;
-
-  if(tp) {
-  GetSystemTimeAsFileTime (&_now.ft);
-  tp->tv_usec=(long)((_now.ns100 / 10ULL) % 100ULL );
-  tp->tv_sec= (long)((_now.ns100 - _W32_FT_OFFSET) / 1000ULL);
-  }
-  /* Always return 0 as per Open Group Base Specifications Issue 6.
- Do not set errno on error.  */
-  return 0;
-}
-
 int qemu_get_thread_id(void)
 {
 return GetCurrentThreadId();
-- 
2.35.1.273.ge6ebfd0e8cbb

[PATCH 0/4] RFC: remove qemu_gettimeofday()

[marcandre . lureau]

From: Marc-André Lureau 

Hi,

Here is a few patches to replace qemu_gettimeofday() helper with functions
provided by GLib.

Marc-André Lureau (4):
  m68k/nios2-semi: fix gettimeofday() result check
  qtest: replace gettimeofday with GTimer
  Replace qemu_gettimeofday() with g_get_real_time()
  oslib: drop qemu_gettimeofday()

 include/sysemu/os-posix.h  |  3 ---
 include/sysemu/os-win32.h  |  6 --
 blockdev.c |  8 
 hw/rtc/m41t80.c|  6 +++---
 hw/virtio/virtio-balloon.c |  9 +
 qapi/qmp-event.c   | 12 +---
 qemu-img.c |  8 
 qga/commands-posix.c   | 11 +--
 softmmu/qtest.c| 39 ++
 target/m68k/m68k-semi.c| 22 ++---
 target/nios2/nios2-semi.c  | 24 +++
 util/oslib-win32.c | 20 ---
 12 files changed, 49 insertions(+), 119 deletions(-)

-- 
2.35.1.273.ge6ebfd0e8cbb

Re: [PATCH 2/4] qtest: replace gettimeofday with GTimer

[Laurent Vivier]

Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :

From: Marc-André Lureau 

glib provides a convenience helper to measure elapsed time. It isn't
subject to wall-clock time changes.

Note that this changes the initial OPENED time, which used to print the
current time.



Time is printed using FMT_timeval ("%ld.%06ld"), but g_timer_elapsed() returns 
a gdouble.

Are you sure it works properly?

Laurent


Signed-off-by: Marc-André Lureau 
---
  softmmu/qtest.c | 39 ++-
  1 file changed, 10 insertions(+), 29 deletions(-)

diff --git a/softmmu/qtest.c b/softmmu/qtest.c
index 8b7cb6aa8e46..b2bbd17d 100644
--- a/softmmu/qtest.c
+++ b/softmmu/qtest.c
@@ -58,12 +58,12 @@ static FILE *qtest_log_fp;
  static QTest *qtest;
  static GString *inbuf;
  static int irq_levels[MAX_IRQ];
-static qemu_timeval start_time;
+static GTimer *timer;
  static bool qtest_opened;
  static void (*qtest_server_send)(void*, const char*);
  static void *qtest_server_send_opaque;
  
-#define FMT_timeval "%ld.%06ld"

+#define FMT_timeval "%.06f"
  
  /**

   * DOC: QTest Protocol
@@ -264,28 +264,13 @@ static int hex2nib(char ch)
  }
  }
  
-static void qtest_get_time(qemu_timeval *tv)

-{
-qemu_gettimeofday(tv);
-tv->tv_sec -= start_time.tv_sec;
-tv->tv_usec -= start_time.tv_usec;
-if (tv->tv_usec < 0) {
-tv->tv_usec += 100;
-tv->tv_sec -= 1;
-}
-}
-
  static void qtest_send_prefix(CharBackend *chr)
  {
-qemu_timeval tv;
-
  if (!qtest_log_fp || !qtest_opened) {
  return;
  }
  
-qtest_get_time();

-fprintf(qtest_log_fp, "[S +" FMT_timeval "] ",
-(long) tv.tv_sec, (long) tv.tv_usec);
+fprintf(qtest_log_fp, "[S +" FMT_timeval "] ", g_timer_elapsed(timer, 
NULL));
  }
  
  static void GCC_FMT_ATTR(1, 2) qtest_log_send(const char *fmt, ...)

@@ -386,12 +371,9 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
  command = words[0];
  
  if (qtest_log_fp) {

-qemu_timeval tv;
  int i;
  
-qtest_get_time();

-fprintf(qtest_log_fp, "[R +" FMT_timeval "]",
-(long) tv.tv_sec, (long) tv.tv_usec);
+fprintf(qtest_log_fp, "[R +" FMT_timeval "]", g_timer_elapsed(timer, 
NULL));
  for (i = 0; words[i]; i++) {
  fprintf(qtest_log_fp, " %s", words[i]);
  }
@@ -846,21 +828,20 @@ static void qtest_event(void *opaque, QEMUChrEvent event)
  for (i = 0; i < ARRAY_SIZE(irq_levels); i++) {
  irq_levels[i] = 0;
  }
-qemu_gettimeofday(_time);
+
+g_clear_pointer(, g_timer_destroy);
+timer = g_timer_new();
  qtest_opened = true;
  if (qtest_log_fp) {
-fprintf(qtest_log_fp, "[I " FMT_timeval "] OPENED\n",
-(long) start_time.tv_sec, (long) start_time.tv_usec);
+fprintf(qtest_log_fp, "[I " FMT_timeval "] OPENED\n", 
g_timer_elapsed(timer, NULL));
  }
  break;
  case CHR_EVENT_CLOSED:
  qtest_opened = false;
  if (qtest_log_fp) {
-qemu_timeval tv;
-qtest_get_time();
-fprintf(qtest_log_fp, "[I +" FMT_timeval "] CLOSED\n",
-(long) tv.tv_sec, (long) tv.tv_usec);
+fprintf(qtest_log_fp, "[I +" FMT_timeval "] CLOSED\n", 
g_timer_elapsed(timer, NULL));
  }
+g_clear_pointer(, g_timer_destroy);
  break;
  default:
  break;

Re: [PATCH 05/12] block/nbd: don't restrict TLS usage to IP sockets

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:23PM +, Daniel P. Berrangé wrote:
> The TLS usage for NBD was restricted to IP sockets because validating
> x509 certificates requires knowledge of the hostname that the client
> is connecting to.
> 
> TLS does not have to use x509 certificates though, as PSK (pre-shared
> keys) provide an alternative credential option. These have no
> requirement for a hostname and can thus be trivially used for UNIX
> sockets.
> 
> Furthermore, with the ability to overide the default hostname for
> TLS validation in the previous patch, it is now also valid to want
> to use x509 certificates with FD passing and UNIX sockets.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  block/nbd.c| 8 ++--
>  blockdev-nbd.c | 6 --
>  qemu-nbd.c | 8 +++-
>  3 files changed, 5 insertions(+), 17 deletions(-)

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[PATCH 2/4] qtest: replace gettimeofday with GTimer

[marcandre . lureau]

From: Marc-André Lureau 

glib provides a convenience helper to measure elapsed time. It isn't
subject to wall-clock time changes.

Note that this changes the initial OPENED time, which used to print the
current time.

Signed-off-by: Marc-André Lureau 
---
 softmmu/qtest.c | 39 ++-
 1 file changed, 10 insertions(+), 29 deletions(-)

diff --git a/softmmu/qtest.c b/softmmu/qtest.c
index 8b7cb6aa8e46..b2bbd17d 100644
--- a/softmmu/qtest.c
+++ b/softmmu/qtest.c
@@ -58,12 +58,12 @@ static FILE *qtest_log_fp;
 static QTest *qtest;
 static GString *inbuf;
 static int irq_levels[MAX_IRQ];
-static qemu_timeval start_time;
+static GTimer *timer;
 static bool qtest_opened;
 static void (*qtest_server_send)(void*, const char*);
 static void *qtest_server_send_opaque;
 
-#define FMT_timeval "%ld.%06ld"
+#define FMT_timeval "%.06f"
 
 /**
  * DOC: QTest Protocol
@@ -264,28 +264,13 @@ static int hex2nib(char ch)
 }
 }
 
-static void qtest_get_time(qemu_timeval *tv)
-{
-qemu_gettimeofday(tv);
-tv->tv_sec -= start_time.tv_sec;
-tv->tv_usec -= start_time.tv_usec;
-if (tv->tv_usec < 0) {
-tv->tv_usec += 100;
-tv->tv_sec -= 1;
-}
-}
-
 static void qtest_send_prefix(CharBackend *chr)
 {
-qemu_timeval tv;
-
 if (!qtest_log_fp || !qtest_opened) {
 return;
 }
 
-qtest_get_time();
-fprintf(qtest_log_fp, "[S +" FMT_timeval "] ",
-(long) tv.tv_sec, (long) tv.tv_usec);
+fprintf(qtest_log_fp, "[S +" FMT_timeval "] ", g_timer_elapsed(timer, 
NULL));
 }
 
 static void GCC_FMT_ATTR(1, 2) qtest_log_send(const char *fmt, ...)
@@ -386,12 +371,9 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 command = words[0];
 
 if (qtest_log_fp) {
-qemu_timeval tv;
 int i;
 
-qtest_get_time();
-fprintf(qtest_log_fp, "[R +" FMT_timeval "]",
-(long) tv.tv_sec, (long) tv.tv_usec);
+fprintf(qtest_log_fp, "[R +" FMT_timeval "]", g_timer_elapsed(timer, 
NULL));
 for (i = 0; words[i]; i++) {
 fprintf(qtest_log_fp, " %s", words[i]);
 }
@@ -846,21 +828,20 @@ static void qtest_event(void *opaque, QEMUChrEvent event)
 for (i = 0; i < ARRAY_SIZE(irq_levels); i++) {
 irq_levels[i] = 0;
 }
-qemu_gettimeofday(_time);
+
+g_clear_pointer(, g_timer_destroy);
+timer = g_timer_new();
 qtest_opened = true;
 if (qtest_log_fp) {
-fprintf(qtest_log_fp, "[I " FMT_timeval "] OPENED\n",
-(long) start_time.tv_sec, (long) start_time.tv_usec);
+fprintf(qtest_log_fp, "[I " FMT_timeval "] OPENED\n", 
g_timer_elapsed(timer, NULL));
 }
 break;
 case CHR_EVENT_CLOSED:
 qtest_opened = false;
 if (qtest_log_fp) {
-qemu_timeval tv;
-qtest_get_time();
-fprintf(qtest_log_fp, "[I +" FMT_timeval "] CLOSED\n",
-(long) tv.tv_sec, (long) tv.tv_usec);
+fprintf(qtest_log_fp, "[I +" FMT_timeval "] CLOSED\n", 
g_timer_elapsed(timer, NULL));
 }
+g_clear_pointer(, g_timer_destroy);
 break;
 default:
 break;
-- 
2.35.1.273.ge6ebfd0e8cbb

Re: [PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

[Laurent Vivier]

Le 04/03/2022 à 17:08, Laurent Vivier a écrit :

Le 04/03/2022 à 16:27, marcandre.lur...@redhat.com a écrit :

From: Marc-André Lureau 

GLib g_get_real_time() is an alternative to gettimeofday().

For semihosting, a few bits are lost on POSIX host, but this shouldn't
be a big concern.

Signed-off-by: Marc-André Lureau 
---
  blockdev.c |  8 
  hw/rtc/m41t80.c    |  6 +++---
  hw/virtio/virtio-balloon.c |  9 +
  qapi/qmp-event.c   | 12 +---
  qemu-img.c |  8 
  qga/commands-posix.c   | 11 +--
  target/m68k/m68k-semi.c    | 22 ++
  target/nios2/nios2-semi.c  | 24 +++-
  8 files changed, 39 insertions(+), 61 deletions(-)


...

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 75dbaab68ea9..3311a4ca0167 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -138,16 +138,7 @@ void qmp_guest_shutdown(bool has_mode, const char *mode, 
Error **errp)
  int64_t qmp_guest_get_time(Error **errp)
  {
-   int ret;
-   qemu_timeval tq;
-
-   ret = qemu_gettimeofday();
-   if (ret < 0) {
-   error_setg_errno(errp, errno, "Failed to get time");
-   return -1;
-   }
-
-   return tq.tv_sec * 10LL + tq.tv_usec * 1000;
+    return g_get_real_time();


now you return microseconds, before it was nanoseconds.

qga/qapi-schema.json:

##
# @guest-get-time:
#
# Get the information about guest's System Time relative to
# the Epoch of 1970-01-01 in UTC.
#
# Returns: Time in nanoseconds.
#
# Since: 1.5
##
{ 'command': 'guest-get-time',
   'returns': 'int' }

Except this problem:

Reviewed-by: Laurent Vivier 


Perhaps you can also remove the windows version from qga/commands-win32.c and move the function to 
qga/commands.c ?


Thanks,
Laurent

Re: [PATCH 1/5] python/utils: add add_visual_margin() text decoration utility

[John Snow]

On Thu, Mar 3, 2022, 5:43 PM Eric Blake  wrote:

> On Thu, Mar 03, 2022 at 03:58:58PM -0500, John Snow wrote:
> > >>> print(add_visual_margin(msg, width=72, name="Commit Message"))
> > ┏━ Commit Message ━━
> > ┃ add_visual_margin() takes a chunk of text and wraps it in a visual
> > ┃ container that force-wraps to a specified width. An optional title
> > ┃ label may be given, and any of the individual glyphs used to draw the
> > ┃ box may be replaced or specified as well.
> > ┗━━━
>
> I see you dropped the right margin compared to earlier versions, but
> agree that this is still a nice visual indicator, and probably easier
> to maintain in this form.  And it got rid of the weird spacing on the
> left when the wrap point hit at the wrong time.
>
> > +Decorate and wrap some text with a visual decoration around it.
> > +
> > +This function assumes that the text decoration characters are single
> > +characters that display using a single monospace column.
> > +
> > +┏━ Example
> ━
> > +┃ This is what this function looks like with text content that's
> > +┃ wrapped to 72 characters. The right-hand margin is left open to
> > +┃ acommodate the occasional unicode character that might make
> > +┃ predicting the total "visual" width of a line difficult. This
> > +┃ provides a visual distinction that's good-enough, though.
> > +
> ┗━━━
>
> Yep - hand-waving away Unicode messiness is certainly easiest ;)
>

I started reading the unicode specification for determining the number of
"visual grapheme clusters" in a string, got a headache, and gave up.

(See also the PyPI package "grapheme", which was last updated for unicode
13.0 and appears unmaintained. It's no small amount of code!)

I could write a function that gives a *pretty good estimate*, but I don't
want to maintain something like that.

Urwid has functions that give *great* estimates, but I'd like to avoid
dependencies.

Leaving the right margin open was the quickest, cleanest hack :) I'm amazed
at how hard this is to accomplish.

(See https://hsivonen.fi/string-length/ which ought to be required reading
in all CS classes from now on.)

Ultimately I took both this decoration and the exception code into utils as
an admission of guilt and a pledge to maintain what I did write ;)


> Reviewed-by: Eric Blake 
>
> [take with a grain of salt - my python is weak. But as you said in the
> cover letter, it's fairly straightforward to reproduce an environment
> where you can see it in action for hands-on testing]
>

I've long since given up on anyone understanding the Python I write.
Eduardo did, so he quit. 

Thanks for the reviews.


>
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.   +1-919-301-3266
> Virtualization:  qemu.org | libvirt.org
>
>

[PULL 13/50] iotests/185: Add post-READY quit tests

[Kevin Wolf]

From: Hanna Reitz 

185 tests quitting qemu while a block job is active.  It does not
specifically test quitting qemu while a mirror or active commit job is
in its READY phase.

Add two test cases for this, where we respectively mirror or commit to
an external QSD instance, which provides a throttled block device.  qemu
is supposed to cancel the job so that it can quit as soon as possible
instead of waiting for the job to complete (which it did before 6.2).

Signed-off-by: Hanna Reitz 
Message-Id: <20220303164814.284974-5-hre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/185 | 190 -
 tests/qemu-iotests/185.out |  48 ++
 2 files changed, 237 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185
index f2ec5c5ceb..8b1143dc16 100755
--- a/tests/qemu-iotests/185
+++ b/tests/qemu-iotests/185
@@ -33,6 +33,12 @@ _cleanup()
 _rm_test_img "${TEST_IMG}.copy"
 _cleanup_test_img
 _cleanup_qemu
+
+if [ -f "$TEST_DIR/qsd.pid" ]; then
+kill -SIGKILL "$(cat "$TEST_DIR/qsd.pid")"
+rm -f "$TEST_DIR/qsd.pid"
+fi
+rm -f "$SOCK_DIR/qsd.sock"
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15
 
@@ -45,7 +51,7 @@ _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
 
-size=64M
+size=$((64 * 1048576))
 TEST_IMG="${TEST_IMG}.base" _make_test_img $size
 
 echo
@@ -216,6 +222,188 @@ wait=1 _cleanup_qemu | grep -v 'JOB_STATUS_CHANGE'
 
 _check_test_img
 
+echo
+echo === Start mirror to throttled QSD and exit qemu ===
+echo
+
+# Mirror to a throttled QSD instance (so that qemu cannot drain the
+# throttling), wait for READY, then write some data to the device,
+# and then quit qemu.
+# (qemu should force-cancel the job and not wait for the data to be
+# written to the target.)
+
+_make_test_img $size
+
+# Will be used by this and the next case
+set_up_throttled_qsd() {
+$QSD \
+--object throttle-group,id=thrgr,limits.bps-total=1048576 \
+--blockdev null-co,node-name=null,size=$size \
+--blockdev throttle,node-name=throttled,throttle-group=thrgr,file=null 
\
+--nbd-server addr.type=unix,addr.path="$SOCK_DIR/qsd.sock" \
+--export nbd,id=exp,node-name=throttled,name=target,writable=true \
+--pidfile "$TEST_DIR/qsd.pid" \
+--daemonize
+}
+
+set_up_throttled_qsd
+
+# Need a virtio-blk device so that qemu-io writes will not block the monitor
+_launch_qemu \
+--blockdev file,node-name=source-proto,filename="$TEST_IMG" \
+--blockdev qcow2,node-name=source-fmt,file=source-proto \
+--device virtio-blk,id=vblk,drive=source-fmt \
+--blockdev "{\"driver\": \"nbd\",
+ \"node-name\": \"target\",
+ \"server\": {
+ \"type\": \"unix\",
+ \"path\": \"$SOCK_DIR/qsd.sock\"
+ },
+ \"export\": \"target\"}"
+
+h=$QEMU_HANDLE
+_send_qemu_cmd $h '{"execute": "qmp_capabilities"}' 'return'
+
+# Use sync=top, so the first pass will not copy the whole image
+_send_qemu_cmd $h \
+'{"execute": "blockdev-mirror",
+  "arguments": {
+  "job-id": "mirror",
+  "device": "source-fmt",
+  "target": "target",
+  "sync": "top"
+  }}' \
+'return' \
+| grep -v JOB_STATUS_CHANGE # Ignore these events during creation
+
+# This too will be used by this and the next case
+# $1: QEMU handle
+# $2: Image size
+wait_for_job_and_quit() {
+h=$1
+size=$2
+
+# List of expected events
+capture_events='BLOCK_JOB_READY JOB_STATUS_CHANGE'
+_wait_event $h 'BLOCK_JOB_READY'
+QEMU_EVENTS= # Ignore all JOB_STATUS_CHANGE events that came before READY
+
+# Write something to the device for post-READY mirroring.  Write it in
+# blocks matching the cluster size, each spaced one block apart, so
+# that the mirror job will have to spawn one request per cluster.
+# Because the number of concurrent requests is limited (to 16), this
+# limits the number of bytes concurrently in flight, which speeds up
+# cancelling the job (in-flight requests still are waited for).
+# To limit the number of bytes in flight, we could alternatively pass
+# something for blockdev-mirror's @buf-size parameter, but
+# block-commit does not have such a parameter, so we need to figure
+# something out that works for both.
+
+cluster_size=65536
+step=$((cluster_size * 2))
+
+echo '--- Writing data to the virtio-blk device ---'
+
+for ofs in $(seq 0 $step $((size - step))); do
+qemu_io_cmd="qemu-io -d vblk/virtio-backend "
+qemu_io_cmd+="\\\"aio_write $ofs $cluster_size\\\""
+
+# Do not include these requests in the reference output
+# (it's just too much)
+silent=yes _send_qemu_cmd $h \
+"{\"execute\": \"human-monitor-command\",
+  \"arguments\": {
+  \"command-line\": \"$qemu_io_cmd\"
+ 

[PULL 06/50] tls: add macros for coroutine-safe TLS variables

[Kevin Wolf]

From: Stefan Hajnoczi 

Compiler optimizations can cache TLS values across coroutine yield
points, resulting in stale values from the previous thread when a
coroutine is re-entered by a new thread.

Serge Guelton developed an __attribute__((noinline)) wrapper and tested
it with clang and gcc. I formatted his idea according to QEMU's coding
style and wrote documentation.

The compiler can still optimize based on analyzing noinline code, so an
asm volatile barrier with an output constraint is required to prevent
unwanted optimizations.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1952483
Suggested-by: Serge Guelton 
Signed-off-by: Stefan Hajnoczi 
Message-Id: <20220222140150.27240-2-stefa...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/qemu/coroutine-tls.h | 165 +++
 1 file changed, 165 insertions(+)
 create mode 100644 include/qemu/coroutine-tls.h

diff --git a/include/qemu/coroutine-tls.h b/include/qemu/coroutine-tls.h
new file mode 100644
index 00..1558a826aa
--- /dev/null
+++ b/include/qemu/coroutine-tls.h
@@ -0,0 +1,165 @@
+/*
+ * QEMU Thread Local Storage for coroutines
+ *
+ * Copyright Red Hat
+ *
+ * SPDX-License-Identifier: LGPL-2.1-or-later
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ * It is forbidden to access Thread Local Storage in coroutines because
+ * compiler optimizations may cause values to be cached across coroutine
+ * re-entry. Coroutines can run in more than one thread through the course of
+ * their life, leading bugs when stale TLS values from the wrong thread are
+ * used as a result of compiler optimization.
+ *
+ * An example is:
+ *
+ * ..code-block:: c
+ *   :caption: A coroutine that may see the wrong TLS value
+ *
+ *   static __thread AioContext *current_aio_context;
+ *   ...
+ *   static void coroutine_fn foo(void)
+ *   {
+ *   aio_notify(current_aio_context);
+ *   qemu_coroutine_yield();
+ *   aio_notify(current_aio_context); // <-- may be stale after yielding!
+ *   }
+ *
+ * This header provides macros for safely defining variables in Thread Local
+ * Storage:
+ *
+ * ..code-block:: c
+ *   :caption: A coroutine that safely uses TLS
+ *
+ *   QEMU_DEFINE_STATIC_CO_TLS(AioContext *, current_aio_context)
+ *   ...
+ *   static void coroutine_fn foo(void)
+ *   {
+ *   aio_notify(get_current_aio_context());
+ *   qemu_coroutine_yield();
+ *   aio_notify(get_current_aio_context()); // <-- safe
+ *   }
+ */
+
+#ifndef QEMU_COROUTINE_TLS_H
+#define QEMU_COROUTINE_TLS_H
+
+/*
+ * To stop the compiler from caching TLS values we define accessor functions
+ * with __attribute__((noinline)) plus asm volatile("") to prevent
+ * optimizations that override noinline.
+ *
+ * The compiler can still analyze noinline code and make optimizations based on
+ * that knowledge, so an inline asm output operand is used to prevent
+ * optimizations that make assumptions about the address of the TLS variable.
+ *
+ * This is fragile and ultimately needs to be solved by a mechanism that is
+ * guaranteed to work by the compiler (e.g. stackless coroutines), but for now
+ * we use this approach to prevent issues.
+ */
+
+/**
+ * QEMU_DECLARE_CO_TLS:
+ * @type: the variable's C type
+ * @var: the variable name
+ *
+ * Declare an extern variable in Thread Local Storage from a header file:
+ *
+ * .. code-block:: c
+ *   :caption: Declaring an extern variable in Thread Local Storage
+ *
+ *   QEMU_DECLARE_CO_TLS(int, my_count)
+ *   ...
+ *   int c = get_my_count();
+ *   set_my_count(c + 1);
+ *   *get_ptr_my_count() = 0;
+ *
+ * This is a coroutine-safe replacement for the __thread keyword and is
+ * equivalent to the following code:
+ *
+ * .. code-block:: c
+ *   :caption: Declaring a TLS variable using __thread
+ *
+ *   extern __thread int my_count;
+ *   ...
+ *   int c = my_count;
+ *   my_count = c + 1;
+ *   *(_count) = 0;
+ */
+#define QEMU_DECLARE_CO_TLS(type, var)   \
+__attribute__((noinline)) type get_##var(void);  \
+__attribute__((noinline)) void set_##var(type v);\
+__attribute__((noinline)) type *get_ptr_##var(void);
+
+/**
+ * QEMU_DEFINE_CO_TLS:
+ * @type: the variable's C type
+ * @var: the variable name
+ *
+ * Define a variable in Thread Local Storage that was previously declared from
+ * a header file with QEMU_DECLARE_CO_TLS():
+ *
+ * .. code-block:: c
+ *   :caption: Defining a variable in Thread Local Storage
+ *
+ *   QEMU_DEFINE_CO_TLS(int, my_count)
+ *
+ * This is a coroutine-safe replacement for the __thread keyword and is
+ * equivalent to the following code:
+ *
+ * .. code-block:: c
+ *   :caption: Defining a TLS variable using __thread
+ *
+ *   __thread int my_count;
+ */
+#define QEMU_DEFINE_CO_TLS(type, var)\
+static __thread type 

[PULL 23/50] block.c: assertions to the block layer permissions API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Now that we "covered" the three main cases where the
permission API was being used under BQL (fuse,
amend and invalidate_cache), we can safely assert for
the permission functions implemented in block.c

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-11-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 12 
 1 file changed, 12 insertions(+)

diff --git a/block.c b/block.c
index fd3f565a16..5351b4f131 100644
--- a/block.c
+++ b/block.c
@@ -2109,6 +2109,7 @@ static bool bdrv_a_allow_b(BdrvChild *a, BdrvChild *b, 
Error **errp)
 
 assert(a->bs);
 assert(a->bs == b->bs);
+GLOBAL_STATE_CODE();
 
 if ((b->perm & a->shared_perm) == b->perm) {
 return true;
@@ -2132,6 +2133,7 @@ static bool bdrv_a_allow_b(BdrvChild *a, BdrvChild *b, 
Error **errp)
 static bool bdrv_parent_perms_conflict(BlockDriverState *bs, Error **errp)
 {
 BdrvChild *a, *b;
+GLOBAL_STATE_CODE();
 
 /*
  * During the loop we'll look at each pair twice. That's correct because
@@ -2213,6 +2215,8 @@ static void bdrv_child_set_perm_abort(void *opaque)
 {
 BdrvChildSetPermState *s = opaque;
 
+GLOBAL_STATE_CODE();
+
 s->child->perm = s->old_perm;
 s->child->shared_perm = s->old_shared_perm;
 }
@@ -2226,6 +2230,7 @@ static void bdrv_child_set_perm(BdrvChild *c, uint64_t 
perm,
 uint64_t shared, Transaction *tran)
 {
 BdrvChildSetPermState *s = g_new(BdrvChildSetPermState, 1);
+GLOBAL_STATE_CODE();
 
 *s = (BdrvChildSetPermState) {
 .child = c,
@@ -2405,6 +2410,7 @@ static int bdrv_node_refresh_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 BdrvChild *c;
 int ret;
 uint64_t cumulative_perms, cumulative_shared_perms;
+GLOBAL_STATE_CODE();
 
 bdrv_get_cumulative_perm(bs, _perms, _shared_perms);
 
@@ -2473,6 +2479,7 @@ static int bdrv_list_refresh_perms(GSList *list, 
BlockReopenQueue *q,
 {
 int ret;
 BlockDriverState *bs;
+GLOBAL_STATE_CODE();
 
 for ( ; list; list = list->next) {
 bs = list->data;
@@ -2540,6 +2547,7 @@ static int bdrv_refresh_perms(BlockDriverState *bs, Error 
**errp)
 int ret;
 Transaction *tran = tran_new();
 g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
+GLOBAL_STATE_CODE();
 
 ret = bdrv_list_refresh_perms(list, NULL, tran, errp);
 tran_finalize(tran, ret);
@@ -2602,6 +2610,7 @@ static void bdrv_filter_default_perms(BlockDriverState 
*bs, BdrvChild *c,
   uint64_t perm, uint64_t shared,
   uint64_t *nperm, uint64_t *nshared)
 {
+GLOBAL_STATE_CODE();
 *nperm = perm & DEFAULT_PERM_PASSTHROUGH;
 *nshared = (shared & DEFAULT_PERM_PASSTHROUGH) | DEFAULT_PERM_UNCHANGED;
 }
@@ -2613,6 +2622,7 @@ static void bdrv_default_perms_for_cow(BlockDriverState 
*bs, BdrvChild *c,
uint64_t *nperm, uint64_t *nshared)
 {
 assert(role & BDRV_CHILD_COW);
+GLOBAL_STATE_CODE();
 
 /*
  * We want consistent read from backing files if the parent needs it.
@@ -2649,6 +2659,7 @@ static void 
bdrv_default_perms_for_storage(BlockDriverState *bs, BdrvChild *c,
 {
 int flags;
 
+GLOBAL_STATE_CODE();
 assert(role & (BDRV_CHILD_METADATA | BDRV_CHILD_DATA));
 
 flags = bdrv_reopen_get_flags(reopen_queue, bs);
@@ -6028,6 +6039,7 @@ static void xdbg_graph_add_edge(XDbgBlockGraphConstructor 
*gr, void *parent,
 {
 BlockPermission qapi_perm;
 XDbgBlockGraphEdge *edge;
+GLOBAL_STATE_CODE();
 
 edge = g_new0(XDbgBlockGraphEdge, 1);
 
-- 
2.35.1

[PULL 35/50] include/block/snapshot: global state API + assertions

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Snapshots run also under the BQL, so they all are
in the global state API. The aiocontext lock that they hold
is currently an overkill and in future could be removed.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-23-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/snapshot.h | 13 +++--
 block/snapshot.c | 28 
 migration/savevm.c   |  2 ++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/include/block/snapshot.h b/include/block/snapshot.h
index 940345692f..50ff924710 100644
--- a/include/block/snapshot.h
+++ b/include/block/snapshot.h
@@ -45,6 +45,13 @@ typedef struct QEMUSnapshotInfo {
 uint64_t icount; /* record/replay step */
 } QEMUSnapshotInfo;
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 int bdrv_snapshot_find(BlockDriverState *bs, QEMUSnapshotInfo *sn_info,
const char *name);
 bool bdrv_snapshot_find_by_id_and_name(BlockDriverState *bs,
@@ -73,9 +80,11 @@ int bdrv_snapshot_load_tmp_by_id_or_name(BlockDriverState 
*bs,
  Error **errp);
 
 
-/* Group operations. All block drivers are involved.
+/*
+ * Group operations. All block drivers are involved.
  * These functions will properly handle dataplane (take aio_context_acquire
- * when appropriate for appropriate block drivers */
+ * when appropriate for appropriate block drivers
+ */
 
 bool bdrv_all_can_snapshot(bool has_devices, strList *devices,
Error **errp);
diff --git a/block/snapshot.c b/block/snapshot.c
index ccacda8bd5..d6f53c3065 100644
--- a/block/snapshot.c
+++ b/block/snapshot.c
@@ -57,6 +57,8 @@ int bdrv_snapshot_find(BlockDriverState *bs, QEMUSnapshotInfo 
*sn_info,
 QEMUSnapshotInfo *sn_tab, *sn;
 int nb_sns, i, ret;
 
+GLOBAL_STATE_CODE();
+
 ret = -ENOENT;
 nb_sns = bdrv_snapshot_list(bs, _tab);
 if (nb_sns < 0) {
@@ -105,6 +107,7 @@ bool bdrv_snapshot_find_by_id_and_name(BlockDriverState *bs,
 bool ret = false;
 
 assert(id || name);
+GLOBAL_STATE_CODE();
 
 nb_sns = bdrv_snapshot_list(bs, _tab);
 if (nb_sns < 0) {
@@ -200,6 +203,7 @@ static BlockDriverState 
*bdrv_snapshot_fallback(BlockDriverState *bs)
 int bdrv_can_snapshot(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
+GLOBAL_STATE_CODE();
 if (!drv || !bdrv_is_inserted(bs) || bdrv_is_read_only(bs)) {
 return 0;
 }
@@ -220,6 +224,9 @@ int bdrv_snapshot_create(BlockDriverState *bs,
 {
 BlockDriver *drv = bs->drv;
 BlockDriverState *fallback_bs = bdrv_snapshot_fallback(bs);
+
+GLOBAL_STATE_CODE();
+
 if (!drv) {
 return -ENOMEDIUM;
 }
@@ -240,6 +247,8 @@ int bdrv_snapshot_goto(BlockDriverState *bs,
 BdrvChild **fallback_ptr;
 int ret, open_ret;
 
+GLOBAL_STATE_CODE();
+
 if (!drv) {
 error_setg(errp, "Block driver is closed");
 return -ENOMEDIUM;
@@ -348,6 +357,8 @@ int bdrv_snapshot_delete(BlockDriverState *bs,
 BlockDriverState *fallback_bs = bdrv_snapshot_fallback(bs);
 int ret;
 
+GLOBAL_STATE_CODE();
+
 if (!drv) {
 error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, bdrv_get_device_name(bs));
 return -ENOMEDIUM;
@@ -380,6 +391,8 @@ int bdrv_snapshot_list(BlockDriverState *bs,
 {
 BlockDriver *drv = bs->drv;
 BlockDriverState *fallback_bs = bdrv_snapshot_fallback(bs);
+
+GLOBAL_STATE_CODE();
 if (!drv) {
 return -ENOMEDIUM;
 }
@@ -419,6 +432,8 @@ int bdrv_snapshot_load_tmp(BlockDriverState *bs,
 {
 BlockDriver *drv = bs->drv;
 
+GLOBAL_STATE_CODE();
+
 if (!drv) {
 error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, bdrv_get_device_name(bs));
 return -ENOMEDIUM;
@@ -447,6 +462,8 @@ int bdrv_snapshot_load_tmp_by_id_or_name(BlockDriverState 
*bs,
 int ret;
 Error *local_err = NULL;
 
+GLOBAL_STATE_CODE();
+
 ret = bdrv_snapshot_load_tmp(bs, id_or_name, NULL, _err);
 if (ret == -ENOENT || ret == -EINVAL) {
 error_free(local_err);
@@ -515,6 +532,8 @@ bool bdrv_all_can_snapshot(bool has_devices, strList 
*devices,
 g_autoptr(GList) bdrvs = NULL;
 GList *iterbdrvs;
 
+GLOBAL_STATE_CODE();
+
 if (bdrv_all_get_snapshot_devices(has_devices, devices, , errp) < 0) 
{
 return false;
 }
@@ -549,6 +568,8 @@ int bdrv_all_delete_snapshot(const char *name,
 g_autoptr(GList) bdrvs = NULL;
 GList *iterbdrvs;
 
+GLOBAL_STATE_CODE();
+
 if (bdrv_all_get_snapshot_devices(has_devices, devices, , errp) < 0) 
{
 return -1;
 }
@@ -588,6 +609,8 @@ int bdrv_all_goto_snapshot(const char *name,
 g_autoptr(GList) bdrvs = NULL;
 GList *iterbdrvs;
 
+GLOBAL_STATE_CODE();
+
 if (bdrv_all_get_snapshot_devices(has_devices, devices, , errp) < 0) 
{
  

[PULL 48/50] tests/qemu-iotests: Rework the checks and spots using GNU sed

[Kevin Wolf]

From: Thomas Huth 

Instead of failing the iotests if GNU sed is not available (or skipping
them completely in the check-block.sh script), it would be better to
simply skip the bash-based tests that rely on GNU sed, so that the other
tests could still be run. Thus we now explicitely use "gsed" (either as
direct program or as a wrapper around "sed" if it's the GNU version)
in the spots that rely on the GNU sed behavior. Statements that use the
"-r" parameter of sed have been switched to use "-E" instead, since this
switch is supported by all sed versions on our supported build hosts
(most also support "-r", but macOS' sed only supports "-E"). With all
these changes in place, we then can also remove the sed checks from the
check-block.sh script, so that "make check-block" can now be run on
systems without GNU sed, too.

Signed-off-by: Thomas Huth 
Message-Id: <20220216125454.465041-1-th...@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 tests/check-block.sh | 12 --
 tests/qemu-iotests/271   |  2 +-
 tests/qemu-iotests/common.filter | 65 
 tests/qemu-iotests/common.rc | 45 +++---
 4 files changed, 57 insertions(+), 67 deletions(-)

diff --git a/tests/check-block.sh b/tests/check-block.sh
index 18f7433901..f59496396c 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -48,18 +48,6 @@ if LANG=C bash --version | grep -q 'GNU bash, version [123]' 
; then
 skip "bash version too old ==> Not running the qemu-iotests."
 fi
 
-if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
-if ! command -v gsed >/dev/null 2>&1; then
-skip "GNU sed not available ==> Not running the qemu-iotests."
-fi
-else
-# Double-check that we're not using BusyBox' sed which says
-# that "This is not GNU sed version 4.0" ...
-if sed --version | grep -q 'not GNU sed' ; then
-skip "BusyBox sed not supported ==> Not running the qemu-iotests."
-fi
-fi
-
 cd tests/qemu-iotests
 
 # QEMU_CHECK_BLOCK_AUTO is used to disable some unstable sub-tests
diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271
index 2775b4d130..c7c2cadda0 100755
--- a/tests/qemu-iotests/271
+++ b/tests/qemu-iotests/271
@@ -896,7 +896,7 @@ _make_test_img -o extended_l2=on 1M
 # Second and third writes in _concurrent_io() are independent and may finish in
 # different order. So, filter offset out to match both possible variants.
 _concurrent_io | $QEMU_IO | _filter_qemu_io | \
-$SED -e 's/\(20480\|40960\)/OFFSET/'
+sed -e 's/\(20480\|40960\)/OFFSET/'
 _concurrent_verify | $QEMU_IO | _filter_qemu_io
 
 # success, all done
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 75cc241580..21819db9c3 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -21,44 +21,44 @@
 
 _filter_date()
 {
-$SED -re 's/[0-9]{4}-[0-9]{2}-[0-9]{2} 
[0-9]{2}:[0-9]{2}:[0-9]{2}/-mm-dd hh:mm:ss/'
+sed -Ee 's/[0-9]{4}-[0-9]{2}-[0-9]{2} 
[0-9]{2}:[0-9]{2}:[0-9]{2}/-mm-dd hh:mm:ss/'
 }
 
 _filter_vmstate_size()
 {
-$SED -r -e 's/[0-9. ]{5} [KMGT]iB/ SIZE/' \
--e 's/[0-9. ]{5} B/   SIZE/'
+sed -E -e 's/[0-9. ]{5} [KMGT]iB/ SIZE/' \
+   -e 's/[0-9. ]{5} B/   SIZE/'
 }
 
 _filter_generated_node_ids()
 {
-$SED -re 's/\#block[0-9]{3,}/NODE_NAME/'
+sed -Ee 's/\#block[0-9]{3,}/NODE_NAME/'
 }
 
 _filter_qom_path()
 {
-$SED -e '/Attached to:/s/\device[[0-9]\+\]/device[N]/g'
+gsed -e '/Attached to:/s/\device[[0-9]\+\]/device[N]/g'
 }
 
 # replace occurrences of the actual TEST_DIR value with TEST_DIR
 _filter_testdir()
 {
-$SED -e "s#$TEST_DIR/#TEST_DIR/#g" \
- -e "s#$SOCK_DIR/#SOCK_DIR/#g" \
- -e "s#SOCK_DIR/fuse-#TEST_DIR/#g"
+sed -e "s#$TEST_DIR/#TEST_DIR/#g" \
+-e "s#$SOCK_DIR/#SOCK_DIR/#g" \
+-e "s#SOCK_DIR/fuse-#TEST_DIR/#g"
 }
 
 # replace occurrences of the actual IMGFMT value with IMGFMT
 _filter_imgfmt()
 {
-$SED -e "s#$IMGFMT#IMGFMT#g"
+sed -e "s#$IMGFMT#IMGFMT#g"
 }
 
 # Replace error message when the format is not supported and delete
 # the output lines after the first one
 _filter_qemu_img_check()
 {
-$SED -e '/allocated.*fragmented.*compressed clusters/d' \
+gsed -e '/allocated.*fragmented.*compressed clusters/d' \
 -e 's/qemu-img: This image format does not support checks/No errors 
were found on the image./' \
 -e '/Image end offset: [0-9]\+/d'
 }
@@ -66,13 +66,14 @@ _filter_qemu_img_check()
 # Removes \r from messages
 _filter_win32()
 {
-$SED -e 's/\r//g'
+gsed -e 's/\r//g'
 }
 
 # sanitize qemu-io output
 _filter_qemu_io()
 {
-_filter_win32 | $SED -e "s/[0-9]* ops\; [0-9/:. sec]* ([0-9/.inf]* 
[EPTGMKiBbytes]*\/sec and [0-9/.inf]* ops\/sec)/X ops\; XX:XX:XX.X (XXX 
YYY\/sec and XXX ops\/sec)/" \
+_filter_win32 | \
+gsed -e "s/[0-9]* ops\; [0-9/:. sec]* 

[PULL 30/50] block.c: add assertions to static functions

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Following the assertion derived from the API split,
propagate the assertion also in the static functions.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-18-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c   | 46 ++-
 block/block-backend.c |  3 +++
 2 files changed, 48 insertions(+), 1 deletion(-)

diff --git a/block.c b/block.c
index fe05df471a..d3b97c3569 100644
--- a/block.c
+++ b/block.c
@@ -438,6 +438,7 @@ BlockDriverState *bdrv_new(void)
 static BlockDriver *bdrv_do_find_format(const char *format_name)
 {
 BlockDriver *drv1;
+GLOBAL_STATE_CODE();
 
 QLIST_FOREACH(drv1, _drivers, list) {
 if (!strcmp(drv1->format_name, format_name)) {
@@ -596,6 +597,8 @@ static int64_t create_file_fallback_truncate(BlockBackend 
*blk,
 int64_t size;
 int ret;
 
+GLOBAL_STATE_CODE();
+
 ret = blk_truncate(blk, minimum_size, false, PREALLOC_MODE_OFF, 0,
_err);
 if (ret < 0 && ret != -ENOTSUP) {
@@ -634,6 +637,8 @@ static int 
create_file_fallback_zero_first_sector(BlockBackend *blk,
 int64_t bytes_to_clear;
 int ret;
 
+GLOBAL_STATE_CODE();
+
 bytes_to_clear = MIN(current_size, BDRV_SECTOR_SIZE);
 if (bytes_to_clear) {
 ret = blk_pwrite_zeroes(blk, 0, bytes_to_clear, BDRV_REQ_MAY_UNMAP);
@@ -896,6 +901,7 @@ static BlockDriver *find_hdev_driver(const char *filename)
 {
 int score_max = 0, score;
 BlockDriver *drv = NULL, *d;
+GLOBAL_STATE_CODE();
 
 QLIST_FOREACH(d, _drivers, list) {
 if (d->bdrv_probe_device) {
@@ -913,6 +919,7 @@ static BlockDriver *find_hdev_driver(const char *filename)
 static BlockDriver *bdrv_do_find_protocol(const char *protocol)
 {
 BlockDriver *drv1;
+GLOBAL_STATE_CODE();
 
 QLIST_FOREACH(drv1, _drivers, list) {
 if (drv1->protocol_name && !strcmp(drv1->protocol_name, protocol)) {
@@ -1021,6 +1028,8 @@ static int find_image_format(BlockBackend *file, const 
char *filename,
 uint8_t buf[BLOCK_PROBE_BUF_SIZE];
 int ret = 0;
 
+GLOBAL_STATE_CODE();
+
 /* Return the raw BlockDriver * to scsi-generic devices or empty drives */
 if (blk_is_sg(file) || !blk_is_inserted(file) || blk_getlength(file) == 0) 
{
 *pdrv = _raw;
@@ -1103,6 +1112,7 @@ static BlockdevDetectZeroesOptions 
bdrv_parse_detect_zeroes(QemuOpts *opts,
 BlockdevDetectZeroesOptions detect_zeroes =
 qapi_enum_parse(_lookup, value,
 BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF, _err);
+GLOBAL_STATE_CODE();
 g_free(value);
 if (local_err) {
 error_propagate(errp, local_err);
@@ -1218,6 +1228,7 @@ static void bdrv_child_cb_drained_end(BdrvChild *child,
 static int bdrv_child_cb_inactivate(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
+GLOBAL_STATE_CODE();
 assert(bs->open_flags & BDRV_O_INACTIVE);
 return 0;
 }
@@ -1244,6 +1255,7 @@ static void bdrv_child_cb_set_aio_ctx(BdrvChild *child, 
AioContext *ctx,
 static void bdrv_temp_snapshot_options(int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options)
 {
+GLOBAL_STATE_CODE();
 *child_flags = (parent_flags & ~BDRV_O_SNAPSHOT) | BDRV_O_TEMPORARY;
 
 /* For temporary files, unconditional cache=unsafe is fine */
@@ -1264,6 +1276,7 @@ static void bdrv_backing_attach(BdrvChild *c)
 BlockDriverState *parent = c->opaque;
 BlockDriverState *backing_hd = c->bs;
 
+GLOBAL_STATE_CODE();
 assert(!parent->backing_blocker);
 error_setg(>backing_blocker,
"node is used as backing hd of '%s'",
@@ -1302,6 +1315,7 @@ static void bdrv_backing_detach(BdrvChild *c)
 {
 BlockDriverState *parent = c->opaque;
 
+GLOBAL_STATE_CODE();
 assert(parent->backing_blocker);
 bdrv_op_unblock_all(c->bs, parent->backing_blocker);
 error_free(parent->backing_blocker);
@@ -1314,6 +1328,7 @@ static int bdrv_backing_update_filename(BdrvChild *c, 
BlockDriverState *base,
 BlockDriverState *parent = c->opaque;
 bool read_only = bdrv_is_read_only(parent);
 int ret;
+GLOBAL_STATE_CODE();
 
 if (read_only) {
 ret = bdrv_reopen_set_read_only(parent, false, errp);
@@ -1345,6 +1360,7 @@ static void bdrv_inherited_options(BdrvChildRole role, 
bool parent_is_format,
int parent_flags, QDict *parent_options)
 {
 int flags = parent_flags;
+GLOBAL_STATE_CODE();
 
 /*
  * First, decide whether to set, clear, or leave BDRV_O_PROTOCOL.
@@ -1486,6 +1502,7 @@ AioContext *bdrv_child_get_parent_aio_context(BdrvChild 
*c)
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
 int open_flags = flags;
+GLOBAL_STATE_CODE();
 
 /*
  * Clear flags that are internal to the block layer before opening the
@@ -1498,6 +1515,8 @@ static int 

[PULL 37/50] block/coroutines: I/O and "I/O or GS" API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

block coroutines functions run in different aiocontext, and are
not protected by the BQL. Therefore are I/O.

On the other side, generated_co_wrapper functions use BDRV_POLL_WHILE,
meaning the caller can either be the main loop or a specific iothread.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-25-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/coroutines.h| 81 +++
 block.c   |  2 ++
 block/block-backend.c |  6 
 block/io.c|  3 ++
 block/nbd.c   |  1 +
 5 files changed, 64 insertions(+), 29 deletions(-)

diff --git a/block/coroutines.h b/block/coroutines.h
index c8c14a29c8..b293e943c8 100644
--- a/block/coroutines.h
+++ b/block/coroutines.h
@@ -30,17 +30,17 @@
 /* For blk_bs() in generated block/block-gen.c */
 #include "sysemu/block-backend.h"
 
+/*
+ * I/O API functions. These functions are thread-safe.
+ *
+ * See include/block/block-io.h for more information about
+ * the I/O API.
+ */
+
 int coroutine_fn bdrv_co_check(BlockDriverState *bs,
BdrvCheckResult *res, BdrvCheckMode fix);
 int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
 
-int generated_co_wrapper
-bdrv_preadv(BdrvChild *child, int64_t offset, unsigned int bytes,
-QEMUIOVector *qiov, BdrvRequestFlags flags);
-int generated_co_wrapper
-bdrv_pwritev(BdrvChild *child, int64_t offset, unsigned int bytes,
- QEMUIOVector *qiov, BdrvRequestFlags flags);
-
 int coroutine_fn
 bdrv_co_common_block_status_above(BlockDriverState *bs,
   BlockDriverState *base,
@@ -52,6 +52,51 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
   int64_t *map,
   BlockDriverState **file,
   int *depth);
+
+int coroutine_fn bdrv_co_readv_vmstate(BlockDriverState *bs,
+   QEMUIOVector *qiov, int64_t pos);
+int coroutine_fn bdrv_co_writev_vmstate(BlockDriverState *bs,
+QEMUIOVector *qiov, int64_t pos);
+
+int coroutine_fn
+nbd_co_do_establish_connection(BlockDriverState *bs, Error **errp);
+
+
+int coroutine_fn
+blk_co_do_preadv(BlockBackend *blk, int64_t offset, int64_t bytes,
+ QEMUIOVector *qiov, BdrvRequestFlags flags);
+
+
+int coroutine_fn
+blk_co_do_pwritev_part(BlockBackend *blk, int64_t offset, int64_t bytes,
+   QEMUIOVector *qiov, size_t qiov_offset,
+   BdrvRequestFlags flags);
+
+int coroutine_fn
+blk_co_do_ioctl(BlockBackend *blk, unsigned long int req, void *buf);
+
+int coroutine_fn
+blk_co_do_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes);
+
+int coroutine_fn blk_co_do_flush(BlockBackend *blk);
+
+
+/*
+ * "I/O or GS" API functions. These functions can run without
+ * the BQL, but only in one specific iothread/main loop.
+ *
+ * See include/block/block-io.h for more information about
+ * the "I/O or GS" API.
+ */
+
+int generated_co_wrapper
+bdrv_preadv(BdrvChild *child, int64_t offset, unsigned int bytes,
+QEMUIOVector *qiov, BdrvRequestFlags flags);
+
+int generated_co_wrapper
+bdrv_pwritev(BdrvChild *child, int64_t offset, unsigned int bytes,
+ QEMUIOVector *qiov, BdrvRequestFlags flags);
+
 int generated_co_wrapper
 bdrv_common_block_status_above(BlockDriverState *bs,
BlockDriverState *base,
@@ -63,46 +108,24 @@ bdrv_common_block_status_above(BlockDriverState *bs,
int64_t *map,
BlockDriverState **file,
int *depth);
-
-int coroutine_fn bdrv_co_readv_vmstate(BlockDriverState *bs,
-   QEMUIOVector *qiov, int64_t pos);
-int coroutine_fn bdrv_co_writev_vmstate(BlockDriverState *bs,
-QEMUIOVector *qiov, int64_t pos);
-
 int generated_co_wrapper
 nbd_do_establish_connection(BlockDriverState *bs, Error **errp);
-int coroutine_fn
-nbd_co_do_establish_connection(BlockDriverState *bs, Error **errp);
-
 
 int generated_co_wrapper
 blk_do_preadv(BlockBackend *blk, int64_t offset, int64_t bytes,
   QEMUIOVector *qiov, BdrvRequestFlags flags);
-int coroutine_fn
-blk_co_do_preadv(BlockBackend *blk, int64_t offset, int64_t bytes,
- QEMUIOVector *qiov, BdrvRequestFlags flags);
-
 
 int generated_co_wrapper
 blk_do_pwritev_part(BlockBackend *blk, int64_t offset, int64_t bytes,
 QEMUIOVector *qiov, size_t qiov_offset,
 BdrvRequestFlags flags);
-int coroutine_fn
-blk_co_do_pwritev_part(BlockBackend *blk, int64_t offset, int64_t bytes,
-   QEMUIOVector *qiov, size_t qiov_offset,
-   BdrvRequestFlags flags);
 
 int 

[PULL 31/50] include/block/blockjob.h: global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

blockjob functions run always under the BQL lock.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-19-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/blockjob.h | 29 ++---
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/include/block/blockjob.h b/include/block/blockjob.h
index 87fbb3985f..6525e16fd5 100644
--- a/include/block/blockjob.h
+++ b/include/block/blockjob.h
@@ -74,6 +74,13 @@ typedef struct BlockJob {
 GSList *nodes;
 } BlockJob;
 
+/*
+ * Global state (GS) API. These functions run under the BQL.
+ *
+ * See include/block/block-global-state.h for more information about
+ * the GS API.
+ */
+
 /**
  * block_job_next:
  * @job: A block job, or %NULL.
@@ -155,6 +162,21 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp);
  */
 void block_job_iostatus_reset(BlockJob *job);
 
+/*
+ * block_job_get_aio_context:
+ *
+ * Returns aio context associated with a block job.
+ */
+AioContext *block_job_get_aio_context(BlockJob *job);
+
+
+/*
+ * Common functions that are neither I/O nor Global State.
+ *
+ * See include/block/block-common.h for more information about
+ * the Common API.
+ */
+
 /**
  * block_job_is_internal:
  * @job: The job to determine if it is user-visible or not.
@@ -170,11 +192,4 @@ bool block_job_is_internal(BlockJob *job);
  */
 const BlockJobDriver *block_job_driver(BlockJob *job);
 
-/*
- * block_job_get_aio_context:
- *
- * Returns aio context associated with a block job.
- */
-AioContext *block_job_get_aio_context(BlockJob *job);
-
 #endif
-- 
2.35.1

[PULL 41/50] block_int-common.h: assertions in the callers of BdrvChildClass function pointers

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-29-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/block.c b/block.c
index e5ba40d028..718e4cae8b 100644
--- a/block.c
+++ b/block.c
@@ -1497,7 +1497,7 @@ const BdrvChildClass child_of_bds = {
 
 AioContext *bdrv_child_get_parent_aio_context(BdrvChild *c)
 {
-IO_CODE();
+GLOBAL_STATE_CODE();
 return c->klass->get_parent_aio_context(c);
 }
 
@@ -2128,6 +2128,7 @@ bool bdrv_is_writable(BlockDriverState *bs)
 
 static char *bdrv_child_user_desc(BdrvChild *c)
 {
+GLOBAL_STATE_CODE();
 return c->klass->get_parent_desc(c);
 }
 
@@ -2844,6 +2845,7 @@ static void bdrv_replace_child_noperm(BdrvChild **childp,
 
 assert(!child->frozen);
 assert(old_bs != new_bs);
+GLOBAL_STATE_CODE();
 
 if (old_bs && new_bs) {
 assert(bdrv_get_aio_context(old_bs) == bdrv_get_aio_context(new_bs));
@@ -2940,6 +2942,7 @@ static void bdrv_attach_child_common_abort(void *opaque)
 BdrvChild *child = *s->child;
 BlockDriverState *bs = child->bs;
 
+GLOBAL_STATE_CODE();
 /*
  * Pass free_empty_child=false, because we still need the child
  * for the AioContext operations on the parent below; those
@@ -3308,6 +3311,7 @@ void bdrv_unref_child(BlockDriverState *parent, BdrvChild 
*child)
 static void bdrv_parent_cb_change_media(BlockDriverState *bs, bool load)
 {
 BdrvChild *c;
+GLOBAL_STATE_CODE();
 QLIST_FOREACH(c, >parents, next_parent) {
 if (c->klass->change_media) {
 c->klass->change_media(c, load);
@@ -3807,6 +3811,7 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 
 assert(!child_class || !flags);
 assert(!child_class == !parent);
+GLOBAL_STATE_CODE();
 
 if (reference) {
 bool options_non_empty = options ? qdict_size(options) : false;
@@ -4193,6 +4198,7 @@ static BlockReopenQueue 
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
  * important to avoid graph changes between the recursive queuing here and
  * bdrv_reopen_multiple(). */
 assert(bs->quiesce_counter > 0);
+GLOBAL_STATE_CODE();
 
 if (bs_queue == NULL) {
 bs_queue = g_new0(BlockReopenQueue, 1);
@@ -7327,6 +7333,7 @@ void bdrv_set_aio_context_ignore(BlockDriverState *bs,
 BdrvChild *child, *parent;
 
 g_assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+GLOBAL_STATE_CODE();
 
 if (old_context == new_context) {
 return;
@@ -7399,6 +7406,7 @@ void bdrv_set_aio_context_ignore(BlockDriverState *bs,
 static bool bdrv_parent_can_set_aio_context(BdrvChild *c, AioContext *ctx,
 GSList **ignore, Error **errp)
 {
+GLOBAL_STATE_CODE();
 if (g_slist_find(*ignore, c)) {
 return true;
 }
-- 
2.35.1

[PATCH v2 2/5] python/utils: add VerboseProcessError

[John Snow]

This adds an Exception that extends the Python stdlib
subprocess.CalledProcessError.

The difference is that the str() method of this exception also adds the
stdout/stderr logs. In effect, if this exception goes unhandled, Python
will print the output in a visually distinct wrapper to the terminal so
that it's easy to spot in a sea of traceback information.

Signed-off-by: John Snow 
Reviewed-by: Eric Blake 
---
 python/qemu/utils/__init__.py | 36 +++
 1 file changed, 36 insertions(+)

diff --git a/python/qemu/utils/__init__.py b/python/qemu/utils/__init__.py
index 5babf40df2..355ac550bc 100644
--- a/python/qemu/utils/__init__.py
+++ b/python/qemu/utils/__init__.py
@@ -18,6 +18,7 @@
 import os
 import re
 import shutil
+from subprocess import CalledProcessError
 import textwrap
 from typing import Optional
 
@@ -26,6 +27,7 @@
 
 
 __all__ = (
+'VerboseProcessError',
 'add_visual_margin',
 'get_info_usernet_hostfwd_port',
 'kvm_available',
@@ -121,3 +123,37 @@ def _wrap(line: str) -> str:
 os.linesep.join(_wrap(line) for line in content.splitlines()),
 _bar(None, top=False),
 ))
+
+
+class VerboseProcessError(CalledProcessError):
+"""
+The same as CalledProcessError, but more verbose.
+
+This is useful for debugging failed calls during test executions.
+The return code, signal (if any), and terminal output will be displayed
+on unhandled exceptions.
+"""
+def summary(self) -> str:
+"""Return the normal CalledProcessError str() output."""
+return super().__str__()
+
+def __str__(self) -> str:
+lmargin = '  '
+width = -len(lmargin)
+sections = []
+
+name = 'output' if self.stderr is None else 'stdout'
+if self.stdout:
+sections.append(add_visual_margin(self.stdout, width, name))
+else:
+sections.append(f"{name}: N/A")
+
+if self.stderr:
+sections.append(add_visual_margin(self.stderr, width, 'stderr'))
+elif self.stderr is not None:
+sections.append("stderr: N/A")
+
+return os.linesep.join((
+self.summary(),
+textwrap.indent(os.linesep.join(sections), prefix=lmargin),
+))
-- 
2.34.1

Re: [PATCH v2 4/9] util: Return valid allocation for qemu_try_memalign() with zero size

[Philippe Mathieu-Daudé]

On 4/3/22 12:21, Peter Maydell wrote:

Currently qemu_try_memalign()'s behaviour if asked to allocate
0 bytes is rather variable:
  * on Windows, we will assert
  * on POSIX platforms, we get the underlying behaviour of
the posix_memalign() or equivalent function, which may be
either "return a valid non-NULL pointer" or "return NULL"

Explictly check for 0 byte allocations, so we get consistent
behaviour across platforms.  We handle them by incrementing the size
so that we return a valid non-NULL pointer that can later be passed
to qemu_vfree().  This is permitted behaviour for the
posix_memalign() API and is the most usual way that underlying
malloc() etc implementations handle a zero-sized allocation request,
because it won't trip up calling code that assumes NULL means an
error.  (This includes our own qemu_memalign(), which will abort on
NULL.)

This change is a preparation for sharing the qemu_try_memalign() code
between Windows and POSIX.

Signed-off-by: Peter Maydell 
---
  util/oslib-posix.c | 3 +++
  util/oslib-win32.c | 4 +++-
  2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 0278902ee79..f7e22f4ff9b 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -209,6 +209,9 @@ void *qemu_try_memalign(size_t alignment, size_t size)
  g_assert(is_power_of_2(alignment));
  }
  
+if (size == 0) {

+size++;
+}
  #if defined(CONFIG_POSIX_MEMALIGN)
  int ret;
  ret = posix_memalign(, alignment, size);
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 05857414695..8c28d70904d 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -48,12 +48,14 @@ void *qemu_try_memalign(size_t alignment, size_t size)
  {
  void *ptr;
  
-g_assert(size != 0);


Better, X_try_Y() functions are not supposed to fail.

Reviewed-by: Philippe Mathieu-Daudé 

[PATCH 3/4] Replace qemu_gettimeofday() with g_get_real_time()

[marcandre . lureau]

From: Marc-André Lureau 

GLib g_get_real_time() is an alternative to gettimeofday().

For semihosting, a few bits are lost on POSIX host, but this shouldn't
be a big concern.

Signed-off-by: Marc-André Lureau 
---
 blockdev.c |  8 
 hw/rtc/m41t80.c|  6 +++---
 hw/virtio/virtio-balloon.c |  9 +
 qapi/qmp-event.c   | 12 +---
 qemu-img.c |  8 
 qga/commands-posix.c   | 11 +--
 target/m68k/m68k-semi.c| 22 ++
 target/nios2/nios2-semi.c  | 24 +++-
 8 files changed, 39 insertions(+), 61 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 42e098b458b1..4b07dbfbdefc 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1235,7 +1235,7 @@ static void internal_snapshot_prepare(BlkActionState 
*common,
 BlockDriverState *bs;
 QEMUSnapshotInfo old_sn, *sn;
 bool ret;
-qemu_timeval tv;
+int64_t rt;
 BlockdevSnapshotInternal *internal;
 InternalSnapshotState *state;
 AioContext *aio_context;
@@ -1305,9 +1305,9 @@ static void internal_snapshot_prepare(BlkActionState 
*common,
 /* 3. take the snapshot */
 sn = >sn;
 pstrcpy(sn->name, sizeof(sn->name), name);
-qemu_gettimeofday();
-sn->date_sec = tv.tv_sec;
-sn->date_nsec = tv.tv_usec * 1000;
+rt = g_get_real_time();
+sn->date_sec = rt / G_USEC_PER_SEC;
+sn->date_nsec = (rt % G_USEC_PER_SEC) * 1000;
 sn->vm_clock_nsec = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 if (replay_mode != REPLAY_MODE_NONE) {
 sn->icount = replay_get_current_icount();
diff --git a/hw/rtc/m41t80.c b/hw/rtc/m41t80.c
index a00971a67e1c..e045c864bb44 100644
--- a/hw/rtc/m41t80.c
+++ b/hw/rtc/m41t80.c
@@ -47,7 +47,7 @@ static uint8_t m41t80_recv(I2CSlave *i2c)
 {
 M41t80State *s = M41T80(i2c);
 struct tm now;
-qemu_timeval tv;
+int64_t rt;
 
 if (s->addr < 0) {
 s->addr = 0;
@@ -57,8 +57,8 @@ static uint8_t m41t80_recv(I2CSlave *i2c)
 }
 switch (s->addr++) {
 case 0:
-qemu_gettimeofday();
-return to_bcd(tv.tv_usec / 1);
+rt = g_get_real_time();
+return to_bcd((rt % G_USEC_PER_SEC) / 1);
 case 1:
 return to_bcd(now.tm_sec);
 case 2:
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index e6c1b0aa46fe..b1bada84cecc 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -452,7 +452,6 @@ static void virtio_balloon_receive_stats(VirtIODevice 
*vdev, VirtQueue *vq)
 VirtQueueElement *elem;
 VirtIOBalloonStat stat;
 size_t offset = 0;
-qemu_timeval tv;
 
 elem = virtqueue_pop(vq, sizeof(VirtQueueElement));
 if (!elem) {
@@ -484,13 +483,7 @@ static void virtio_balloon_receive_stats(VirtIODevice 
*vdev, VirtQueue *vq)
 s->stats[tag] = val;
 }
 s->stats_vq_offset = offset;
-
-if (qemu_gettimeofday() < 0) {
-warn_report("%s: failed to get time of day", __func__);
-goto out;
-}
-
-s->stats_last_update = tv.tv_sec;
+s->stats_last_update = g_get_real_time() / G_USEC_PER_SEC;
 
 out:
 if (balloon_stats_enabled(s)) {
diff --git a/qapi/qmp-event.c b/qapi/qmp-event.c
index 19d3cd003833..025716b3ec37 100644
--- a/qapi/qmp-event.c
+++ b/qapi/qmp-event.c
@@ -20,15 +20,13 @@
 
 static void timestamp_put(QDict *qdict)
 {
-int err;
 QDict *ts;
-qemu_timeval tv;
+int64_t rt = g_get_real_time();
 
-err = qemu_gettimeofday();
-/* Put -1 to indicate failure of getting host time */
-ts = qdict_from_jsonf_nofail("{ 'seconds': %lld, 'microseconds': %lld }",
- err < 0 ? -1LL : (long long)tv.tv_sec,
- err < 0 ? -1LL : (long long)tv.tv_usec);
+ts = qdict_from_jsonf_nofail("{ 'seconds': %" G_GINT64_FORMAT
+ ", 'microseconds': %" G_GINT64_FORMAT "}",
+ rt / G_USEC_PER_SEC,
+ rt % G_USEC_PER_SEC);
 qdict_put(qdict, "timestamp", ts);
 }
 
diff --git a/qemu-img.c b/qemu-img.c
index 6fe2466032f9..e26773909684 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -3304,11 +3304,11 @@ static int img_snapshot(int argc, char **argv)
 char *filename, *snapshot_name = NULL;
 int c, ret = 0, bdrv_oflags;
 int action = 0;
-qemu_timeval tv;
 bool quiet = false;
 Error *err = NULL;
 bool image_opts = false;
 bool force_share = false;
+int64_t rt;
 
 bdrv_oflags = BDRV_O_RDWR;
 /* Parse commandline parameters */
@@ -3405,9 +3405,9 @@ static int img_snapshot(int argc, char **argv)
 memset(, 0, sizeof(sn));
 pstrcpy(sn.name, sizeof(sn.name), snapshot_name);
 
-qemu_gettimeofday();
-sn.date_sec = tv.tv_sec;
-sn.date_nsec = tv.tv_usec * 1000;
+rt = g_get_real_time();
+sn.date_sec = rt / G_USEC_PER_SEC;
+sn.date_nsec = (rt % G_USEC_PER_SEC) * 

[PULL 10/50] os-posix: Add os_set_daemonize()

[Kevin Wolf]

From: Hanna Reitz 

The daemonizing functions in os-posix (os_daemonize() and
os_setup_post()) only daemonize the process if the static `daemonize`
variable is set.  Right now, it can only be set by os_parse_cmd_args().

In order to use os_daemonize() and os_setup_post() from the storage
daemon to have it be daemonized, we need some other way to set this
`daemonize` variable, because I would rather not tap into the system
emulator's arg-parsing code.  Therefore, this patch adds an
os_set_daemonize() function, which will return an error on os-win32
(because daemonizing is not supported there).

Signed-off-by: Hanna Reitz 
Message-Id: <20220303164814.284974-2-hre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/sysemu/os-posix.h | 1 +
 include/sysemu/os-win32.h | 8 
 os-posix.c| 6 ++
 3 files changed, 15 insertions(+)

diff --git a/include/sysemu/os-posix.h b/include/sysemu/os-posix.h
index 2edf33658a..dd64fb401d 100644
--- a/include/sysemu/os-posix.h
+++ b/include/sysemu/os-posix.h
@@ -55,6 +55,7 @@ int os_mlock(void);
 typedef struct timeval qemu_timeval;
 #define qemu_gettimeofday(tp) gettimeofday(tp, NULL)
 
+int os_set_daemonize(bool d);
 bool is_daemonized(void);
 
 /**
diff --git a/include/sysemu/os-win32.h b/include/sysemu/os-win32.h
index 43f569b5c2..77075a 100644
--- a/include/sysemu/os-win32.h
+++ b/include/sysemu/os-win32.h
@@ -77,6 +77,14 @@ typedef struct {
 } qemu_timeval;
 int qemu_gettimeofday(qemu_timeval *tp);
 
+static inline int os_set_daemonize(bool d)
+{
+if (d) {
+return -ENOTSUP;
+}
+return 0;
+}
+
 static inline bool is_daemonized(void)
 {
 return false;
diff --git a/os-posix.c b/os-posix.c
index ae6c9f2a5e..24692c8593 100644
--- a/os-posix.c
+++ b/os-posix.c
@@ -317,6 +317,12 @@ bool is_daemonized(void)
 return daemonize;
 }
 
+int os_set_daemonize(bool d)
+{
+daemonize = d;
+return 0;
+}
+
 int os_mlock(void)
 {
 #ifdef HAVE_MLOCKALL
-- 
2.35.1

[PULL 11/50] qsd: Add pre-init argument parsing pass

[Kevin Wolf]

From: Hanna Reitz 

In contrast to qemu-nbd (where it is called --fork) and the system
emulator, QSD does not have a --daemonize switch yet.  Just like them,
QSD allows setting up block devices and exports on the command line.
When doing so, it is often necessary for whoever invoked the QSD to wait
until these exports are fully set up.  A --daemonize switch allows
precisely this, by virtue of the parent process exiting once everything
is set up.

Note that there are alternative ways of waiting for all exports to be
set up, for example:
- Passing the --pidfile option and waiting until the respective file
  exists (but I do not know if there is a way of implementing this
  without a busy wait loop)
- Set up some network server (e.g. on a Unix socket) and have the QSD
  connect to it after all arguments have been processed by appending
  corresponding --chardev and --monitor options to the command line,
  and then wait until the QSD connects

Having a --daemonize option would make this simpler, though, without
having to rely on additional tools (to set up a network server) or busy
waiting.

Implementing a --daemonize switch means having to fork the QSD process.
Ideally, we should do this as early as possible: All the parent process
has to do is to wait for the child process to signal completion of its
set-up phase, and therefore there is basically no initialization that
needs to be done before the fork.  On the other hand, forking after
initialization steps means having to consider how those steps (like
setting up the block layer or QMP) interact with a later fork, which is
often not trivial.

In order to fork this early, we must scan the command line for
--daemonize long before our current process_options() call.  Instead of
adding custom new code to do so, just reuse process_options() and give
it a @pre_init_pass argument to distinguish the two passes.  I believe
there are some other switches but --daemonize that deserve parsing in
the first pass:

- --help and --version are supposed to only print some text and then
  immediately exit (so any initialization we do would be for naught).
  This changes behavior, because now "--blockdev inv-drv --help" will
  print a help text instead of complaining about the --blockdev
  argument.
  Note that this is similar in behavior to other tools, though: "--help"
  is generally immediately acted upon when finding it in the argument
  list, potentially before other arguments (even ones before it) are
  acted on.  For example, "ls /does-not-exist --help" prints a help text
  and does not complain about ENOENT.

- --pidfile does not need initialization, and is already exempted from
  the sequential order that process_options() claims to strictly follow
  (the PID file is only created after all arguments are processed, not
  at the time the --pidfile argument appears), so it makes sense to
  include it in the same category as --daemonize.

- Invalid arguments should always be reported as soon as possible.  (The
  same caveat with --help applies: That means that "--blockdev inv-drv
  --inv-arg" will now complain about --inv-arg, not inv-drv.)

This patch does make some references to --daemonize without having
implemented it yet, but that will happen in the next patch.

Signed-off-by: Hanna Reitz 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Message-Id: <20220303164814.284974-3-hre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 storage-daemon/qemu-storage-daemon.c | 43 
 1 file changed, 38 insertions(+), 5 deletions(-)

diff --git a/storage-daemon/qemu-storage-daemon.c 
b/storage-daemon/qemu-storage-daemon.c
index 504d33aa91..b798954edb 100644
--- a/storage-daemon/qemu-storage-daemon.c
+++ b/storage-daemon/qemu-storage-daemon.c
@@ -177,7 +177,23 @@ static int getopt_set_loc(int argc, char **argv, const 
char *optstring,
 return c;
 }
 
-static void process_options(int argc, char *argv[])
+/**
+ * Process QSD command-line arguments.
+ *
+ * This is done in two passes:
+ *
+ * First (@pre_init_pass is true), we do a pass where all global
+ * arguments pertaining to the QSD process (like --help or --daemonize)
+ * are processed.  This pass is done before most of the QEMU-specific
+ * initialization steps (e.g. initializing the block layer or QMP), and
+ * so must only process arguments that are not really QEMU-specific.
+ *
+ * Second (@pre_init_pass is false), we (sequentially) process all
+ * QEMU/QSD-specific arguments.  Many of these arguments are effectively
+ * translated to QMP commands (like --blockdev for blockdev-add, or
+ * --export for block-export-add).
+ */
+static void process_options(int argc, char *argv[], bool pre_init_pass)
 {
 int c;
 
@@ -196,11 +212,26 @@ static void process_options(int argc, char *argv[])
 };
 
 /*
- * In contrast to the system emulator, options are processed in the order
- * they are given on the command lines. This means that things must be
- * defined first before they can be 

[PULL 15/50] main loop: macros to mark GS and I/O functions

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Righ now, IO_CODE and IO_OR_GS_CODE are nop, as there isn't
really a way to check that a function is only called in I/O.
On the other side, we can use qemu_in_main_thread() to check if
we are in the main loop.

The usage of macros makes easy to extend them in the future without
making changes in all callers. They will also visually help understanding
in which category each function is, without looking at the header.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-3-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/qemu/main-loop.h | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index bc42b5939d..7a4d6a0920 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -269,6 +269,24 @@ bool qemu_mutex_iothread_locked(void);
  */
 bool qemu_in_main_thread(void);
 
+/* Mark and check that the function is part of the global state API. */
+#define GLOBAL_STATE_CODE() \
+do {\
+assert(qemu_in_main_thread());  \
+} while (0)
+
+/* Mark and check that the function is part of the I/O API. */
+#define IO_CODE()   \
+do {\
+/* nop */   \
+} while (0)
+
+/* Mark and check that the function is part of the "I/O OR GS" API. */
+#define IO_OR_GS_CODE() \
+do {\
+/* nop */   \
+} while (0)
+
 /**
  * qemu_mutex_lock_iothread: Lock the main loop mutex.
  *
-- 
2.35.1

[PULL 19/50] block/export/fuse.c: allow writable exports to take RESIZE permission

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Allow writable exports to get BLK_PERM_RESIZE permission
from creation, in fuse_export_create().
In this way, there is no need to give the permission in
fuse_do_truncate(), which might be run in an iothread.

Permissions should be set only in the main thread, so
in any case if an iothread tries to set RESIZE, it will
be blocked.

Also assert in fuse_do_truncate that if we give the
RESIZE permission we can then restore the original ones.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Hanna Reitz 
Message-Id: <20220303151616.325444-7-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/export/fuse.c | 25 ++---
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/block/export/fuse.c b/block/export/fuse.c
index fdda8e3c81..5029e70f84 100644
--- a/block/export/fuse.c
+++ b/block/export/fuse.c
@@ -86,8 +86,8 @@ static int fuse_export_create(BlockExport *blk_exp,
 
 assert(blk_exp_args->type == BLOCK_EXPORT_TYPE_FUSE);
 
-/* For growable exports, take the RESIZE permission */
-if (args->growable) {
+/* For growable and writable exports, take the RESIZE permission */
+if (args->growable || blk_exp_args->writable) {
 uint64_t blk_perm, blk_shared_perm;
 
 blk_get_perm(exp->common.blk, _perm, _shared_perm);
@@ -392,14 +392,23 @@ static int fuse_do_truncate(const FuseExport *exp, 
int64_t size,
 {
 uint64_t blk_perm, blk_shared_perm;
 BdrvRequestFlags truncate_flags = 0;
-int ret;
+bool add_resize_perm;
+int ret, ret_check;
+
+/* Growable and writable exports have a permanent RESIZE permission */
+add_resize_perm = !exp->growable && !exp->writable;
 
 if (req_zero_write) {
 truncate_flags |= BDRV_REQ_ZERO_WRITE;
 }
 
-/* Growable exports have a permanent RESIZE permission */
-if (!exp->growable) {
+if (add_resize_perm) {
+
+if (!qemu_in_main_thread()) {
+/* Changing permissions like below only works in the main thread */
+return -EPERM;
+}
+
 blk_get_perm(exp->common.blk, _perm, _shared_perm);
 
 ret = blk_set_perm(exp->common.blk, blk_perm | BLK_PERM_RESIZE,
@@ -412,9 +421,11 @@ static int fuse_do_truncate(const FuseExport *exp, int64_t 
size,
 ret = blk_truncate(exp->common.blk, size, true, prealloc,
truncate_flags, NULL);
 
-if (!exp->growable) {
+if (add_resize_perm) {
 /* Must succeed, because we are only giving up the RESIZE permission */
-blk_set_perm(exp->common.blk, blk_perm, blk_shared_perm, _abort);
+ret_check = blk_set_perm(exp->common.blk, blk_perm,
+ blk_shared_perm, _abort);
+assert(ret_check == 0);
 }
 
 return ret;
-- 
2.35.1

[PULL 25/50] assertions for block_int global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-13-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 15 +++
 block/backup.c  |  1 +
 block/block-backend.c   |  3 +++
 block/commit.c  |  2 ++
 block/dirty-bitmap.c|  1 +
 block/io.c  |  1 +
 block/mirror.c  |  4 
 block/monitor/bitmap-qmp-cmds.c |  6 ++
 block/stream.c  |  2 ++
 blockdev.c  |  7 +++
 10 files changed, 42 insertions(+)

diff --git a/block.c b/block.c
index 5351b4f131..1751b14baf 100644
--- a/block.c
+++ b/block.c
@@ -665,6 +665,8 @@ int coroutine_fn bdrv_co_create_opts_simple(BlockDriver 
*drv,
 Error *local_err = NULL;
 int ret;
 
+GLOBAL_STATE_CODE();
+
 size = qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0);
 buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
 prealloc = qapi_enum_parse(_lookup, buf,
@@ -2504,6 +2506,8 @@ void bdrv_get_cumulative_perm(BlockDriverState *bs, 
uint64_t *perm,
 uint64_t cumulative_perms = 0;
 uint64_t cumulative_shared_perms = BLK_PERM_ALL;
 
+GLOBAL_STATE_CODE();
+
 QLIST_FOREACH(c, >parents, next_parent) {
 cumulative_perms |= c->perm;
 cumulative_shared_perms &= c->shared_perm;
@@ -2562,6 +2566,8 @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, 
uint64_t shared,
 Transaction *tran = tran_new();
 int ret;
 
+GLOBAL_STATE_CODE();
+
 bdrv_child_set_perm(c, perm, shared, tran);
 
 ret = bdrv_refresh_perms(c->bs, _err);
@@ -2592,6 +2598,8 @@ int bdrv_child_refresh_perms(BlockDriverState *bs, 
BdrvChild *c, Error **errp)
 uint64_t parent_perms, parent_shared;
 uint64_t perms, shared;
 
+GLOBAL_STATE_CODE();
+
 bdrv_get_cumulative_perm(bs, _perms, _shared);
 bdrv_child_perm(bs, c->bs, c, c->role, NULL,
 parent_perms, parent_shared, , );
@@ -2736,6 +2744,7 @@ void bdrv_default_perms(BlockDriverState *bs, BdrvChild 
*c,
 uint64_t perm, uint64_t shared,
 uint64_t *nperm, uint64_t *nshared)
 {
+GLOBAL_STATE_CODE();
 if (role & BDRV_CHILD_FILTERED) {
 assert(!(role & (BDRV_CHILD_DATA | BDRV_CHILD_METADATA |
  BDRV_CHILD_COW)));
@@ -3093,6 +3102,8 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
 BdrvChild *child = NULL;
 Transaction *tran = tran_new();
 
+GLOBAL_STATE_CODE();
+
 ret = bdrv_attach_child_common(child_bs, child_name, child_class,
child_role, perm, shared_perm, opaque,
, tran, errp);
@@ -7486,6 +7497,8 @@ bool bdrv_recurse_can_replace(BlockDriverState *bs,
 {
 BlockDriverState *filtered;
 
+GLOBAL_STATE_CODE();
+
 if (!bs || !bs->drv) {
 return false;
 }
@@ -7657,6 +7670,7 @@ static bool append_strong_runtime_options(QDict *d, 
BlockDriverState *bs)
  * would result in exactly bs->backing. */
 static bool bdrv_backing_overridden(BlockDriverState *bs)
 {
+GLOBAL_STATE_CODE();
 if (bs->backing) {
 return strcmp(bs->auto_backing_file,
   bs->backing->bs->filename);
@@ -8045,6 +8059,7 @@ static BlockDriverState 
*bdrv_do_skip_filters(BlockDriverState *bs,
  */
 BlockDriverState *bdrv_skip_implicit_filters(BlockDriverState *bs)
 {
+GLOBAL_STATE_CODE();
 return bdrv_do_skip_filters(bs, true);
 }
 
diff --git a/block/backup.c b/block/backup.c
index 21d5983779..5cfd0b999c 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -372,6 +372,7 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 
 assert(bs);
 assert(target);
+GLOBAL_STATE_CODE();
 
 /* QMP interface protects us from these cases */
 assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
diff --git a/block/block-backend.c b/block/block-backend.c
index be7adce246..93dc5cd8d6 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1106,6 +1106,7 @@ static void blk_root_change_media(BdrvChild *child, bool 
load)
  */
 bool blk_dev_has_removable_media(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 return !blk->dev || (blk->dev_ops && blk->dev_ops->change_media_cb);
 }
 
@@ -1123,6 +1124,7 @@ bool blk_dev_has_tray(BlockBackend *blk)
  */
 void blk_dev_eject_request(BlockBackend *blk, bool force)
 {
+GLOBAL_STATE_CODE();
 if (blk->dev_ops && blk->dev_ops->eject_request_cb) {
 blk->dev_ops->eject_request_cb(blk->dev_opaque, force);
 }
@@ -1145,6 +1147,7 @@ bool blk_dev_is_tray_open(BlockBackend *blk)
  */
 bool blk_dev_is_medium_locked(BlockBackend *blk)
 {
+GLOBAL_STATE_CODE();
 if (blk->dev_ops && blk->dev_ops->is_medium_locked) {
 return blk->dev_ops->is_medium_locked(blk->dev_opaque);
 }
diff --git a/block/commit.c b/block/commit.c
index 

[PULL 14/50] main-loop.h: introduce qemu_in_main_thread()

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

When invoked from the main loop, this function is the same
as qemu_mutex_iothread_locked, and returns true if the BQL is held.
When invoked from iothreads or tests, it returns true only
if the current AioContext is the Main Loop.

This essentially just extends qemu_mutex_iothread_locked to work
also in unit tests or other users like storage-daemon, that run
in the Main Loop but end up using the implementation in
stubs/iothread-lock.c.

Using qemu_mutex_iothread_locked in unit tests defaults to false
because they use the implementation in stubs/iothread-lock,
making all assertions added in next patches fail despite the
AioContext is still the main loop.

See the comment in the function header for more information.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-2-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/qemu/main-loop.h | 24 
 softmmu/cpus.c   |  5 +
 stubs/iothread-lock.c|  5 +
 3 files changed, 34 insertions(+)

diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index 8dbc6fcb89..bc42b5939d 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -242,9 +242,33 @@ AioContext *iohandler_get_aio_context(void);
  * must always be taken outside other locks.  This function helps
  * functions take different paths depending on whether the current
  * thread is running within the main loop mutex.
+ *
+ * This function should never be used in the block layer, because
+ * unit tests, block layer tools and qemu-storage-daemon do not
+ * have a BQL.
+ * Please instead refer to qemu_in_main_thread().
  */
 bool qemu_mutex_iothread_locked(void);
 
+/**
+ * qemu_in_main_thread: return whether it's possible to safely access
+ * the global state of the block layer.
+ *
+ * Global state of the block layer is not accessible from I/O threads
+ * or worker threads; only from threads that "own" the default
+ * AioContext that qemu_get_aio_context() returns.  For tests, block
+ * layer tools and qemu-storage-daemon there is a designated thread that
+ * runs the event loop for qemu_get_aio_context(), and that is the
+ * main thread.
+ *
+ * For emulators, however, any thread that holds the BQL can act
+ * as the block layer main thread; this will be any of the actual
+ * main thread, the vCPU threads or the RCU thread.
+ *
+ * For clarity, do not use this function outside the block layer.
+ */
+bool qemu_in_main_thread(void);
+
 /**
  * qemu_mutex_lock_iothread: Lock the main loop mutex.
  *
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index d1ff3cfea1..1681844b61 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -481,6 +481,11 @@ bool qemu_mutex_iothread_locked(void)
 return get_iothread_locked();
 }
 
+bool qemu_in_main_thread(void)
+{
+return qemu_mutex_iothread_locked();
+}
+
 /*
  * The BQL is taken from so many places that it is worth profiling the
  * callers directly, instead of funneling them all through a single function.
diff --git a/stubs/iothread-lock.c b/stubs/iothread-lock.c
index 5b45b7fc8b..ff7386e42c 100644
--- a/stubs/iothread-lock.c
+++ b/stubs/iothread-lock.c
@@ -6,6 +6,11 @@ bool qemu_mutex_iothread_locked(void)
 return false;
 }
 
+bool qemu_in_main_thread(void)
+{
+return qemu_get_current_aio_context() == qemu_get_aio_context();
+}
+
 void qemu_mutex_lock_iothread_impl(const char *file, int line)
 {
 }
-- 
2.35.1

[PULL 16/50] include/block/block: split header into I/O and global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

block.h currently contains a mix of functions:
some of them run under the BQL and modify the block layer graph,
others are instead thread-safe and perform I/O in iothreads.
Some others can only be called by either the main loop or the
iothread running the AioContext (and not other iothreads),
and using them in another thread would cause deadlocks, and therefore
it is not ideal to define them as I/O.

It is not easy to understand which function is part of which
group (I/O vs GS vs "I/O or GS"), and this patch aims to clarify it.

The "GS" functions need the BQL, and often use
aio_context_acquire/release and/or drain to be sure they
can modify the graph safely.
The I/O function are instead thread safe, and can run in
any AioContext.
"I/O or GS" functions run instead in the main loop or in
a single iothread, and use BDRV_POLL_WHILE().

By splitting the header in two files, block-io.h
and block-global-state.h we have a clearer view on what
needs what kind of protection. block-common.h
contains common structures shared by both headers.

block.h is left there for legacy and to avoid changing
all includes in all c files that use the block APIs.

Assertions are added in the next patch.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-4-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block-common.h   | 418 ++
 include/block/block-global-state.h | 253 +
 include/block/block-io.h   | 367 
 include/block/block.h  | 879 +
 block.c|   3 +
 block/meson.build  |   7 +-
 6 files changed, 1069 insertions(+), 858 deletions(-)
 create mode 100644 include/block/block-common.h
 create mode 100644 include/block/block-global-state.h
 create mode 100644 include/block/block-io.h

diff --git a/include/block/block-common.h b/include/block/block-common.h
new file mode 100644
index 00..0c5dc4a86a
--- /dev/null
+++ b/include/block/block-common.h
@@ -0,0 +1,418 @@
+/*
+ * QEMU System Emulator block driver
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#ifndef BLOCK_COMMON_H
+#define BLOCK_COMMON_H
+
+#include "block/aio.h"
+#include "block/aio-wait.h"
+#include "qemu/iov.h"
+#include "qemu/coroutine.h"
+#include "block/accounting.h"
+#include "block/dirty-bitmap.h"
+#include "block/blockjob.h"
+#include "qemu/hbitmap.h"
+#include "qemu/transactions.h"
+
+/*
+ * generated_co_wrapper
+ *
+ * Function specifier, which does nothing but mark functions to be
+ * generated by scripts/block-coroutine-wrapper.py
+ *
+ * Read more in docs/devel/block-coroutine-wrapper.rst
+ */
+#define generated_co_wrapper
+
+/* block.c */
+typedef struct BlockDriver BlockDriver;
+typedef struct BdrvChild BdrvChild;
+typedef struct BdrvChildClass BdrvChildClass;
+
+typedef struct BlockDriverInfo {
+/* in bytes, 0 if irrelevant */
+int cluster_size;
+/* offset at which the VM state can be saved (0 if not possible) */
+int64_t vm_state_offset;
+bool is_dirty;
+/*
+ * True if this block driver only supports compressed writes
+ */
+bool needs_compressed_writes;
+} BlockDriverInfo;
+
+typedef struct BlockFragInfo {
+uint64_t allocated_clusters;
+uint64_t total_clusters;
+uint64_t fragmented_clusters;
+uint64_t compressed_clusters;
+} BlockFragInfo;
+
+typedef enum {
+BDRV_REQ_COPY_ON_READ   = 0x1,
+BDRV_REQ_ZERO_WRITE = 0x2,
+
+/*
+ * The BDRV_REQ_MAY_UNMAP flag is used in write_zeroes requests to indicate
+ * that the block driver should unmap (discard) blocks if it is guaranteed
+ * that the result will read back as zeroes. The flag is only passed to the
+ * driver if the block device is opened with BDRV_O_UNMAP.
+ */
+BDRV_REQ_MAY_UNMAP  = 0x4,
+
+BDRV_REQ_FUA 

[PULL 24/50] include/block/block_int: split header into I/O and global state API

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Similarly to the previous patch, split block_int.h
in block_int-io.h and block_int-global-state.h

block_int-common.h contains the structures shared between
the two headers, and the functions that can't be categorized as
I/O or global state.

Assertions are added in the next patch.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-12-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int-common.h   | 1180 +++
 include/block/block_int-global-state.h |  312 +
 include/block/block_int-io.h   |  179 +++
 include/block/block_int.h  | 1489 +---
 blockdev.c |5 +
 5 files changed, 1679 insertions(+), 1486 deletions(-)
 create mode 100644 include/block/block_int-common.h
 create mode 100644 include/block/block_int-global-state.h
 create mode 100644 include/block/block_int-io.h

diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
new file mode 100644
index 00..b92e3630fd
--- /dev/null
+++ b/include/block/block_int-common.h
@@ -0,0 +1,1180 @@
+/*
+ * QEMU System Emulator block driver
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#ifndef BLOCK_INT_COMMON_H
+#define BLOCK_INT_COMMON_H
+
+#include "block/accounting.h"
+#include "block/block.h"
+#include "block/aio-wait.h"
+#include "qemu/queue.h"
+#include "qemu/coroutine.h"
+#include "qemu/stats64.h"
+#include "qemu/timer.h"
+#include "qemu/hbitmap.h"
+#include "block/snapshot.h"
+#include "qemu/throttle.h"
+#include "qemu/rcu.h"
+
+#define BLOCK_FLAG_LAZY_REFCOUNTS   8
+
+#define BLOCK_OPT_SIZE  "size"
+#define BLOCK_OPT_ENCRYPT   "encryption"
+#define BLOCK_OPT_ENCRYPT_FORMAT"encrypt.format"
+#define BLOCK_OPT_COMPAT6   "compat6"
+#define BLOCK_OPT_HWVERSION "hwversion"
+#define BLOCK_OPT_BACKING_FILE  "backing_file"
+#define BLOCK_OPT_BACKING_FMT   "backing_fmt"
+#define BLOCK_OPT_CLUSTER_SIZE  "cluster_size"
+#define BLOCK_OPT_TABLE_SIZE"table_size"
+#define BLOCK_OPT_PREALLOC  "preallocation"
+#define BLOCK_OPT_SUBFMT"subformat"
+#define BLOCK_OPT_COMPAT_LEVEL  "compat"
+#define BLOCK_OPT_LAZY_REFCOUNTS"lazy_refcounts"
+#define BLOCK_OPT_ADAPTER_TYPE  "adapter_type"
+#define BLOCK_OPT_REDUNDANCY"redundancy"
+#define BLOCK_OPT_NOCOW "nocow"
+#define BLOCK_OPT_EXTENT_SIZE_HINT  "extent_size_hint"
+#define BLOCK_OPT_OBJECT_SIZE   "object_size"
+#define BLOCK_OPT_REFCOUNT_BITS "refcount_bits"
+#define BLOCK_OPT_DATA_FILE "data_file"
+#define BLOCK_OPT_DATA_FILE_RAW "data_file_raw"
+#define BLOCK_OPT_COMPRESSION_TYPE  "compression_type"
+#define BLOCK_OPT_EXTL2 "extended_l2"
+
+#define BLOCK_PROBE_BUF_SIZE512
+
+enum BdrvTrackedRequestType {
+BDRV_TRACKED_READ,
+BDRV_TRACKED_WRITE,
+BDRV_TRACKED_DISCARD,
+BDRV_TRACKED_TRUNCATE,
+};
+
+/*
+ * That is not quite good that BdrvTrackedRequest structure is public,
+ * as block/io.c is very careful about incoming offset/bytes being
+ * correct. Be sure to assert bdrv_check_request() succeeded after any
+ * modification of BdrvTrackedRequest object out of block/io.c
+ */
+typedef struct BdrvTrackedRequest {
+BlockDriverState *bs;
+int64_t offset;
+int64_t bytes;
+enum BdrvTrackedRequestType type;
+
+bool serialising;
+int64_t overlap_offset;
+int64_t overlap_bytes;
+
+QLIST_ENTRY(BdrvTrackedRequest) list;
+Coroutine *co; /* owner, used for deadlock detection */
+CoQueue wait_queue; /* coroutines blocked on this request */
+
+struct BdrvTrackedRequest *waiting_for;
+} BdrvTrackedRequest;
+
+
+struct BlockDriver {
+const char *format_name;
+int instance_size;
+
+/*
+ * Set to true if the BlockDriver is a block 

[PULL 29/50] GS and IO CODE macros for blockjob_int.h

[Kevin Wolf]

From: Emanuele Giuseppe Esposito 

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20220303151616.325444-17-eespo...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 blockjob.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/blockjob.c b/blockjob.c
index 10815a89fe..d79a52d204 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -84,6 +84,7 @@ BlockJob *block_job_get(const char *id)
 void block_job_free(Job *job)
 {
 BlockJob *bjob = container_of(job, BlockJob, job);
+GLOBAL_STATE_CODE();
 
 block_job_remove_all_bdrv(bjob);
 ratelimit_destroy(>limit);
@@ -299,6 +300,7 @@ bool block_job_set_speed(BlockJob *job, int64_t speed, 
Error **errp)
 
 int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n)
 {
+IO_CODE();
 return ratelimit_calculate_delay(>limit, n);
 }
 
@@ -434,6 +436,7 @@ void *block_job_create(const char *job_id, const 
BlockJobDriver *driver,
 {
 BlockJob *job;
 int ret;
+GLOBAL_STATE_CODE();
 
 if (job_id == NULL && !(flags & JOB_INTERNAL)) {
 job_id = bdrv_get_device_name(bs);
@@ -498,6 +501,7 @@ void block_job_iostatus_reset(BlockJob *job)
 void block_job_user_resume(Job *job)
 {
 BlockJob *bjob = container_of(job, BlockJob, job);
+GLOBAL_STATE_CODE();
 block_job_iostatus_reset(bjob);
 }
 
@@ -505,6 +509,7 @@ BlockErrorAction block_job_error_action(BlockJob *job, 
BlockdevOnError on_err,
 int is_read, int error)
 {
 BlockErrorAction action;
+IO_CODE();
 
 switch (on_err) {
 case BLOCKDEV_ON_ERROR_ENOSPC:
-- 
2.35.1

[PULL 07/50] util/async: replace __thread with QEMU TLS macros

[Kevin Wolf]

From: Stefan Hajnoczi 

QEMU TLS macros must be used to make TLS variables safe with coroutines.

Signed-off-by: Stefan Hajnoczi 
Message-Id: <20220222140150.27240-3-stefa...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 util/async.c | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/util/async.c b/util/async.c
index 08d25feef5..2ea1172f3e 100644
--- a/util/async.c
+++ b/util/async.c
@@ -32,6 +32,7 @@
 #include "qemu/rcu_queue.h"
 #include "block/raw-aio.h"
 #include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
 #include "trace.h"
 
 /***/
@@ -675,12 +676,13 @@ void aio_context_release(AioContext *ctx)
 qemu_rec_mutex_unlock(>lock);
 }
 
-static __thread AioContext *my_aiocontext;
+QEMU_DEFINE_STATIC_CO_TLS(AioContext *, my_aiocontext)
 
 AioContext *qemu_get_current_aio_context(void)
 {
-if (my_aiocontext) {
-return my_aiocontext;
+AioContext *ctx = get_my_aiocontext();
+if (ctx) {
+return ctx;
 }
 if (qemu_mutex_iothread_locked()) {
 /* Possibly in a vCPU thread.  */
@@ -691,6 +693,6 @@ AioContext *qemu_get_current_aio_context(void)
 
 void qemu_set_current_aio_context(AioContext *ctx)
 {
-assert(!my_aiocontext);
-my_aiocontext = ctx;
+assert(!get_my_aiocontext());
+set_my_aiocontext(ctx);
 }
-- 
2.35.1

[PATCH v2 01/12] crypto: mandate a hostname when checking x509 creds on a client

[Daniel P . Berrangé]

Currently the TLS session object assumes that the caller will always
provide a hostname when using x509 creds on a client endpoint. This
relies on the caller to detect and report an error if the user has
configured QEMU with x509 credentials on a UNIX socket. The migration
code has such a check, but it is too broad, reporting an error when
the user has configured QEMU with PSK credentials on a UNIX socket,
where hostnames are irrelevant.

Putting the check into the TLS session object credentials validation
code ensures we report errors in only the scenario that matters.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---
 crypto/tlssession.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/crypto/tlssession.c b/crypto/tlssession.c
index a8db8c76d1..b302d835d2 100644
--- a/crypto/tlssession.c
+++ b/crypto/tlssession.c
@@ -373,6 +373,12 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession 
*session,
session->hostname);
 goto error;
 }
+} else {
+if (session->creds->endpoint ==
+QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
+error_setg(errp, "No hostname for certificate validation");
+goto error;
+}
 }
 }
 
-- 
2.34.1

Re: [PATCH v2 8/9] util: Put qemu_vfree() in memalign.c

[Philippe Mathieu-Daudé]

On 4/3/22 12:21, Peter Maydell wrote:

qemu_vfree() is the companion free function to qemu_memalign(); put
it in memalign.c so the allocation and free functions are together.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-id: 20220226180723.1706285-9-peter.mayd...@linaro.org
---
  util/memalign.c| 11 +++
  util/oslib-posix.c |  6 --
  util/oslib-win32.c |  6 --
  3 files changed, 11 insertions(+), 12 deletions(-)


Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH 09/12] tests/qemu-iotests: convert NBD TLS test to use standard filters

[Eric Blake]

On Thu, Mar 03, 2022 at 04:03:27PM +, Daniel P. Berrangé wrote:
> Using standard filters is more future proof than rolling our own.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
>  tests/qemu-iotests/233 | 29 -
>  tests/qemu-iotests/233.out |  9 -
>  2 files changed, 16 insertions(+), 22 deletions(-)

Reviewed-by: Eric Blake 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org