Re: [PULL 0/6] Block layer patches

2024-03-26 Thread Peter Maydell
On Tue, 26 Mar 2024 at 13:54, Kevin Wolf wrote: > > The following changes since commit 096ae430a7b5a704af4cd94dca7200d6cb069991: > > Merge tag 'pull-qapi-2024-03-26' of https://repo.or.cz/qemu/armbru into > staging (2024-03-26 09:50:21 +) > > are available in the Git repository at: > >

Re: [RFC 0/8] virtio,vhost: Add VIRTIO_F_IN_ORDER support

2024-03-26 Thread Jonah Palmer
On 3/26/24 2:34 PM, Eugenio Perez Martin wrote: On Tue, Mar 26, 2024 at 5:49 PM Jonah Palmer wrote: On 3/25/24 4:33 PM, Eugenio Perez Martin wrote: On Mon, Mar 25, 2024 at 5:52 PM Jonah Palmer wrote: On 3/22/24 7:18 AM, Eugenio Perez Martin wrote: On Thu, Mar 21, 2024 at 4:57 PM

Re: [RFC 0/8] virtio,vhost: Add VIRTIO_F_IN_ORDER support

2024-03-26 Thread Eugenio Perez Martin
On Tue, Mar 26, 2024 at 5:49 PM Jonah Palmer wrote: > > > > On 3/25/24 4:33 PM, Eugenio Perez Martin wrote: > > On Mon, Mar 25, 2024 at 5:52 PM Jonah Palmer > > wrote: > >> > >> > >> > >> On 3/22/24 7:18 AM, Eugenio Perez Martin wrote: > >>> On Thu, Mar 21, 2024 at 4:57 PM Jonah Palmer > >>>

Re: [PATCH-for-9.0? v2 3/4] qtest/libqos: Reduce size_to_prdtl() declaration scope

2024-03-26 Thread Thomas Huth
On 26/03/2024 18.10, Philippe Mathieu-Daudé wrote: Since size_to_prdtl() is only used within ahci.c, declare it statically. This removes the last use of "inlined function with external linkage". See previous commit and commit 9de9fa5cf2 for rationale. Suggested-by: Peter Maydell Signed-off-by:

Re: [PATCH-for-9.0? v2 3/4] qtest/libqos: Reduce size_to_prdtl() declaration scope

2024-03-26 Thread Peter Maydell
On Tue, 26 Mar 2024 at 17:10, Philippe Mathieu-Daudé wrote: > > Since size_to_prdtl() is only used within ahci.c, > declare it statically. This removes the last use > of "inlined function with external linkage". See > previous commit and commit 9de9fa5cf2 for rationale. > > Suggested-by: Peter

[PATCH-for-9.0? v2 3/4] qtest/libqos: Reduce size_to_prdtl() declaration scope

2024-03-26 Thread Philippe Mathieu-Daudé
Since size_to_prdtl() is only used within ahci.c, declare it statically. This removes the last use of "inlined function with external linkage". See previous commit and commit 9de9fa5cf2 for rationale. Suggested-by: Peter Maydell Signed-off-by: Philippe Mathieu-Daudé ---

Re: [RFC 0/8] virtio,vhost: Add VIRTIO_F_IN_ORDER support

2024-03-26 Thread Jonah Palmer
On 3/25/24 4:33 PM, Eugenio Perez Martin wrote: On Mon, Mar 25, 2024 at 5:52 PM Jonah Palmer wrote: On 3/22/24 7:18 AM, Eugenio Perez Martin wrote: On Thu, Mar 21, 2024 at 4:57 PM Jonah Palmer wrote: The goal of these patches is to add support to a variety of virtio and vhost devices

Re: [PATCH for-9.1 v2 09/11] hostmem: add a new memory backend based on POSIX shm_open()

2024-03-26 Thread David Hildenbrand
+mode = 0; +oflag = O_RDWR | O_CREAT | O_EXCL; +backend_name = host_memory_backend_get_name(backend); + +/* + * Some operating systems allow creating anonymous POSIX shared memory + * objects (e.g. FreeBSD provides the SHM_ANON constant), but this is not + * defined by

Re: [PATCH for-9.1 v2 04/11] vhost-user-server: don't abort if we can't set fd non-blocking

2024-03-26 Thread Eric Blake
On Tue, Mar 26, 2024 at 02:39:29PM +0100, Stefano Garzarella wrote: > In vhost-user-server we set all fd received from the other peer > in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.) > if we fail, it's not really a problem, because we don't use these > fd with blocking

Re: [PATCH for-9.1 v2 03/11] libvhost-user: mask F_INFLIGHT_SHMFD if memfd is not supported

2024-03-26 Thread Eric Blake
On Tue, Mar 26, 2024 at 02:39:28PM +0100, Stefano Garzarella wrote: > libvhost-user will panic when receiving VHOST_USER_GET_INFLIGHT_FD > message if MFD_ALLOW_SEALING is not defined, since it's not able > to create a memfd. > > VHOST_USER_GET_INFLIGHT_FD is used only if >

Re: [PATCH for-9.1 v2 02/11] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-03-26 Thread David Hildenbrand
On 26.03.24 15:34, Eric Blake wrote: On Tue, Mar 26, 2024 at 02:39:27PM +0100, Stefano Garzarella wrote: In vu_message_write() we use sendmsg() to send the message header, then a write() to send the payload. If sendmsg() fails we should avoid sending the payload, since we were unable to send

Re: [PATCH for-9.1 v2 02/11] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-03-26 Thread Eric Blake
On Tue, Mar 26, 2024 at 02:39:27PM +0100, Stefano Garzarella wrote: > In vu_message_write() we use sendmsg() to send the message header, > then a write() to send the payload. > > If sendmsg() fails we should avoid sending the payload, since we > were unable to send the header. > > Discovered

Re: [PATCH for-9.1 v2 01/11] libvhost-user: set msg.msg_control to NULL when it is empty

2024-03-26 Thread David Hildenbrand
On 26.03.24 14:39, Stefano Garzarella wrote: On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if the `struct msghdr` has the field `msg_controllen` set to 0, but `msg_control` is not NULL. Signed-off-by: Stefano Garzarella --- subprojects/libvhost-user/libvhost-user.c | 1 + 1

Re: [PATCH for-9.1 v2 01/11] libvhost-user: set msg.msg_control to NULL when it is empty

2024-03-26 Thread Eric Blake
On Tue, Mar 26, 2024 at 02:39:26PM +0100, Stefano Garzarella wrote: > On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if > the `struct msghdr` has the field `msg_controllen` set to 0, but > `msg_control` is not NULL. > > Signed-off-by: Stefano Garzarella > --- >

[PATCH for-9.1 v2 04/11] vhost-user-server: don't abort if we can't set fd non-blocking

2024-03-26 Thread Stefano Garzarella
In vhost-user-server we set all fd received from the other peer in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.) if we fail, it's not really a problem, because we don't use these fd with blocking operations, but only to map memory. In these cases a failure is not bad, so let's

[PATCH for-9.1 v2 00/11] vhost-user: support any POSIX system (tested on macOS, FreeBSD, OpenBSD)

2024-03-26 Thread Stefano Garzarella
v1: https://patchew.org/QEMU/20240228114759.44758-1-sgarz...@redhat.com/ v2: - fixed Author email and little typos in some patches - added memory-backend-shm (patches 9, 10, 11) [Daniel, David, Markus] - removed changes to memory-backend-file (ex patch 9) - used memory-backend-shm in

[PATCH for-9.1 v2 08/11] contrib/vhost-user-blk: enable it on any POSIX system

2024-03-26 Thread Stefano Garzarella
Let's make the code more portable by using the "qemu/bswap.h" API and adding defines from block/file-posix.c to support O_DIRECT in other systems (e.g. macOS). vhost-user-server.c is a dependency, let's enable it for any POSIX system. Signed-off-by: Stefano Garzarella --- meson.build

[PATCH for-9.1 v2 01/11] libvhost-user: set msg.msg_control to NULL when it is empty

2024-03-26 Thread Stefano Garzarella
On some OS (e.g. macOS) sendmsg() returns -1 (errno EINVAL) if the `struct msghdr` has the field `msg_controllen` set to 0, but `msg_control` is not NULL. Signed-off-by: Stefano Garzarella --- subprojects/libvhost-user/libvhost-user.c | 1 + 1 file changed, 1 insertion(+) diff --git

[PATCH for-9.1 v2 11/11] tests/qtest/vhost-user-test: add a test case for memory-backend-shm

2024-03-26 Thread Stefano Garzarella
`memory-backend-shm` can be used with vhost-user devices, so let's add a new test case for it. Signed-off-by: Stefano Garzarella --- tests/qtest/vhost-user-test.c | 23 +++ 1 file changed, 23 insertions(+) diff --git a/tests/qtest/vhost-user-test.c

[PATCH for-9.1 v2 07/11] libvhost-user: enable it on any POSIX system

2024-03-26 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific so let's enable libvhost-user for any POSIX system. Compiling it on macOS and FreeBSD some problems came up: - avoid to include linux/vhost.h which is avaibale only on Linux (vhost_types.h contains many of the things we need) - macOS doesn't

[PATCH for-9.1 v2 10/11] tests/qtest/vhost-user-blk-test: use memory-backend-shm

2024-03-26 Thread Stefano Garzarella
`memory-backend-memfd` is available only on Linux while the new `memory-backend-shm` can be used on any POSIX-compliant operating system. Let's use it so we can run the test in multiple environments. Signed-off-by: Stefano Garzarella --- tests/qtest/vhost-user-blk-test.c | 2 +- 1 file changed,

[PATCH for-9.1 v2 03/11] libvhost-user: mask F_INFLIGHT_SHMFD if memfd is not supported

2024-03-26 Thread Stefano Garzarella
libvhost-user will panic when receiving VHOST_USER_GET_INFLIGHT_FD message if MFD_ALLOW_SEALING is not defined, since it's not able to create a memfd. VHOST_USER_GET_INFLIGHT_FD is used only if VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD is negotiated. So, let's mask that feature if the backend is not

[PATCH for-9.1 v2 02/11] libvhost-user: fail vu_message_write() if sendmsg() is failing

2024-03-26 Thread Stefano Garzarella
In vu_message_write() we use sendmsg() to send the message header, then a write() to send the payload. If sendmsg() fails we should avoid sending the payload, since we were unable to send the header. Discovered before fixing the issue with the previous patch, where sendmsg() failed on macOS due

[PULL 6/6] iotests: add test for stream job with an unaligned prefetch read

2024-03-26 Thread Kevin Wolf
From: Fiona Ebner Previously, bdrv_pad_request() could not deal with a NULL qiov when a read needed to be aligned. During prefetch, a stream job will pass a NULL qiov. Add a test case to cover this scenario. By accident, also covers a previous race during shutdown, where block graph changes

[PULL 4/6] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-03-26 Thread Kevin Wolf
From: Fiona Ebner The old_bs variable in bdrv_next() is currently determined by looking at the old block backend. However, if the block graph changes before the next bdrv_next() call, it might be that the associated BDS is not the same that was referenced previously. In that case, the wrong BDS

[PULL 3/6] block/io: accept NULL qiov in bdrv_pad_request

2024-03-26 Thread Kevin Wolf
From: Stefan Reiter Some operations, e.g. block-stream, perform reads while discarding the results (only copy-on-read matters). In this case, they will pass NULL as the target QEMUIOVector, which will however trip bdrv_pad_request, since it wants to extend its passed vector. In particular, this

[PULL 5/6] block-backend: fix edge case in bdrv_next_cleanup() where BDS associated to BB changes

2024-03-26 Thread Kevin Wolf
From: Fiona Ebner Same rationale as for commit "block-backend: fix edge case in bdrv_next() where BDS associated to BB changes". The block graph might change between the bdrv_next() call and the bdrv_next_cleanup() call, so it could be that the associated BDS is not the same that was referenced

[PULL 0/6] Block layer patches

2024-03-26 Thread Kevin Wolf
The following changes since commit 096ae430a7b5a704af4cd94dca7200d6cb069991: Merge tag 'pull-qapi-2024-03-26' of https://repo.or.cz/qemu/armbru into staging (2024-03-26 09:50:21 +) are available in the Git repository at: https://repo.or.cz/qemu/kevin.git tags/for-upstream for you to

[PULL 2/6] vdpa-dev: Fix initialisation order to restore VDUSE compatibility

2024-03-26 Thread Kevin Wolf
VDUSE requires that virtqueues are first enabled before the DRIVER_OK status flag is set; with the current API of the kernel module, it is impossible to enable the opposite order in our block export code because userspace is not notified when a virtqueue is enabled. This requirement also mathces

[PULL 1/6] tests/qemu-iotests: Test 157 and 227 require virtio-blk

2024-03-26 Thread Kevin Wolf
From: Thomas Huth Tests 157 and 227 use the virtio-blk device, so we have to mark these tests accordingly to be skipped if this devices is not available (e.g. when running the tests with qemu-system-avr only). Signed-off-by: Thomas Huth Message-ID: <20240325154737.1305063-1-th...@redhat.com>

[PATCH for-9.1 v2 09/11] hostmem: add a new memory backend based on POSIX shm_open()

2024-03-26 Thread Stefano Garzarella
shm_open() creates and opens a new POSIX shared memory object. A POSIX shared memory object allows creating memory backend with an associated file descriptor that can be shared with external processes (e.g. vhost-user). The new `memory-backend-shm` can be used as an alternative when

[PATCH for-9.1 v2 06/11] vhost-user: enable frontends on any POSIX system

2024-03-26 Thread Stefano Garzarella
The vhost-user protocol is not really Linux-specific so let's enable vhost-user frontends for any POSIX system. In vhost_net.c we use VHOST_FILE_UNBIND which is defined in a Linux specific header, let's define it for other systems as well. Signed-off-by: Stefano Garzarella --- meson.build

[PATCH for-9.1 v2 05/11] contrib/vhost-user-blk: fix bind() using the right size of the address

2024-03-26 Thread Stefano Garzarella
On macOS passing `-s /tmp/vhost.socket` parameter to the vhost-user-blk application, the bind was done on `/tmp/vhost.socke` pathname, missing the last character. This sounds like one of the portability problems described in the unix(7) manpage: Pathname sockets When binding a socket

Re: [PATCH v3 0/4] fix two edge cases related to stream block jobs

2024-03-26 Thread Kevin Wolf
Am 25.03.2024 um 21:11 hat Stefan Hajnoczi geschrieben: > On Fri, Mar 22, 2024 at 10:50:05AM +0100, Fiona Ebner wrote: > > Changes in v3: > > * Also deal with edge case in bdrv_next_cleanup(). Haven't run > > into an actual issue there, but at least the caller in > >

Re: [PATCH v3 2/4] block-backend: fix edge case in bdrv_next() where BDS associated to BB changes

2024-03-26 Thread Kevin Wolf
Am 22.03.2024 um 10:50 hat Fiona Ebner geschrieben: > The old_bs variable in bdrv_next() is currently determined by looking > at the old block backend. However, if the block graph changes before > the next bdrv_next() call, it might be that the associated BDS is not > the same that was referenced

Re: [PATCH v1] coroutine: avoid inserting duplicate coroutine to co_queue_wakeup

2024-03-26 Thread zhuyangyang via
On Mon, 25 Mar 2024 11:00:31 -0500 Eric Blake wrote: > > util/async.c | 13 - > > 1 file changed, 12 insertions(+), 1 deletion(-) > > > > diff --git a/util/async.c b/util/async.c > > index 0467890052..25fc1e6083 100644 > > --- a/util/async.c > > +++ b/util/async.c > > @@ -705,7

Re: [PATCH] tests/qemu-iotests: Test 157 and 227 require virtio-blk

2024-03-26 Thread Kevin Wolf
Am 25.03.2024 um 16:47 hat Thomas Huth geschrieben: > Tests 157 and 227 use the virtio-blk device, so we have to mark these > tests accordingly to be skipped if this devices is not available (e.g. > when running the tests with qemu-system-avr only). > > Signed-off-by: Thomas Huth Thanks,

Re: [PATCH v1] coroutine: avoid inserting duplicate coroutine to co_queue_wakeup

2024-03-26 Thread zhuyangyang via
On Mon, 25 Mar 2024 11:50:41 -0400, Stefan Hajnoczi wrote: > On Mon, Mar 25, 2024 at 05:18:50PM +0800, zhuyangyang wrote: > > If g_main_loop_run()/aio_poll() is called in the coroutine context, > > the pending coroutine may be woken up repeatedly, and the co_queue_wakeup > > may be disordered. >

Question about block graph lock limitation with generated co-wrappers

2024-03-26 Thread Fiona Ebner
Hi, we have a custom block driver downstream, which currently calls bdrv_get_info() (for its file child) in the bdrv_refresh_limits() callback. However, with graph locking, this doesn't work anymore. AFAICT, the reason is the following: The block driver has a backing file option. During

Re: [PATCH 1/2] copy-before-write: allow specifying minimum cluster size

2024-03-26 Thread Markus Armbruster
Fiona Ebner writes: > Useful to make discard-source work in the context of backup fleecing > when the fleecing image has a larger granularity than the backup > target. > > Copy-before-write operations will use at least this granularity and in > particular, discard requests to the source node