Re: [PATCH for-9.0 v3] vdpa-dev: Fix initialisation order to restore VDUSE compatibility

2024-03-18 Thread Eugenio Perez Martin
On Mon, Mar 18, 2024 at 10:02 AM Michael S. Tsirkin wrote: > > On Mon, Mar 18, 2024 at 12:31:26PM +0800, Jason Wang wrote: > > On Fri, Mar 15, 2024 at 11:59 PM Kevin Wolf wrote: > > > > > > VDUSE requires that virtqueues are first enabled before the DRIVER_OK > > > status flag is set; with the

Re: [PATCH v2] virtio-blk: iothread-vq-mapping coroutine pool sizing

2024-03-18 Thread Stefan Hajnoczi
On Tue, Mar 12, 2024 at 11:12:04AM -0400, Stefan Hajnoczi wrote: > It is possible to hit the sysctl vm.max_map_count limit when the > coroutine pool size becomes large. Each coroutine requires two mappings > (one for the stack and one for the guard page). QEMU can crash with > "failed to set up

RE: [EXTERNAL] [PATCH v3 for 9.1 5/6] vhost/vhost-user: Add VIRTIO_F_NOTIFICATION_DATA to vhost feature bits

2024-03-18 Thread Srujana Challa
> Subject: [EXTERNAL] [PATCH v3 for 9.1 5/6] vhost/vhost-user: Add > VIRTIO_F_NOTIFICATION_DATA to vhost feature bits > > Prioritize security for external emails: Confirm sender and content safety > before clicking links or opening attachments > >

qemu-img: very bad performance with the default number of coroutines (8) in presence of compression. Fixed adding "-m 1" to the cmdline.

2024-03-18 Thread Claudio Fontana
Hello, pretty much the $Subject. We had reports of very bad performance with the default options when running: $ time qemu-img convert -c -p -O qcow2 file.qcow2 file.disk real7m13.220s user6m56.682s sys 0m13.038s I studied this and noticed that for every coroutine a thread was

Re: [PATCH for-9.0 0/2] nbd: Fix server crash on reset with iothreads

2024-03-18 Thread Stefan Hajnoczi
On Thu, Mar 14, 2024 at 05:58:23PM +0100, Kevin Wolf wrote: > Kevin Wolf (2): > nbd/server: Fix race in draining the export > iotests: Add test for reset/AioContext switches with NBD exports > > nbd/server.c | 15 ++--- >

[PULL 09/15] tests/qemu-iotests: Restrict test 130 to the 'file' protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth Using "-drive ...,backing.file.filename=..." only works with the file protocol, but not with URIs, so mark this test accordingly. Signed-off-by: Thomas Huth Message-ID: <2024031508.153201-5-th...@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf ---

[PULL 06/15] tests/qemu-iotests: Fix test 033 for running with non-file protocols

2024-03-18 Thread Kevin Wolf
From: Thomas Huth When running iotest 033 with the ssh protocol, it fails with: 033 fail [14:48:31] [14:48:41] 10.2soutput mismatch --- /.../tests/qemu-iotests/033.out +++ /.../tests/qemu-iotests/scratch/qcow2-ssh-033/033.out.bad @@ -174,6 +174,7 @@ 512 bytes, X

[PULL 02/15] nbd/server: Fix race in draining the export

2024-03-18 Thread Kevin Wolf
When draining an NBD export, nbd_drained_begin() first sets client->quiescing so that nbd_client_receive_next_request() won't start any new request coroutines. Then nbd_drained_poll() tries to makes sure that we wait for any existing request coroutines by checking that client->nb_requests has

[PULL 12/15] tests/qemu-iotests: Restrict tests that use --image-opts to the 'file' protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth These tests 188, 189 and 198 use qemu-io with --image-opts with additional hard-coded parameters for the file protocol, so they cannot work for other protocols. Thus we have to limit these tests to the file protocol only. Signed-off-by: Thomas Huth Message-ID:

[PULL 10/15] tests/qemu-iotests: Restrict test 134 and 158 to the 'file' protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth Commit b25b387fa592 updated the iotests 134 and 158 to use the --image-opts parameter for qemu-io with file protocol related options, but forgot to update the _supported_proto line accordingly. So let's do that now. Fixes: b25b387fa5 ("qcow2: convert QCow2 to use QCryptoBlock

[PULL 11/15] tests/qemu-iotests: Restrict test 156 to the 'file' protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth The test fails completely when you try to use it with a different protocol, e.g. with "./check -ssh -qcow2 156". The test uses some hand-crafted JSON statements which cannot work with other protocols, thus let's change this test to only support the 'file' protocol.

[PULL 13/15] tests/qemu-iotests: Fix some tests that use --image-opts for other protocols

2024-03-18 Thread Kevin Wolf
From: Thomas Huth Tests 263, 284 and detect-zeroes-registered-buf use qemu-io with --image-opts so we have to enforce IMGOPTSSYNTAX=true here to get $TEST_IMG in shape for other protocols than "file". Signed-off-by: Thomas Huth Message-ID: <2024031508.153201-9-th...@redhat.com>

[PULL 04/15] blockdev: Fix blockdev-snapshot-sync error reporting for no medium

2024-03-18 Thread Kevin Wolf
From: Markus Armbruster When external_snapshot_abort() rejects a BlockDriverState without a medium, it creates an error like this: error_setg(errp, "Device '%s' has no medium", device); Trouble is @device can be null. My system formats null as "(null)", but other systems might crash.

[PULL 01/15] mirror: Don't call job_pause_point() under graph lock

2024-03-18 Thread Kevin Wolf
Calling job_pause_point() while holding the graph reader lock potentially results in a deadlock: bdrv_graph_wrlock() first drains everything, including the mirror job, which pauses it. The job is only unpaused at the end of the drain section, which is when the graph writer lock has been

[PULL 08/15] tests/qemu-iotests: Restrict test 114 to the 'file' protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth iotest 114 uses "truncate" and the qcow2.py script on the destination file, which both cannot deal with URIs. Thus this test needs the "file" protocol, otherwise it fails with an error message like this: truncate: cannot open

[PULL 14/15] tests/qemu-iotests: Restrict tests using "--blockdev file" to the file protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth Tests that use "--blockdev" with the "file" driver cannot work with other protocols, so we should mark them accordingly. Signed-off-by: Thomas Huth Message-ID: <2024031508.153201-10-th...@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf ---

[PULL 03/15] iotests: Add test for reset/AioContext switches with NBD exports

2024-03-18 Thread Kevin Wolf
This replicates the scenario in which the bug was reported. Unfortunately this relies on actually executing a guest (so that the firmware initialises the virtio-blk device and moves it to its configured iothread), so this can't make use of the qtest accelerator like most other test cases. I tried

[PULL 15/15] iotests: adapt to output change for recently introduced 'detached header' field

2024-03-18 Thread Kevin Wolf
From: Fiona Ebner Failure was noticed when running the tests for the qcow2 image format. Fixes: 0bd779e27e ("crypto: Introduce 'detached-header' field in QCryptoBlockInfoLUKS") Signed-off-by: Fiona Ebner Message-ID: <20240216101415.293769-1-f.eb...@proxmox.com> Reviewed-by: Daniel P. Berrangé

[PULL 00/15] Block layer patches

2024-03-18 Thread Kevin Wolf
The following changes since commit ba49d760eb04630e7b15f423ebecf6c871b8f77b: Merge tag 'pull-maintainer-final-130324-1' of https://gitlab.com/stsquad/qemu into staging (2024-03-13 15:12:14 +) are available in the Git repository at: https://repo.or.cz/qemu/kevin.git tags/for-upstream

[PULL 05/15] qemu-img: Fix Column Width and Improve Formatting in snapshot list

2024-03-18 Thread Kevin Wolf
From: Abhiram Tilak When running the command `qemu-img snapshot -l SNAPSHOT` the output of VM_CLOCK (measures the offset between host and VM clock) cannot to accommodate values in the order of thousands (4-digit). This line [1] hints on the problem. Additionally, the column width for the

[PULL 07/15] tests/qemu-iotests: Restrict test 066 to the 'file' protocol

2024-03-18 Thread Kevin Wolf
From: Thomas Huth The hand-crafted json statement in this test only works if the test is run with the "file" protocol, so mark this test accordingly. Signed-off-by: Thomas Huth Message-ID: <2024031508.153201-3-th...@redhat.com> Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf ---

Re: [PATCH for-9.0] mirror: Don't call job_pause_point() under graph lock

2024-03-18 Thread Stefan Hajnoczi
On Mon, Mar 18, 2024 at 12:37:19PM +0100, Kevin Wolf wrote: > Am 14.03.2024 um 15:29 hat Stefan Hajnoczi geschrieben: > > On Wed, Mar 13, 2024 at 04:30:00PM +0100, Kevin Wolf wrote: > > > Calling job_pause_point() while holding the graph reader lock > > > potentially results in a deadlock:

Re: [PATCH] iotests: adapt to output change for recently introduced 'detached header' field

2024-03-18 Thread Kevin Wolf
Am 16.02.2024 um 11:14 hat Fiona Ebner geschrieben: > Failure was noticed when running the tests for the qcow2 image format. > > Fixes: 0bd779e27e ("crypto: Introduce 'detached-header' field in > QCryptoBlockInfoLUKS") > Signed-off-by: Fiona Ebner Thanks, applied to the block branch. Kevin

Re: [PATCH 0/9] tests/qemu-iotests: Fix running with "check -ssh -qcow2"

2024-03-18 Thread Kevin Wolf
Am 15.03.2024 um 12:10 hat Thomas Huth geschrieben: > I recently wanted to check for some changes that I did to the URI handling > in the block layer code, but I had to discover that a lot of iotests only > work with the raw file format when using a protocol that is not "file", > i.e. "./check

Re: [PATCH v3 for 9.1 0/6] virtio, vhost: Add VIRTIO_F_NOTIFICATION_DATA support

2024-03-18 Thread Jiri Pirko
Mon, Mar 18, 2024 at 12:22:02PM CET, jonah.pal...@oracle.com wrote: > > >On 3/16/24 11:45 AM, Jiri Pirko wrote: >> Fri, Mar 15, 2024 at 05:55:51PM CET, jonah.pal...@oracle.com wrote: >> > The goal of these patches are to add support to a variety of virtio and >> > vhost devices for the

Re: [PATCH] blockdev: Fix blockdev-snapshot-sync error reporting for no medium

2024-03-18 Thread Kevin Wolf
Am 06.03.2024 um 15:28 hat Markus Armbruster geschrieben: > When external_snapshot_abort() rejects a BlockDriverState without a > medium, it creates an error like this: > > error_setg(errp, "Device '%s' has no medium", device); > > Trouble is @device can be null. My system formats null

Re: [PATCH for-9.0] mirror: Don't call job_pause_point() under graph lock

2024-03-18 Thread Kevin Wolf
Am 14.03.2024 um 15:29 hat Stefan Hajnoczi geschrieben: > On Wed, Mar 13, 2024 at 04:30:00PM +0100, Kevin Wolf wrote: > > Calling job_pause_point() while holding the graph reader lock > > potentially results in a deadlock: bdrv_graph_wrlock() first drains > > everything, including the mirror job,

Re: [PATCH v3 for 9.1 0/6] virtio, vhost: Add VIRTIO_F_NOTIFICATION_DATA support

2024-03-18 Thread Jonah Palmer
On 3/16/24 11:45 AM, Jiri Pirko wrote: Fri, Mar 15, 2024 at 05:55:51PM CET, jonah.pal...@oracle.com wrote: The goal of these patches are to add support to a variety of virtio and vhost devices for the VIRTIO_F_NOTIFICATION_DATA transport feature. This feature indicates that a driver will

Re: [PATCH 00/10] Reduce usage of QERR_ macros further

2024-03-18 Thread Markus Armbruster
Markus Armbruster writes: > Philippe posted "[PATCH v2 00/22] qapi: Kill 'qapi/qmp/qerror.h' for > good" a couple of months ago. I cherry-picked just its simplest parts > for now. > > Markus Armbruster (1): > error: Drop superfluous #include "qapi/qmp/qerror.h" Queued for 9.1 with PATCH 08's

Re: [PATCH 08/10] qapi: Correct error message for 'vcpu_dirty_limit' parameter

2024-03-18 Thread Philippe Mathieu-Daudé
On 18/3/24 09:58, Markus Armbruster wrote: Philippe Mathieu-Daudé writes: On 12/3/24 16:26, Zhao Liu wrote: On Tue, Mar 12, 2024 at 03:13:41PM +0100, Markus Armbruster wrote: Date: Tue, 12 Mar 2024 15:13:41 +0100 From: Markus Armbruster Subject: [PATCH 08/10] qapi: Correct error message

Re: [PATCH for-9.0 v3] vdpa-dev: Fix initialisation order to restore VDUSE compatibility

2024-03-18 Thread Michael S. Tsirkin
On Fri, Mar 15, 2024 at 04:59:49PM +0100, Kevin Wolf wrote: > VDUSE requires that virtqueues are first enabled before the DRIVER_OK > status flag is set; with the current API of the kernel module, it is > impossible to enable the opposite order in our block export code because > userspace is not

Re: [PATCH for-9.0 v3] vdpa-dev: Fix initialisation order to restore VDUSE compatibility

2024-03-18 Thread Michael S. Tsirkin
On Mon, Mar 18, 2024 at 12:31:26PM +0800, Jason Wang wrote: > On Fri, Mar 15, 2024 at 11:59 PM Kevin Wolf wrote: > > > > VDUSE requires that virtqueues are first enabled before the DRIVER_OK > > status flag is set; with the current API of the kernel module, it is > > impossible to enable the

Re: [PATCH 08/10] qapi: Correct error message for 'vcpu_dirty_limit' parameter

2024-03-18 Thread Markus Armbruster
Philippe Mathieu-Daudé writes: > On 12/3/24 16:26, Zhao Liu wrote: >> On Tue, Mar 12, 2024 at 03:13:41PM +0100, Markus Armbruster wrote: >>> Date: Tue, 12 Mar 2024 15:13:41 +0100 >>> From: Markus Armbruster >>> Subject: [PATCH 08/10] qapi: Correct error message for 'vcpu_dirty_limit' >>>