[PATCH v5 4/9] test-throttle: test read only and write only

2023-07-27 Thread zhenwei pi
Reviewed-by: Alberto Garcia Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- tests/unit/test-throttle.c | 66 ++ 1 file changed, 66 insertions(+) diff --git a/tests/unit/test-throttle.c b/tests/unit/test-throttle.c index a60b5fe22e..5547837a58

[PATCH v5 5/9] cryptodev: use NULL throttle timer cb for read direction

2023-07-27 Thread zhenwei pi
Operations on a cryptodev are considered as *write* only, the callback of read direction is never invoked. Use NULL instead of an unreachable path(cryptodev_backend_throttle_timer_cb on read direction). The dummy read timer(never invoked) is already removed here, it means that the 'FIXME' tag is

[PATCH v5 7/9] throttle: use THROTTLE_MAX/ARRAY_SIZE for hard code

2023-07-27 Thread zhenwei pi
The first dimension of both to_check and bucket_types_size/bucket_types_units is used as throttle direction, use THROTTLE_MAX instead of hard coded number. Also use ARRAY_SIZE() to avoid hard coded number for the second dimension. Hanna noticed that the two array should be static. Yes, turn them

[PATCH v5 8/9] fsdev: Use ThrottleDirection instread of bool is_write

2023-07-27 Thread zhenwei pi
'bool is_write' style is obsolete from throttle framework, adapt fsdev to the new style. Cc: Greg Kurz Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- fsdev/qemu-fsdev-throttle.c | 14 +++--- fsdev/qemu-fsdev-throttle.h | 4 ++-- hw/9pfs/cofile.c| 4 ++-- 3

Re: Re: [PATCH v4 9/9] block/throttle-groups: Use ThrottleDirection instread of bool is_write

2023-07-27 Thread zhenwei pi
On 7/28/23 00:12, Hanna Czenczek wrote: On 24.07.23 12:09, zhenwei pi wrote: 'bool is_write' style is obsolete from throttle framework, adapt block throttle groups to the new style. Use a simple python script to test the new style:   #!/usr/bin/python3 import subprocess import random import

[PATCH v5 6/9] throttle: use enum ThrottleDirection instead of bool is_write

2023-07-27 Thread zhenwei pi
enum ThrottleDirection is already there, use ThrottleDirection instead of 'bool is_write' for throttle API, also modify related codes from block, fsdev, cryptodev and tests. Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- backends/cryptodev.c| 9 +

[PATCH v5 9/9] block/throttle-groups: Use ThrottleDirection instread of bool is_write

2023-07-27 Thread zhenwei pi
'bool is_write' style is obsolete from throttle framework, adapt block throttle groups to the new style: - use ThrottleDirection instead of 'bool is_write'. Ex, schedule_next_request(ThrottleGroupMember *tgm, bool is_write) -> schedule_next_request(ThrottleGroupMember *tgm, ThrottleDirection

[PATCH v5 2/9] test-throttle: use enum ThrottleDirection

2023-07-27 Thread zhenwei pi
Use enum ThrottleDirection instead in the throttle test codes. Reviewed-by: Alberto Garcia Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- tests/unit/test-throttle.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/unit/test-throttle.c

[PATCH v5 1/9] throttle: introduce enum ThrottleDirection

2023-07-27 Thread zhenwei pi
Use enum ThrottleDirection instead of number index. Reviewed-by: Alberto Garcia Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- include/qemu/throttle.h | 11 --- util/throttle.c | 16 +--- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git

[PATCH v5 0/9] Misc fixes for throttle

2023-07-27 Thread zhenwei pi
v4 -> v5: - typo fix in 'throttle: use enum ThrottleDirection instead of bool is_write': '@throttle: throttle direction' -> '@direction: throttle direction' - several 'for (i = 0; i < 2; i++)' left, use 'for (dir = THROTTLE_READ; dir < THROTTLE_MAX; dir++)' instead. 'true/false' left,

[PATCH v5 3/9] throttle: support read-only and write-only

2023-07-27 Thread zhenwei pi
Only one direction is necessary in several scenarios: - a read-only disk - operations on a device are considered as *write* only. For example, encrypt/decrypt/sign/verify operations on a cryptodev use a single *write* timer(read timer callback is defined, but never invoked). Allow a single

Re: [PATCH] vhost-user-scsi: support reconnect to backend

2023-07-27 Thread Raphael Norwitz
> On Jul 25, 2023, at 6:19 AM, Li Feng wrote: > > Thanks for your comments. > >> 2023年7月25日 上午1:21,Raphael Norwitz 写道: >> >> Very excited to see this. High level looks good modulo a few small things. >> >> My major concern is around existing vhost-user-scsi backends which don’t >> support

[PULL for-8.1 6/8] block/blkio: retry blkio_connect() if it fails using `fd`

2023-07-27 Thread Stefan Hajnoczi
From: Stefano Garzarella libblkio 1.3.0 added support of "fd" property for virtio-blk-vhost-vdpa driver. In QEMU, starting from commit cad2ccc395 ("block/blkio: use qemu_open() to support fd passing for virtio-blk") we are using `blkio_get_int(..., "fd")` to check if the "fd" property is

[PULL for-8.1 5/8] block/blkio: move blkio_connect() in the drivers functions

2023-07-27 Thread Stefan Hajnoczi
From: Stefano Garzarella This is in preparation for the next patch, where for virtio-blk drivers we need to handle the failure of blkio_connect(). Let's also rename the *_open() functions to *_connect() to make the code reflect the changes applied. Signed-off-by: Stefano Garzarella

Re: [PATCH v2 0/4] block/blkio: fix opening virtio-blk drivers

2023-07-27 Thread Stefan Hajnoczi
On Thu, Jul 27, 2023 at 06:10:16PM +0200, Stefano Garzarella wrote: > There is a problem with virtio-blk-vhost-vdpa. > The first patch does some preparation changes. The second and third patches > fix the issues, the last patch tries to prepare QEMU for a future version of > libblkio where we can

[PULL for-8.1 0/8] Block patches

2023-07-27 Thread Stefan Hajnoczi
The following changes since commit 66547f416a61e0cb711dc76821890242432ba193: block/nvme: invoke blk_io_plug_call() outside q->lock (2023-07-17 09:17:41 -0400) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git tags/block-pull-request for you to fetch changes up

[PULL for-8.1 1/8] block/blkio: enable the completion eventfd

2023-07-27 Thread Stefan Hajnoczi
From: Stefano Garzarella Until libblkio 1.3.0, virtio-blk drivers had completion eventfd notifications enabled from the start, but from the next releases this is no longer the case, so we have to explicitly enable them. In fact, the libblkio documentation says they could be disabled, so we

[PULL for-8.1 2/8] block/blkio: do not use open flags in qemu_open()

2023-07-27 Thread Stefan Hajnoczi
From: Stefano Garzarella qemu_open() in blkio_virtio_blk_common_open() is used to open the character device (e.g. /dev/vhost-vdpa-0 or /dev/vfio/vfio) or in the future eventually the unix socket. In all these cases we cannot open the path in read-only mode, when the `read-only` option of

[PULL for-8.1 8/8] block/blkio: use blkio_set_int("fd") to check fd support

2023-07-27 Thread Stefan Hajnoczi
From: Stefano Garzarella Setting the `fd` property fails with virtio-blk-* libblkio drivers that do not support fd passing since https://gitlab.com/libblkio/libblkio/-/merge_requests/208. Getting the `fd` property, on the other hand, always succeeds for virtio-blk-* libblkio drivers even when

[PULL for-8.1 3/8] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Stefan Hajnoczi
From: Sam Li The g_file_get_contents() function returns a g_boolean. If it fails, the returned value will be 0 instead of -1. Solve the issue by skipping assigning ret value. This issue was found by Matthew Rosato using virtio-blk-{pci,ccw} backed by an NVMe partition e.g. /dev/nvme0n1p1 on

[PULL for-8.1 7/8] block/blkio: fall back on using `path` when `fd` setting fails

2023-07-27 Thread Stefan Hajnoczi
From: Stefano Garzarella qemu_open() fails if called with an unix domain socket in this way: -blockdev node-name=drive0,driver=virtio-blk-vhost-user,path=vhost-user-blk.sock,cache.direct=on: Could not open 'vhost-user-blk.sock': No such device or address Since virtio-blk-vhost-user does

[PULL for-8.1 4/8] block: Fix pad_request's request restriction

2023-07-27 Thread Stefan Hajnoczi
From: Hanna Czenczek bdrv_pad_request() relies on requests' lengths not to exceed SIZE_MAX, which bdrv_check_qiov_request() does not guarantee. bdrv_check_request32() however will guarantee this, and both of bdrv_pad_request()'s callers (bdrv_co_preadv_part() and bdrv_co_pwritev_part()) already

Re: [PATCH v4] block-jobs: flush target at the end of .run()

2023-07-27 Thread Vladimir Sementsov-Ogievskiy
On 27.07.23 18:27, Hanna Czenczek wrote: On 25.07.23 19:40, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Actually block job is not completed without this final flush. It's rather unexpected to have broken target when job was successfully completed long ago and now we

Re: [PATCH v4 9/9] block/throttle-groups: Use ThrottleDirection instread of bool is_write

2023-07-27 Thread Hanna Czenczek
On 24.07.23 12:09, zhenwei pi wrote: 'bool is_write' style is obsolete from throttle framework, adapt block throttle groups to the new style. Use a simple python script to test the new style: #!/usr/bin/python3 import subprocess import random import time commands = ['virsh blkdeviotune jammy

[PATCH v2 1/4] block/blkio: move blkio_connect() in the drivers functions

2023-07-27 Thread Stefano Garzarella
This is in preparation for the next patch, where for virtio-blk drivers we need to handle the failure of blkio_connect(). Let's also rename the *_open() functions to *_connect() to make the code reflect the changes applied. Signed-off-by: Stefano Garzarella --- block/blkio.c | 67

[PATCH v2 0/4] block/blkio: fix opening virtio-blk drivers

2023-07-27 Thread Stefano Garzarella
There is a problem with virtio-blk-vhost-vdpa. The first patch does some preparation changes. The second and third patches fix the issues, the last patch tries to prepare QEMU for a future version of libblkio where we can use blkio_set_fd() to check whether the property is supported or not. While

Re: [PATCH v8 0/4] hw/ufs: Add Universal Flash Storage (UFS) support

2023-07-27 Thread Stefan Hajnoczi
On Thu, Jul 27, 2023 at 05:45:17PM +0900, Jeuk Kim wrote: > Since v7: > In ufs-test.c, make the following changes > - Change TIMEOUT from 5 to 10 (Thomas's review comment) > - Rename the temporary file to "qemu-ufs." (Thomas's review comment) > - Use "-blockdev" instead of "-drive" (Stefan's

Re: [PATCH v4 7/9] throttle: use THROTTLE_MAX/ARRAY_SIZE for hard code

2023-07-27 Thread Hanna Czenczek
On 24.07.23 12:09, zhenwei pi wrote: The first dimension of both to_check and bucket_types_size/bucket_types_units is used as throttle direction, use THROTTLE_MAX instead of hard coded number. Also use ARRAY_SIZE() to avoid hard coded number for the second dimension. Hanna noticed that the two

[PATCH v2 4/4] block/blkio: use blkio_set_int("fd") to check fd support

2023-07-27 Thread Stefano Garzarella
Setting the `fd` property fails with virtio-blk-* libblkio drivers that do not support fd passing since https://gitlab.com/libblkio/libblkio/-/merge_requests/208. Getting the `fd` property, on the other hand, always succeeds for virtio-blk-* libblkio drivers even when they don't support fd

[PATCH v2 3/4] block/blkio: fall back on using `path` when `fd` setting fails

2023-07-27 Thread Stefano Garzarella
qemu_open() fails if called with an unix domain socket in this way: -blockdev node-name=drive0,driver=virtio-blk-vhost-user,path=vhost-user-blk.sock,cache.direct=on: Could not open 'vhost-user-blk.sock': No such device or address Since virtio-blk-vhost-user does not support fd passing,

Re: [PATCH v4 6/9] throttle: use enum ThrottleDirection instead of bool is_write

2023-07-27 Thread Hanna Czenczek
On 24.07.23 12:09, zhenwei pi wrote: enum ThrottleDirection is already there, use ThrottleDirection instead of 'bool is_write' for throttle API, also modify related codes from block, fsdev, cryptodev and tests. Signed-off-by: zhenwei pi --- backends/cryptodev.c| 9 +

[PATCH v2 2/4] block/blkio: retry blkio_connect() if it fails using `fd`

2023-07-27 Thread Stefano Garzarella
libblkio 1.3.0 added support of "fd" property for virtio-blk-vhost-vdpa driver. In QEMU, starting from commit cad2ccc395 ("block/blkio: use qemu_open() to support fd passing for virtio-blk") we are using `blkio_get_int(..., "fd")` to check if the "fd" property is supported for all the virtio-blk-*

Re: [PATCH v8 0/4] hw/ufs: Add Universal Flash Storage (UFS) support

2023-07-27 Thread Stefan Hajnoczi
On Thu, Jul 27, 2023 at 05:45:17PM +0900, Jeuk Kim wrote: > Since v7: > In ufs-test.c, make the following changes > - Change TIMEOUT from 5 to 10 (Thomas's review comment) > - Rename the temporary file to "qemu-ufs." (Thomas's review comment) > - Use "-blockdev" instead of "-drive" (Stefan's

Re: Open qcow2 on multiple hosts simultaneously.

2023-07-27 Thread Stefan Hajnoczi
On Wed, Jun 21, 2023 at 02:16:30PM +0200, kvaps wrote: > > QCOW2 caches two forms of data, cluster metadata (L1/L2 data, refcount > > table, etc) and mutable header information (file size, snapshot > > entries, etc). This data is discarded after the last piece of incoming > > migration data is

Re: [PATCH v4 8/9] fsdev: Use ThrottleDirection instread of bool is_write

2023-07-27 Thread Hanna Czenczek
On 24.07.23 12:09, zhenwei pi wrote: 'bool is_write' style is obsolete from throttle framework, adapt fsdev to the new style. Cc: Greg Kurz Signed-off-by: zhenwei pi --- fsdev/qemu-fsdev-throttle.c | 14 +++--- fsdev/qemu-fsdev-throttle.h | 4 ++-- hw/9pfs/cofile.c|

Re: [PATCH v4] block-jobs: flush target at the end of .run()

2023-07-27 Thread Hanna Czenczek
On 25.07.23 19:40, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Actually block job is not completed without this final flush. It's rather unexpected to have broken target when job was successfully completed long ago and now we fail to flush or process just

[PATCH] block/iscsi: Document why we use raw malloc()

2023-07-27 Thread Peter Maydell
In block/iscsi.c we use a raw malloc() call, which is unusual given the project standard is to use the glib memory allocation functions. Document why we do so, to avoid it being converted to g_malloc() by mistake. Signed-off-by: Peter Maydell --- There aren't many uses of raw malloc() in the

Re: [PATCH] block/iscsi: Document why we use raw malloc()

2023-07-27 Thread Peter Maydell
On Thu, 27 Jul 2023 at 16:07, Peter Maydell wrote: > > In block/iscsi.c we use a raw malloc() call, which is unusual > given the project standard is to use the glib memory allocation > functions. Document why we do so, to avoid it being converted > to g_malloc() by mistake. > > Signed-off-by:

Re: [PATCH v2 1/1] qemu-nbd: regression with arguments passing into nbd_client_thread()

2023-07-27 Thread Eric Blake
On Thu, Jul 27, 2023 at 12:58:28PM +0200, Denis V. Lunev wrote: > Unfortunately > commit 03b67621445d601c9cdc7dfe25812e9f19b81488 > Author: Denis V. Lunev > Date: Mon Jul 17 16:55:40 2023 +0200 > qemu-nbd: pass structure into nbd_client_thread instead of plain char* > has

Re: [PATCH v2] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Stefan Hajnoczi
On Thu, Jul 27, 2023 at 01:15:32PM +0100, Daniel P. Berrangé wrote: > On Thu, Jul 27, 2023 at 07:58:44PM +0800, Sam Li wrote: > > The g_file_get_contents() function returns a g_boolean. If it fails, the > > returned value will be 0 instead of -1. Solve the issue by skipping > > assigning ret

Re: [PATCH v2] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Stefan Hajnoczi
On Thu, Jul 27, 2023 at 07:58:44PM +0800, Sam Li wrote: > The g_file_get_contents() function returns a g_boolean. If it fails, the > returned value will be 0 instead of -1. Solve the issue by skipping > assigning ret value. > > This issue was found by Matthew Rosato using virtio-blk-{pci,ccw}

Re: [PATCH v2 0/6] python/machine: use socketpair() for console socket

2023-07-27 Thread Peter Maydell
On Tue, 25 Jul 2023 at 19:04, John Snow wrote: > > Like we did for the QMP socket, use socketpair() for the console socket > so that hopefully there isn't a race condition during early boot where > data might get dropped on the floor. > > May or may not help with various race conditions where

Re: Lost partition tables on ide-hd + ahci drive

2023-07-27 Thread Simon Rowe
On Wednesday, 14 June 2023 Simon Rowe wrote: > We've also seen a handful of similar reports. Again, just the MBR sector > overwritten by what looks to be guest data (e.g. log messages). The > common thread with our incidents is again a SATA disk under the AHCI > controller, we have a network

Re: [PATCH 1/2] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Sam Li
Matthew Rosato 于2023年7月27日周四 19:46写道: > > On 7/5/23 10:54 AM, Matthew Rosato wrote: > > On 6/4/23 2:16 AM, Sam Li wrote: > >> The g_file_get_contents() function returns a g_boolean. If it fails, the > >> returned value will be 0 instead of -1. Solve the issue by skipping > >> assigning ret value.

Re: [PATCH v2] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Daniel P . Berrangé
On Thu, Jul 27, 2023 at 07:58:44PM +0800, Sam Li wrote: > The g_file_get_contents() function returns a g_boolean. If it fails, the > returned value will be 0 instead of -1. Solve the issue by skipping > assigning ret value. > > This issue was found by Matthew Rosato using virtio-blk-{pci,ccw}

[PATCH v2] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Sam Li
The g_file_get_contents() function returns a g_boolean. If it fails, the returned value will be 0 instead of -1. Solve the issue by skipping assigning ret value. This issue was found by Matthew Rosato using virtio-blk-{pci,ccw} backed by an NVMe partition e.g. /dev/nvme0n1p1 on s390x.

Re: [PATCH 1/2] block/file-posix: fix g_file_get_contents return path

2023-07-27 Thread Matthew Rosato
On 7/5/23 10:54 AM, Matthew Rosato wrote: > On 6/4/23 2:16 AM, Sam Li wrote: >> The g_file_get_contents() function returns a g_boolean. If it fails, the >> returned value will be 0 instead of -1. Solve the issue by skipping >> assigning ret value. >> >> This issue was found by Matthew Rosato using

[PATCH v2 1/1] qemu-nbd: regression with arguments passing into nbd_client_thread()

2023-07-27 Thread Denis V. Lunev
Unfortunately commit 03b67621445d601c9cdc7dfe25812e9f19b81488 Author: Denis V. Lunev Date: Mon Jul 17 16:55:40 2023 +0200 qemu-nbd: pass structure into nbd_client_thread instead of plain char* has introduced a regression. struct NbdClientOpts resides on stack inside 'if' block.

Re: [PATCH 1/1] qemu-nbd: regression with arguments passing into nbd_client_thread()

2023-07-27 Thread Denis V. Lunev
On 7/26/23 19:57, Eric Blake wrote: On Wed, Jul 26, 2023 at 04:52:47PM +0200, Denis V. Lunev wrote: Unfortunately commit 03b67621445d601c9cdc7dfe25812e9f19b81488 Author: Denis V. Lunev Date: Mon Jul 17 16:55:40 2023 +0200 qemu-nbd: pass structure into nbd_client_thread

Re: [PATCH v2 4/6] python/machine: use socketpair() for console connections

2023-07-27 Thread Daniel P . Berrangé
On Thu, Jul 27, 2023 at 11:22:35AM +0530, Ani Sinha wrote: > > > > On 26-Jul-2023, at 10:51 PM, John Snow wrote: > > > > > > > > On Wed, Jul 26, 2023, 6:50 AM Ani Sinha wrote: > > > > > > > On 25-Jul-2023, at 11:33 PM, John Snow wrote: > > > > > > Create a socketpair for the console

[PATCH v8 4/4] tests/qtest: Introduce tests for UFS

2023-07-27 Thread Jeuk Kim
This patch includes the following tests Test mmio read Test ufs device initialization and ufs-lu recognition Test I/O (Performs a write followed by a read to verify) Signed-off-by: Jeuk Kim Acked-by: Thomas Huth Reviewed-by: Stefan Hajnoczi --- MAINTAINERS | 1 +

[PATCH v8 1/4] hw/ufs: Initial commit for emulated Universal-Flash-Storage

2023-07-27 Thread Jeuk Kim
Universal Flash Storage (UFS) is a high-performance mass storage device with a serial interface. It is primarily used as a high-performance data storage device for embedded applications. This commit contains code for UFS device to be recognized as a UFS PCI device. Patches to handle UFS logical

[PATCH v8 0/4] hw/ufs: Add Universal Flash Storage (UFS) support

2023-07-27 Thread Jeuk Kim
Since v7: In ufs-test.c, make the following changes - Change TIMEOUT from 5 to 10 (Thomas's review comment) - Rename the temporary file to "qemu-ufs." (Thomas's review comment) - Use "-blockdev" instead of "-drive" (Stefan's review comment) Since v6: - Add tests/qtest/ufs-test.c to test ufs

[PATCH v8 3/4] hw/ufs: Support for UFS logical unit

2023-07-27 Thread Jeuk Kim
This commit adds support for ufs logical unit. The LU handles processing for the SCSI command, unit descriptor query request. This commit enables the UFS device to process IO requests. Signed-off-by: Jeuk Kim Reviewed-by: Stefan Hajnoczi --- hw/ufs/lu.c | 1445

[PATCH v8 2/4] hw/ufs: Support for Query Transfer Requests

2023-07-27 Thread Jeuk Kim
This commit makes the UFS device support query and nop out transfer requests. The next patch would be support for UFS logical unit and scsi command transfer request. Signed-off-by: Jeuk Kim Reviewed-by: Stefan Hajnoczi --- hw/ufs/trace-events | 1 + hw/ufs/ufs.c| 980

Re: [PATCH v2 4/6] python/machine: use socketpair() for console connections

2023-07-27 Thread Ani Sinha
> On 27-Jul-2023, at 11:22 AM, Ani Sinha wrote: > > > >> On 26-Jul-2023, at 10:51 PM, John Snow wrote: >> >> >> >> On Wed, Jul 26, 2023, 6:50 AM Ani Sinha wrote: >> >> >>> On 25-Jul-2023, at 11:33 PM, John Snow wrote: >>> >>> Create a socketpair for the console output. This should

Re: [PATCH v2 4/6] python/machine: use socketpair() for console connections

2023-07-27 Thread Ani Sinha
> On 26-Jul-2023, at 10:51 PM, John Snow wrote: > > > > On Wed, Jul 26, 2023, 6:50 AM Ani Sinha wrote: > > > > On 25-Jul-2023, at 11:33 PM, John Snow wrote: > > > > Create a socketpair for the console output. This should help eliminate > > race conditions around console text early in