Re: [PATCH 0/2] linux-aio: fix unbalanced plugged counter in laio_io_unplug()

Thank you for finding this and fixing it. This issue has been giving us grief for months, and this patch appears to resolve the problem. In our case, it seemed to have much greater severity with the RHEL / CentOS 7.x Linux 3.10 kernel when tied to SolidFire iSCSI based storage. This caused it to

Re: [PATCH 5/9] error-report: introduce ErrorReportDetailedFunc

On Thu, Jun 16, 2022 at 6:41 AM wrote: > From: Marc-André Lureau > > Remove monitor dependency from error printing code, by allowing programs > to set a callback for when to use "detailed" reporting or not. > > Signed-off-by: Marc-André Lureau > --- > include/qemu/error-report.h | 4

Re: [PATCH 4/8] virtio: categorize callbacks in GS

On Thu, Jun 09, 2022 at 10:37:23AM -0400, Emanuele Giuseppe Esposito wrote: > All the callbacks below are always running in the main loop. > > The callbacks are the following: > - start/stop_ioeventfd: these are the callbacks where > blk_set_aio_context(iothread) is done, so they are called in

Re: [PATCH 01/20] io: add a QIOChannelNull equivalent to /dev/null

On Tue, May 24, 2022 at 04:14:31PM -0500, Eric Blake wrote: > On Tue, May 24, 2022 at 12:02:16PM +0100, Daniel P. Berrangé wrote: > > This is for code which needs a portable equivalent to a QIOChannelFile > > connected to /dev/null. > > > > Signed-off-by: Daniel P. Berrangé > > --- > >

Re: [PATCH 0/4] hw/nvme: add support for TP4084

On Jun 16 07:57, Keith Busch wrote: > On Thu, Jun 16, 2022 at 12:42:49PM +0200, Klaus Jensen wrote: > > On Jun 8 03:28, Niklas Cassel via wrote: > > > Hello there, > > > > > > considering that Linux v5.19-rc1 is out which includes support for > > > NVMe TP4084: > > >

Re: [PATCH v2 03/10] qga: treat get-guest-fsinfo as "best effort"

Hi On Thu, Jun 16, 2022 at 6:27 PM John Snow wrote: > In some container environments, there may be references to block devices > witnessable from a container through /proc/self/mountinfo that reference > devices we simply don't have access to in the container, and could not > provide

[PATCH v2 08/10] tests/vm: remove ubuntu.i386 VM test

Ubuntu 18.04 is out of our support window, and Ubuntu 20.04 does not support i386 anymore. The debian project does, but they do not provide any cloud images for it, a new expect-style script would have to be written. Since we have i386 cross-compiler tests hosted on GitLab CI, we don't need to

Re: [PATCH v2 03/10] qga: treat get-guest-fsinfo as "best effort"

On Thu, Jun 16, 2022 at 10:36 AM Marc-André Lureau wrote: > > Hi > > On Thu, Jun 16, 2022 at 6:27 PM John Snow wrote: >> >> In some container environments, there may be references to block devices >> witnessable from a container through /proc/self/mountinfo that reference >> devices we simply

[PATCH v2 10/10] tests/vm: add 512MB extra memory per core

If you try to run a 16 or 32 threaded test, you're going to run out of memory very quickly with qom-test and a few others. Bump the memory limit to try to scale with larger-core machines. Add this magic in early to the configuration process so that the config file, if provided, can still override

[PATCH v2 09/10] tests/vm: remove duplicate 'centos' VM test

This is listed twice by accident; we require genisoimage to run the test, so remove the unconditional entry. Signed-off-by: John Snow --- tests/vm/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index

[PATCH v2 05/10] tests/vm: switch CentOS 8 to CentOS 8 Stream

The old CentOS image didn't work anymore because it was already EOL at the beginning of 2022. Signed-off-by: John Snow Reviewed-by: Thomas Huth --- tests/vm/centos | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/vm/centos b/tests/vm/centos index

[PATCH v2 07/10] tests/vm: update sha256sum for ubuntu.aarch64

This checksum changes weekly; use a fixed point image and update the checksum so we don't have to re-download it quite so much. Note: Just like the centos.aarch64 test, this test currently seems very flaky when run as a TCG test. Signed-off-by: John Snow --- tests/vm/ubuntu.aarch64 | 8

[PATCH v2 02/10] tests/qemu-iotests: skip 108 when FUSE is not loaded

In certain container environments we may not have FUSE at all, so skip the test in this circumstance too. Signed-off-by: John Snow --- tests/qemu-iotests/108 | 5 + 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108 index 9e923d6a59f..54e935acf28

[PATCH v2 00/10] Improve reliability of VM tests

This patch series attempts to improve the reliability of several of the VM test targets. In particular, both CentOS 8 tests are non-functional because CentOS 8 was EOL at the beginning of this calendar year, with repositories and mirrors going offline. Notably, I also remove the ubuntu.i386 test

[PATCH v2 03/10] qga: treat get-guest-fsinfo as "best effort"

In some container environments, there may be references to block devices witnessable from a container through /proc/self/mountinfo that reference devices we simply don't have access to in the container, and could not provide information about. Instead of failing the entire fsinfo command, return

[PATCH v2 01/10] tests/qemu-iotests: hotfix for 307, 223 output

Fixes: 58a6fdcc Signed-off-by: John Snow Tested-by: Daniel P. Berrangé Reviewed-by: Daniel P. Berrangé --- tests/qemu-iotests/223.out | 4 ++-- tests/qemu-iotests/307.out | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/223.out

[PATCH v2 04/10] tests/vm: use 'cp' instead of 'ln' for temporary vm images

If the initial setup fails, you've permanently altered the state of the downloaded image in an unknowable way. Use 'cp' like our other test setup scripts do. Signed-off-by: John Snow Reviewed-by: Thomas Huth --- tests/vm/centos | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: [PATCH 0/4] hw/nvme: add support for TP4084

On Thu, Jun 16, 2022 at 12:42:49PM +0200, Klaus Jensen wrote: > On Jun 8 03:28, Niklas Cassel via wrote: > > Hello there, > > > > considering that Linux v5.19-rc1 is out which includes support for > > NVMe TP4084: > >

Re: [PATCH 0/9] Preliminary patches for subproject split

Hi On Thu, Jun 16, 2022 at 5:12 PM Paolo Bonzini wrote: > > On 6/16/22 14:40, marcandre.lur...@redhat.com wrote: > > From: Marc-André Lureau > > > > Hi, > > > > Here is another subset of the large "subproject(qga)"" series I intend to > > send > > soon after

[PATCH v7 11/18] job.h: rename job API functions called with job_mutex held

With the *nop* job_lock/unlock placed, rename the job functions of the job API that are always under job_mutex, adding "_locked" suffix. List of functions that get this suffix: job_txn_unref job_txn_add_job job_ref job_unref job_enter_cond job_finish_sync

[PATCH v7 08/18] jobs: use job locks also in the unit tests

Add missing job synchronization in the unit tests, with explicit locks. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 76 - tests/unit/test-block-iothread.c |

[PATCH v7 15/18] job: detect change of aiocontext within job coroutine

From: Paolo Bonzini We want to make sure access of job->aio_context is always done under either BQL or job_mutex. The problem is that using aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond makes the coroutine immediately resume, so we can't hold the job lock. And caching it

[PATCH v7 12/18] block_job: rename block_job functions called with job_mutex held

Just as for the job API, rename block_job functions that are always called under job lock. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 3 ++- block/backup.c | 4 ++-- blockdev.c | 12 +++- blockjob.c

[PATCH v7 13/18] job.h: define unlocked functions

All these functions assume that the lock is not held, and acquire it internally. These functions will be useful when job_lock is globally applied, as they will allow callers to access the job struct fields without worrying about the job lock. Update also the comments in blockjob.c (and move them

[PATCH v7 00/18] job: replace AioContext lock with job_mutex

In this series, we want to remove the AioContext lock and instead use the already existent job_mutex to protect the job structures and list. This is part of the work to get rid of AioContext lock usage in favour of smaller granularity locks. In order to simplify reviewer's job, job lock/unlock

[PATCH v7 04/18] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- include/block/aio-wait.h | 17

[PATCH v7 16/18] jobs: protect job.aio_context with BQL and job_mutex

In order to make it thread safe, implement a "fake rwlock", where we allow reads under BQL *or* job_mutex held, but writes only under BQL *and* job_mutex. The only write we have is in child_job_set_aio_ctx, which always happens under drain (so the job is paused). For this reason, introduce

[PATCH v7 03/18] job.c: API functions not used outside should be static

job_event_* functions can all be static, as they are not used outside job.c. Same applies for job_txn_add_job(). Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 18 -- job.c | 22 +++--- 2 files

[PATCH v7 01/18] job.c: make job_mutex and job_lock/unlock() public

job mutex will be used to protect the job struct elements and list, replacing AioContext locks. Right now use a shared lock for all jobs, in order to keep things simple. Once the AioContext lock is gone, we can introduce per-job locks. To simplify the switch from aiocontext to job lock,

[PATCH v7 07/18] jobs: add job lock in find_* functions

Both blockdev.c and job-qmp.c have TOC/TOU conditions, because they first search for the job and then perform an action on it. Therefore, we need to do the search + action under the same job mutex critical section. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*.

[PATCH v7 10/18] jobs: rename static functions called with job_mutex held

With the *nop* job_lock/unlock placed, rename the static functions that are always under job_mutex, adding "_locked" suffix. List of functions that get this suffix: job_txn_refjob_txn_del_job job_txn_apply job_state_transition job_should_pause

[PATCH v7 05/18] job.h: add _locked duplicates for job API functions called with and without job_mutex

In preparation to the job_lock/unlock usage, create _locked duplicates of some functions, since they will be sometimes called with job_mutex held (mostly within job.c), and sometimes without (mostly from JobDrivers using the job API). Therefore create a _locked version of such function, so that

[PATCH v7 18/18] block_job_query: remove atomic read

Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function is called under job_mutex, just remove the atomic. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 2 +- 1 file changed, 1

[PATCH v7 14/18] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

We are always using the given bs AioContext, so there is no need to take the job ones (which is identical anyways). This also reduces the point we need to check when protecting job.aio_context field. Reviewed-by: Stefan Hajnoczi Signed-off-by: Emanuele Giuseppe Esposito --- block/commit.c | 4

[PATCH v7 17/18] job.c: enable job lock/unlock and remove Aiocontext locks

Change the job_{lock/unlock} and macros to use job_mutex. Now that they are not nop anymore, remove the aiocontext to avoid deadlocks. Therefore: - when possible, remove completely the aiocontext lock/unlock pair - if it is used by some other function too, reduce the locking section as much as

[PATCH v7 09/18] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, using the helpers prepared in previous commit. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito

[PATCH v7 06/18] jobs: protect jobs with job_lock/unlock

Introduce the job locking mechanism through the whole job API, following the comments in job.h and requirements of job-monitor (like the functions in job-qmp.c, assume lock is held) and job-driver (like in mirror.c and all other JobDriver, lock is not held). Use the _locked helpers introduced

[PATCH v7 02/18] job.h: categorize fields in struct Job

Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 61 +++--- 1 file changed, 36 insertions(+), 25 deletions(-) diff --git

Re: [PATCH 0/9] Preliminary patches for subproject split

On 6/16/22 14:40, marcandre.lur...@redhat.com wrote: From: Marc-André Lureau Hi, Here is another subset of the large "subproject(qga)"" series I intend to send soon after (https://gitlab.com/marcandre.lureau/qemu/-/commits/qga). Hi, I took a peek there and I have a question. While the

[PATCH 5/9] error-report: introduce ErrorReportDetailedFunc

From: Marc-André Lureau Remove monitor dependency from error printing code, by allowing programs to set a callback for when to use "detailed" reporting or not. Signed-off-by: Marc-André Lureau --- include/qemu/error-report.h | 4 +++- bsd-user/main.c | 2 +-

[PATCH 7/9] qapi: move QEMU-specific dispatch code in monitor

From: Marc-André Lureau Make QMP-dispatch code free from QEMU-specific OOB dispatch/async coroutine handling. This will allow to move the base code to qemu-common, and clear other users from potential mis-ususe (QGA doesn't have OOB or coroutine). To do that, introduce an optional callback

[PATCH 6/9] error-report: add a callback to overwrite error_vprintf

From: Marc-André Lureau error_vprintf() is implemented in monitor.c, which overrides the default implementation from stubs/, while avoiding a direct dependency to the monitor from error-report.c. However, the stub solution isn't working when moving error-report.c and stubs/error-printf.c in a

[PATCH 8/9] scripts/qapi-gen: add -i option

From: Marc-André Lureau Replace hard-coded "qemu/osdep.h" include with a qapi-gen option to specify the headers to include. This will allow to substitute QEMU osdep.h with glib.h for example, for projects with different global headers. For historical reasons, we can keep the default as

[PATCH 2/9] error-report: misc comment fix

From: Marc-André Lureau Signed-off-by: Marc-André Lureau --- util/error-report.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/error-report.c b/util/error-report.c index 5edb2e604061..98f242b75bbf 100644 --- a/util/error-report.c +++ b/util/error-report.c @@ -390,7

[PATCH 0/9] Preliminary patches for subproject split

From: Marc-André Lureau Hi, Here is another subset of the large "subproject(qga)"" series I intend to send soon after (https://gitlab.com/marcandre.lureau/qemu/-/commits/qga). Thanks Marc-André Lureau (9): monitor: make error_vprintf_unless_qmp() static error-report: misc comment fix

[PATCH 9/9] scripts/qapi: add required system includes to visitor

From: Marc-André Lureau The generated visitor code includes abort() & assert(), we shouldn't rely on the global "-i" headers to include the necessary system headers. Signed-off-by: Marc-André Lureau --- scripts/qapi/visit.py | 4 1 file changed, 4 insertions(+) diff --git

[PATCH 3/9] error-report: introduce "detailed" variable

From: Marc-André Lureau Let's use a more explicit variable "detailed" instead of calling monitor_cur() multiple times. Signed-off-by: Marc-André Lureau --- util/error-report.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/util/error-report.c b/util/error-report.c

[PATCH 4/9] error-report: simplify print_loc()

From: Marc-André Lureau Pass the program name as "prefix" argument to print_loc() if printing with "details". This allows to get rid of monitor_cur() call in print_loc(). Signed-off-by: Marc-André Lureau --- util/error-report.c | 8 1 file changed, 4 insertions(+), 4 deletions(-)

[PATCH 1/9] monitor: make error_vprintf_unless_qmp() static

From: Marc-André Lureau Not needed outside monitor.c. Remove the needless stub. Signed-off-by: Marc-André Lureau --- include/monitor/monitor.h | 1 - monitor/monitor.c | 3 ++- stubs/error-printf.c | 5 - 3 files changed, 2 insertions(+), 7 deletions(-) diff --git

Re: [PATCH 0/4] hw/nvme: add support for TP4084

On Jun 8 03:28, Niklas Cassel via wrote: > Hello there, > > considering that Linux v5.19-rc1 is out which includes support for > NVMe TP4084: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/nvme/host/core.c?id=354201c53e61e493017b15327294b0c8ab522d69 > > I

[PATCH] block/rbd: support driver-specific reopen

Hello everyone, what do you think? Tell me if something needs to be clarified or improved. Raphael OpenPGP_0xCDB1EBB785C5EB7E.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature