[RFC v3 6/8] stubs: add memory_region_from_host() and memory_region_get_fd()

2022-07-07 Thread Stefan Hajnoczi
The blkio block driver will need to look up the file descriptor for a given pointer. This is possible in softmmu builds where the memory API is available for querying guest RAM. Add stubs so tools like qemu-img that link the block layer still build successfully. In this case there is no guest RAM

[RFC v3 7/8] blkio: implement BDRV_REQ_REGISTERED_BUF optimization

2022-07-07 Thread Stefan Hajnoczi
Avoid bounce buffers when QEMUIOVector elements are within previously registered bdrv_register_buf() buffers. The idea is that emulated storage controllers will register guest RAM using bdrv_register_buf() and set the BDRV_REQ_REGISTERED_BUF on I/O requests. Therefore no blkio_map_mem_region()

[RFC v3 8/8] virtio-blk: use BDRV_REQ_REGISTERED_BUF optimization hint

2022-07-07 Thread Stefan Hajnoczi
Register guest RAM using BlockRAMRegistrar and set the BDRV_REQ_REGISTERED_BUF flag so block drivers can optimize memory accesses in I/O requests. This is for vdpa-blk, vhost-user-blk, and other I/O interfaces that rely on DMA mapping/unmapping. Signed-off-by: Stefan Hajnoczi ---

[RFC v3 5/8] block: add BlockRAMRegistrar

2022-07-07 Thread Stefan Hajnoczi
Emulated devices and other BlockBackend users wishing to take advantage of blk_register_buf() all have the same repetitive job: register RAMBlocks with the BlockBackend using RAMBlockNotifier. Add a BlockRAMRegistrar API to do this. A later commit will use this from hw/block/virtio-blk.c.

[RFC v3 4/8] block: add BDRV_REQ_REGISTERED_BUF request flag

2022-07-07 Thread Stefan Hajnoczi
Block drivers may optimize I/O requests accessing buffers previously registered with bdrv_register_buf(). Checking whether all elements of a request's QEMUIOVector are within previously registered buffers is expensive, so we need a hint from the user to avoid costly checks. Add a

[RFC v3 1/8] blkio: add io_uring block driver using libblkio

2022-07-07 Thread Stefan Hajnoczi
libblkio (https://gitlab.com/libblkio/libblkio/) is a library for high-performance disk I/O. It currently supports io_uring and virtio-blk-vhost-vdpa with additional drivers under development. One of the reasons for developing libblkio is that other applications besides QEMU can use it. This will

[RFC v3 3/8] block: pass size to bdrv_unregister_buf()

2022-07-07 Thread Stefan Hajnoczi
The only implementor of bdrv_register_buf() is block/nvme.c, where the size is not needed when unregistering a buffer. This is because util/vfio-helpers.c can look up mappings by address. Future block drivers that implement bdrv_register_buf() may not be able to do their job given only the buffer

[RFC v3 2/8] numa: call ->ram_block_removed() in ram_block_notifer_remove()

2022-07-07 Thread Stefan Hajnoczi
When a RAMBlockNotifier is added, ->ram_block_added() is called with all existing RAMBlocks. There is no equivalent ->ram_block_removed() call when a RAMBlockNotifier is removed. The util/vfio-helpers.c code (the sole user of RAMBlockNotifier) is fine with this asymmetry because it does not rely

[RFC v3 0/8] blkio: add libblkio BlockDriver

2022-07-07 Thread Stefan Hajnoczi
v3: - Add virtio-blk-vhost-vdpa for vdpa-blk devices including VDUSE - Add discard and write zeroes support - Rebase and adopt latest libblkio APIs v2: - Add BDRV_REQ_REGISTERED_BUF to bs.supported_write_flags [Stefano] - Use new blkioq_get_num_completions() API - Implement .bdrv_refresh_limits()

[PATCH] hw/block/hd-geometry: Do not override specified bios-chs-trans

2022-07-07 Thread Lev Kujawski
For small disk images (<4 GiB), QEMU and SeaBIOS default to the LARGE/ECHS disk translation method, but it is not uncommon for other BIOS software to use LBA in these cases as well. Some operating system boot loaders (e.g., NT 4) do not handle LARGE translations outside of fixed configurations.

Re: [PATCH 5/9] error-report: introduce ErrorReportDetailedFunc

2022-07-07 Thread Marc-André Lureau
Hi On Thu, Jul 7, 2022 at 4:13 PM Markus Armbruster wrote: > marcandre.lur...@redhat.com writes: > > > From: Marc-André Lureau > > > > Remove monitor dependency from error printing code, by allowing programs > > to set a callback for when to use "detailed" reporting or not. > > > >

Re: [PATCH 6/9] error-report: add a callback to overwrite error_vprintf

2022-07-07 Thread Marc-André Lureau
Hi On Thu, Jul 7, 2022 at 4:18 PM Markus Armbruster wrote: > marcandre.lur...@redhat.com writes: > > > From: Marc-André Lureau > > > > error_vprintf() is implemented in monitor.c, which overrides the > > default implementation from stubs/, while avoiding a direct dependency > > to the monitor

Re: [PATCH 1/9] monitor: make error_vprintf_unless_qmp() static

2022-07-07 Thread Marc-André Lureau
Hi On Thu, Jul 7, 2022 at 4:25 PM Markus Armbruster wrote: > marcandre.lur...@redhat.com writes: > > > From: Marc-André Lureau > > > > Not needed outside monitor.c. Remove the needless stub. > > > > Signed-off-by: Marc-André Lureau > > --- > > include/monitor/monitor.h | 1 - > >

Re: [PATCH v3 10/13] tests/vm: Remove docker cross-compile test from CentOS VM

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 11:48:35AM -0400, John Snow wrote: > On Thu, Jul 7, 2022 at 4:33 AM Daniel P. Berrangé wrote: > > > > On Thu, Jul 07, 2022 at 12:03:07AM -0400, John Snow wrote: > > > The fedora container has since been split apart, so there's no suitable > > > nearby target that would

Re: [PATCH v3 10/13] tests/vm: Remove docker cross-compile test from CentOS VM

2022-07-07 Thread John Snow
On Thu, Jul 7, 2022 at 4:33 AM Daniel P. Berrangé wrote: > > On Thu, Jul 07, 2022 at 12:03:07AM -0400, John Snow wrote: > > The fedora container has since been split apart, so there's no suitable > > nearby target that would support "test-mingw" as it requires both x32 > > and x64 support -- so

Re: [PATCH v3 09/13] tests/vm: upgrade Ubuntu 18.04 VM to 20.04

2022-07-07 Thread John Snow
On Thu, Jul 7, 2022 at 7:05 AM Richard Henderson wrote: > > On 7/7/22 09:33, John Snow wrote: > > 18.04 has fallen out of our support window, so move ubuntu.aarch64 > > forward to ubuntu 20.04, which is now our oldest supported Ubuntu > > release. > > Ah. Squash with patch 5? Can do. I left it

Re: [PATCH v3 01/13] qga: treat get-guest-fsinfo as "best effort"

2022-07-07 Thread John Snow
On Thu, Jul 7, 2022 at 4:40 AM Marc-André Lureau wrote: > > Hi > > On Thu, Jul 7, 2022 at 8:10 AM John Snow wrote: >> >> In some container environments, there may be references to block devices >> witnessable from a container through /proc/self/mountinfo that reference >> devices we simply don't

[QEMU 1/1] nvme: Fix misleading macro when mixed with ternary operator

2022-07-07 Thread Darren Kenny
Using the Parfait source code analyser and issue was found in hw/nvme/ctrl.c where the macros NVME_CAP_SET_CMBS and NVME_CAP_SET_PMRS are called with a ternary operatore in the second parameter, resulting in a potentially unexpected expansion of the form: x ? a: b & FLAG_TEST which will result

Re: [PATCH 1/9] monitor: make error_vprintf_unless_qmp() static

2022-07-07 Thread Markus Armbruster
marcandre.lur...@redhat.com writes: > From: Marc-André Lureau > > Not needed outside monitor.c. Remove the needless stub. > > Signed-off-by: Marc-André Lureau > --- > include/monitor/monitor.h | 1 - > monitor/monitor.c | 3 ++- > stubs/error-printf.c | 5 - > 3 files changed,

Re: [PATCH 6/9] error-report: add a callback to overwrite error_vprintf

2022-07-07 Thread Markus Armbruster
marcandre.lur...@redhat.com writes: > From: Marc-André Lureau > > error_vprintf() is implemented in monitor.c, which overrides the > default implementation from stubs/, while avoiding a direct dependency > to the monitor from error-report.c. > > However, the stub solution isn't working when

Re: [PATCH 5/9] error-report: introduce ErrorReportDetailedFunc

2022-07-07 Thread Markus Armbruster
marcandre.lur...@redhat.com writes: > From: Marc-André Lureau > > Remove monitor dependency from error printing code, by allowing programs > to set a callback for when to use "detailed" reporting or not. > > Signed-off-by: Marc-André Lureau > --- > include/qemu/error-report.h | 4 +++-

Re: [PATCH v3 09/13] tests/vm: upgrade Ubuntu 18.04 VM to 20.04

2022-07-07 Thread Richard Henderson
On 7/7/22 09:33, John Snow wrote: 18.04 has fallen out of our support window, so move ubuntu.aarch64 forward to ubuntu 20.04, which is now our oldest supported Ubuntu release. Ah. Squash with patch 5? r~ Signed-off-by: John Snow --- tests/vm/ubuntu.aarch64 | 14 +++--- 1 file

Re: [PATCH v3 05/13] tests/vm: update sha256sum for ubuntu.aarch64

2022-07-07 Thread Richard Henderson
On 7/7/22 09:33, John Snow wrote: This checksum changes weekly; use a fixed point image and update the checksum so we don't have to re-download it quite so much. Note: Just like the centos.aarch64 test, this test currently seems very flaky when run as a TCG test. Signed-off-by: John Snow ---

Re: [PULL 0/2] Block patches

2022-07-07 Thread Richard Henderson
On 7/7/22 13:42, Stefan Hajnoczi wrote: The following changes since commit 8e9398e3b1a860b8c29c670c1b6c36afe8d87849: Merge tag 'pull-ppc-20220706' of https://gitlab.com/danielhb/qemu into staging (2022-07-07 06:21:05 +0530) are available in the Git repository at:

Re: [PATCH v2] hw: m25p80: Add Block Protect and Top Bottom bits for write protect

2022-07-07 Thread Francisco Iglesias
Hi Iris On [2022 Jul 06] Wed 19:16:26, Iris Chen wrote: > Signed-off-by: Iris Chen A couple of suggestions below if you would like to go for a v3 but otherwise: Reviewed-by: Francisco Iglesias Thanks, Best regards, Francisco Iglesias > --- > Addressing all comments. > In reponse to this

Re: [RFC v1] util/aio: Keep notification disabled as much as possible

2022-07-07 Thread Chao Gao
On Thu, Jul 07, 2022 at 10:04:23AM +0100, Stefan Hajnoczi wrote: > >Does this patch solve the issue? The idea is to defer >poll_set_started(false) for as long as possible. Good idea! It is straightforward. > >diff --git a/util/aio-posix.c b/util/aio-posix.c >index 731f3826c0..536f8b2534 100644

Re: [RFC v1] util/aio: Keep notification disabled as much as possible

2022-07-07 Thread Stefan Hajnoczi
On Wed, Jul 06, 2022 at 10:12:14PM +0800, Chao Gao wrote: > On Wed, Jul 06, 2022 at 12:59:29PM +0100, Stefan Hajnoczi wrote: > >On Fri, Jul 01, 2022 at 05:13:48PM +0800, Chao Gao wrote: > >> When measuring FIO read performance (cache=writethrough, bs=4k, > >> iodepth=64) in > >> VMs, we observe

Re: [PATCH v3 10/13] tests/vm: Remove docker cross-compile test from CentOS VM

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:07AM -0400, John Snow wrote: > The fedora container has since been split apart, so there's no suitable > nearby target that would support "test-mingw" as it requires both x32 > and x64 support -- so either fedora-cross-win32 nor fedora-cross-win64 > would be truly

Re: [PATCH v3 09/13] tests/vm: upgrade Ubuntu 18.04 VM to 20.04

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:06AM -0400, John Snow wrote: > 18.04 has fallen out of our support window, so move ubuntu.aarch64 > forward to ubuntu 20.04, which is now our oldest supported Ubuntu > release. > > Signed-off-by: John Snow > --- > tests/vm/ubuntu.aarch64 | 14 +++--- > 1

Re: [PATCH v3 01/13] qga: treat get-guest-fsinfo as "best effort"

2022-07-07 Thread Marc-André Lureau
Hi On Thu, Jul 7, 2022 at 8:10 AM John Snow wrote: > In some container environments, there may be references to block devices > witnessable from a container through /proc/self/mountinfo that reference > devices we simply don't have access to in the container, and cannot > provide information

[PULL 0/2] Block patches

2022-07-07 Thread Stefan Hajnoczi
The following changes since commit 8e9398e3b1a860b8c29c670c1b6c36afe8d87849: Merge tag 'pull-ppc-20220706' of https://gitlab.com/danielhb/qemu into staging (2022-07-07 06:21:05 +0530) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git tags/block-pull-request for

Re: [PATCH v3 07/13] tests/vm: remove duplicate 'centos' VM test

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:04AM -0400, John Snow wrote: > This is listed twice by accident; we require genisoimage to run the > test, so remove the unconditional entry. > > Signed-off-by: John Snow > Reviewed-by: Thomas Huth > --- > tests/vm/Makefile.include | 2 +- > 1 file changed, 1

Re: [PATCH v3 06/13] tests/vm: remove ubuntu.i386 VM test

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:03AM -0400, John Snow wrote: > Ubuntu 18.04 is out of our support window, and Ubuntu 20.04 does not > support i386 anymore. The debian project does, but they do not provide > any cloud images for it, a new expect-style script would have to be > written. > > Since we

Re: [PATCH v3 03/13] tests/vm: switch CentOS 8 to CentOS 8 Stream

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:00AM -0400, John Snow wrote: > The old CentOS image didn't work anymore because it was already EOL at > the beginning of 2022. > > Signed-off-by: John Snow > Reviewed-by: Thomas Huth > --- > tests/vm/centos | 6 +++--- > 1 file changed, 3 insertions(+), 3

Re: [PATCH v3 02/13] tests/vm: use 'cp' instead of 'ln' for temporary vm images

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:02:59AM -0400, John Snow wrote: > If the initial setup fails, you've permanently altered the state of the > downloaded image in an unknowable way. Use 'cp' like our other test > setup scripts do. > > Signed-off-by: John Snow > Reviewed-by: Thomas Huth > --- >

Re: [PATCH v3 01/13] qga: treat get-guest-fsinfo as "best effort"

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:02:58AM -0400, John Snow wrote: > In some container environments, there may be references to block devices > witnessable from a container through /proc/self/mountinfo that reference > devices we simply don't have access to in the container, and cannot > provide

Re: [PATCH v3 08/13] tests/vm: add 1GB extra memory per core

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:05AM -0400, John Snow wrote: > If you try to run a 16 or 32 threaded test, you're going to run out of > memory very quickly with qom-test and a few others. Bump the memory > limit to try to scale with larger-core machines. > > Granted, this means that a 16 core

Re: [PATCH v3 05/13] tests/vm: update sha256sum for ubuntu.aarch64

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:02AM -0400, John Snow wrote: > This checksum changes weekly; use a fixed point image and update the > checksum so we don't have to re-download it quite so much. > > Note: Just like the centos.aarch64 test, this test currently seems very > flaky when run as a TCG

[PULL 2/2] block/io_uring: clarify that short reads can happen

2022-07-07 Thread Stefan Hajnoczi
Jens Axboe has confirmed that short reads are rare but can happen: https://lore.kernel.org/io-uring/YsU%2FCGkl9ZXUI+Tj@stefanha-x1.localdomain/T/#m729963dc577d709b709c191922e98ec79d7eef54 The luring_resubmit_short_read() comment claimed they were only due to a specific io_uring bug that was fixed

[PULL 1/2] io_uring: fix short read slow path

2022-07-07 Thread Stefan Hajnoczi
From: Dominique Martinet sqeq.off here is the offset to read within the disk image, so obviously not 'nread' (the amount we just read), but as the author meant to write its current value incremented by the amount we just read. Normally recent versions of linux will not issue short reads, but it

Re: [PATCH v3 04/13] tests/vm: switch centos.aarch64 to CentOS 8 Stream

2022-07-07 Thread Daniel P . Berrangé
On Thu, Jul 07, 2022 at 12:03:01AM -0400, John Snow wrote: > Switch this test over to using a cloud image like the base CentOS8 VM > test, which helps make this script a bit simpler too. > > Note: At time of writing, this test seems pretty flaky when run without > KVM support for aarch64. Certain

Re: [PATCH] block/io_uring: clarify that short reads can happen

2022-07-07 Thread Stefan Hajnoczi
On Wed, Jul 06, 2022 at 09:03:41AM +0100, Stefan Hajnoczi wrote: > Jens Axboe has confirmed that short reads are rare but can happen: > https://lore.kernel.org/io-uring/YsU%2FCGkl9ZXUI+Tj@stefanha-x1.localdomain/T/#m729963dc577d709b709c191922e98ec79d7eef54 > > The luring_resubmit_short_read()

Re: [PATCH] block/io_uring: clarify that short reads can happen

2022-07-07 Thread Stefano Garzarella
On Wed, Jul 06, 2022 at 09:03:41AM +0100, Stefan Hajnoczi wrote: Jens Axboe has confirmed that short reads are rare but can happen: https://lore.kernel.org/io-uring/YsU%2FCGkl9ZXUI+Tj@stefanha-x1.localdomain/T/#m729963dc577d709b709c191922e98ec79d7eef54 The luring_resubmit_short_read() comment

Re: [PATCH] hw/nvme: force nvme-ns param 'shared' to false if no nvme-subsys node

2022-07-07 Thread Klaus Jensen
On Jun 28 14:22, Niklas Cassel wrote: > Since commit 916b0f0b5264 ("hw/nvme: change nvme-ns 'shared' default") > the default value of nvme-ns param 'shared' is set to true, regardless > if there is a nvme-subsys node or not. > > On a system without a nvme-subsys node, a namespace will never be