Re: [PATCH v2 18/18] migration/ram: Add direct-io support to precopy file migration

2024-06-07 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 04:05:48PM -0300, Fabiano Rosas wrote: >> We've recently added support for direct-io with multifd, which brings >> performance benefits, but creates a non-uniform user interface by >> coupling direct-io with the multifd capability.

RE: [PATCH v7 6/7] migration/multifd: implement qpl compression and decompression

2024-06-06 Thread Fabiano Rosas
"Liu, Yuan1" writes: >> -Original Message- >> From: Fabiano Rosas >> Sent: Thursday, June 6, 2024 6:26 AM >> To: Liu, Yuan1 ; pet...@redhat.com; >> pbonz...@redhat.com; marcandre.lur...@redhat.com; berra...@redhat.com; >> th...@redhat.com;

Re: [PATCH v7 7/7] tests/migration-test: add qpl compression test

2024-06-05 Thread Fabiano Rosas
the software path for testing. > > Signed-off-by: Yuan Liu > Reviewed-by: Nanhai Zou > Reviewed-by: Peter Xu Reviewed-by: Fabiano Rosas

Re: [PATCH v7 6/7] migration/multifd: implement qpl compression and decompression

2024-06-05 Thread Fabiano Rosas
Yuan Liu writes: > QPL compression and decompression will use IAA hardware first. > If IAA hardware is not available, it will automatically fall > back to QPL software path, if the software job also fails, > the uncompressed page is sent directly. > > Signed-off-by: Yuan Liu > Reviewed-by:

Re: [PATCH v7 5/7] migration/multifd: implement initialization of qpl compression

2024-06-05 Thread Fabiano Rosas
h processing. > > Signed-off-by: Yuan Liu > Reviewed-by: Nanhai Zou Reviewed-by: Fabiano Rosas

Re: [PATCH v7 3/7] configure: add --enable-qpl build option

2024-06-05 Thread Fabiano Rosas
by: Yuan Liu > Reviewed-by: Nanhai Zou Reviewed-by: Fabiano Rosas

Re: [PATCH v7 1/7] docs/migration: add qpl compression feature

2024-06-05 Thread Fabiano Rosas
Yuan Liu writes: > add Intel Query Processing Library (QPL) compression method > introduction > > Signed-off-by: Yuan Liu > Reviewed-by: Nanhai Zou Just some nits if you need to respin. Otherwise I can touch up in the migration tree. Reviewed-by: Fabiano Rosas > --- >

Re: [PATCH 7/7] tests/migration-test: add uadk compression test

2024-06-05 Thread Fabiano Rosas
Shameer Kolothum via writes: > Signed-off-by: Shameer Kolothum Reviewed-by: Fabiano Rosas

Re: [PATCH 6/7] migration/multifd: Switch to no compression when no hardware support

2024-06-05 Thread Fabiano Rosas
ed-off-by: Shameer Kolothum Reviewed-by: Fabiano Rosas

Re: [PATCH 1/7] docs/migration: add uadk compression feature

2024-06-05 Thread Fabiano Rosas
Shameer Kolothum via writes: > Document UADK(User Space Accelerator Development Kit) library details > and how to use that for migration. > > Signed-off-by: Shameer Kolothum > --- > docs/devel/migration/uadk-compression.rst | 144 ++ Missing an entry in the features.rst

Re: [PATCH 5/7] migration/multifd: Add UADK based compression and decompression

2024-06-05 Thread Fabiano Rosas
Shameer Kolothum via writes: > Uses UADK wd_do_comp_sync() API to (de)compress a normal page using > hardware accelerator. > > Signed-off-by: Shameer Kolothum A couple of comments below. Reviewed-by: Fabiano Rosas > --- > migration/mu

Re: [PATCH 4/7] migration/multifd: Add UADK initialization

2024-06-05 Thread Fabiano Rosas
Shameer Kolothum via writes: > Initialize UADK session and allocate buffers required. The actual > compression/decompression will only be done in a subsequent patch. > > Signed-off-by: Shameer Kolothum > --- > migration/multifd-uadk.c | 207 ++- > 1 file

Re: [RFC PATCH] migration/savevm: do not schedule snapshot_save_job_bh in qemu_aio_context

2024-06-05 Thread Fabiano Rosas
Fiona Ebner writes: > The fact that the snapshot_save_job_bh() is scheduled in the main > loop's qemu_aio_context AioContext means that it might get executed > during a vCPU thread's aio_poll(). But saving of the VM state cannot > happen while the guest or devices are active and can lead to

Re: [PATCH v2 06/18] monitor: Stop removing non-duplicated fds

2024-06-05 Thread Fabiano Rosas
"Dr. David Alan Gilbert" writes: > * Fabiano Rosas (faro...@suse.de) wrote: >> Peter Xu writes: >> >> > On Thu, May 23, 2024 at 04:05:36PM -0300, Fabiano Rosas wrote: >> >> We've been up until now cleaning up any file descriptors that have >&g

Re: [PATCH 2/7] configure: Add uadk option

2024-06-04 Thread Fabiano Rosas
Shameer Kolothum via writes: > Add --enable-uadk and --disable-uadk options to enable and disable > UADK compression accelerator. This is for using UADK based hardware > accelerators for live migration. > > Signed-off-by: Shameer Kolothum Reviewed-by: Fabiano Rosas

Re: [PATCH 3/7] migration/multifd: add uadk compression framework

2024-06-04 Thread Fabiano Rosas
Shameer Kolothum via writes: > Adds the skeleton to support uadk compression method. > Complete functionality will be added in subsequent patches. > > Signed-off-by: Shameer Kolothum Reviewed-by: Fabiano Rosas

Re: [PATCH v3 0/4] tests/qtest/migration-test: Improve and enable on ppc64

2024-06-04 Thread Fabiano Rosas
On Thu, 30 May 2024 17:44:48 +1000, Nicholas Piggin wrote: > Since v2: > - Fixed subject typo noticed by Thomas. > - Drop the non-ppc patches from the series. > > Thanks, > Nick > > [...] Queued, thanks!

Re: [PULL 25/72] Adds migration support for Branch History Rolling Buffer (BHRB) internal state.

2024-05-31 Thread Fabiano Rosas
Nicholas Piggin writes: > From: Glenn Miles > > Reviewed-by: Nicholas Piggin > Signed-off-by: Glenn Miles > Signed-off-by: Nicholas Piggin > --- > target/ppc/machine.c | 21 + > 1 file changed, 21 insertions(+) > > diff --git a/target/ppc/machine.c b/target/ppc/machine.c

Re: [PATCH v2 4/4] tests/qtest/migration-test: Add a postcopy memfile test

2024-05-31 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 30, 2024 at 07:54:07PM +1000, Nicholas Piggin wrote: >> Postcopy requires userfaultfd support, which requires tmpfs if a memory >> file is used. >> >> This adds back support for /dev/shm memory files, but adds preallocation >> to skip environments where that mount

Re: [PATCH v2 2/4] tests/qtest/migration-test: Enable test_mode_reboot

2024-05-31 Thread Fabiano Rosas
f-by: Nicholas Piggin Reviewed-by: Fabiano Rosas

Re: [PATCH v2 13/18] monitor: fdset: Match against O_DIRECT

2024-05-31 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 04:05:43PM -0300, Fabiano Rosas wrote: >> We're about to enable the use of O_DIRECT in the migration code and >> due to the alignment restrictions imposed by filesystems we need to >> make sure the flag is only used

Re: [PATCH v2 11/18] migration/multifd: Add direct-io support

2024-05-31 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 04:05:41PM -0300, Fabiano Rosas wrote: >> When multifd is used along with mapped-ram, we can take benefit of a >> filesystem that supports the O_DIRECT flag and perform direct I/O in >> the multifd threads. This brings a si

Re: [PATCH v2 06/18] monitor: Stop removing non-duplicated fds

2024-05-31 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 04:05:36PM -0300, Fabiano Rosas wrote: >> We've been up until now cleaning up any file descriptors that have >> been passed into QEMU and never duplicated[1,2]. A file descriptor >> without duplicates indicates that no part

Re: [PATCH v2 05/18] monitor: Introduce monitor_fdset_*free

2024-05-31 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 04:05:35PM -0300, Fabiano Rosas wrote: >> Introduce two new functions to remove and free no longer used fds and >> fdsets. >> >> We need those to decouple the remove/free routines from >> monitor_fdset_cleanup() which

Re: [PATCH v2 01/18] migration: Fix file migration with fdset

2024-05-31 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 04:05:31PM -0300, Fabiano Rosas wrote: >> When the "file:" migration support was added we missed the special >> case in the qemu_open_old implementation that allows for a particular >> file name format to be used to ref

Re: [PATCH] tests/qtest/migrate-test: Add a postcopy memfile test

2024-05-29 Thread Fabiano Rosas
Peter Xu writes: > On Wed, May 29, 2024 at 09:54:30AM -0300, Fabiano Rosas wrote: >> Nicholas Piggin writes: >> >> > Postcopy requires userfaultfd support, which requires tmpfs if a memory >> > file is used. >> > >> > This adds back suppor

Re: [PULL 6/9] virtio-gpu: fix v2 migration

2024-05-29 Thread Fabiano Rosas
Fiona Ebner writes: > CC-ing stable, because this already is an issue in 9.0.0 > Thank you for pointing this out. I was expecting b4 to find the tag, but I just now noticed that the CC was added by Peter as a reply to the message and not originally via the patch headers, so I should have added

Re: [PATCH] tests/qtest/migrate-test: Add a postcopy memfile test

2024-05-29 Thread Fabiano Rosas
Nicholas Piggin writes: > Postcopy requires userfaultfd support, which requires tmpfs if a memory > file is used. > > This adds back support for /dev/shm memory files, but adds preallocation > to skip environments where that mount is limited in size. > > Signed-off-by: Nicholas Piggin > --- > >

Re: [PATCH] tests/qtest/migrate-test: Use regular file file for shared-memory tests

2024-05-28 Thread Fabiano Rosas
Peter Xu writes: > On Tue, May 28, 2024 at 09:35:22AM -0400, Peter Xu wrote: >> On Tue, May 28, 2024 at 02:27:57PM +1000, Nicholas Piggin wrote: >> > There is no need to use /dev/shm for file-backed memory devices, and >> > it is too small to be usable in gitlab CI. Switch to using a regular >>

Re: [RFC PATCH 4/4] ci: Add the new migration device tests

2024-05-28 Thread Fabiano Rosas
Peter Xu writes: > On Mon, May 27, 2024 at 08:59:00PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Thu, May 23, 2024 at 05:19:22PM -0300, Fabiano Rosas wrote: >> >> We have two new migration tests that check cross version >> >

Re: [PATCH] tests/qtest/migrate-test: Use regular file file for shared-memory tests

2024-05-28 Thread Fabiano Rosas
Nicholas Piggin writes: > There is no need to use /dev/shm for file-backed memory devices, and > it is too small to be usable in gitlab CI. Switch to using a regular > file in /tmp/ which will usually have more space available. > > Signed-off-by: Nicholas Piggin > --- > Am I missing something?

Re: [RFC PATCH 4/4] ci: Add the new migration device tests

2024-05-27 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 05:19:22PM -0300, Fabiano Rosas wrote: >> We have two new migration tests that check cross version >> compatibility. One uses the vmstate-static-checker.py script to >> compare the vmstate structures from two different QEMU versio

Re: [RFC PATCH 3/4] tests/qtest/migration: Add support for simple device tests

2024-05-27 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 05:19:21PM -0300, Fabiano Rosas wrote: >> The current migration-tests are almost entirely focused on catching >> bugs on the migration code itself, not on the device migration >> infrastructure (vmstate). That means we miss catc

Re: [RFC PATCH 2/4] tests/qtest/migration: Add a test that runs vmstate-static-checker

2024-05-27 Thread Fabiano Rosas
Peter Xu writes: > On Thu, May 23, 2024 at 05:19:20PM -0300, Fabiano Rosas wrote: >> We have the vmstate-static-checker script that takes the output of: >> '$QEMU -M $machine -dump-vmstate' for two different QEMU versions and >> compares them to check for compatibility br

Re: [RFC PATCH 3/3] tests/qtest/migration-test: Enable test_ignore_shared

2024-05-27 Thread Fabiano Rosas
Peter Xu writes: > On Mon, May 27, 2024 at 09:42:28AM -0300, Fabiano Rosas wrote: >> However, there is an issue here still on all archs - which might very >> well have been the original issue - which is the fact that the >> containers on the Gitlab CI have limits o

Re: [PATCH 3/3] tests/qtest/migration-test: Use custom asm bios for ppc64

2024-05-27 Thread Fabiano Rosas
Nicholas Piggin writes: > Similar to other archs, build a custom bios memory updater. Running the > test with OF code is a cool trick, but SLOF takes a long time to boot. > This reduces test time by around 3x (150s to 50s). > > Signed-off-by: Nicholas Piggin Reviewed-by: Fabiano Rosas

Re: [RFC PATCH 3/3] tests/qtest/migration-test: Enable test_ignore_shared

2024-05-27 Thread Fabiano Rosas
Nicholas Piggin writes: > This was said to be broken on aarch64, but if it works on others, > let's try enable it. It's already starting to bitrot... Yeah, look at the state of this... I don't know what the issue was on aarch64, but I'm all for enabling this test globally and then we deal with

Re: [PATCH V1 20/26] migration: cpr-exec mode

2024-05-24 Thread Fabiano Rosas
Steve Sistare writes: > Add the cpr-exec migration mode. Usage: > qemu-system-$arch -machine memfd-alloc=on ... > migrate_set_parameter mode cpr-exec > migrate_set_parameter cpr-exec-args \ > ... -incoming > migrate -d > > The migrate command stops the VM, saves state to the

Re: [PATCH V1 05/26] migration: precreate vmstate

2024-05-24 Thread Fabiano Rosas
f-by: Steve Sistare Reviewed-by: Fabiano Rosas

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-24 Thread Fabiano Rosas
Steve Sistare writes: > This patch series adds the live migration cpr-exec mode. In this mode, QEMU > stops the VM, writes VM state to the migration URI, and directly exec's a > new version of QEMU on the same host, replacing the original process while > retaining its PID. Guest RAM is

Re: [PATCH V1 23/26] migration: misc cpr-exec blockers

2024-05-24 Thread Fabiano Rosas
Steve Sistare writes: > Add blockers for cpr-exec migration mode for devices and options that do > not support it. > > Signed-off-by: Steve Sistare > --- > accel/xen/xen-all.c| 5 + > backends/hostmem-epc.c | 12 ++-- > hw/vfio/migration.c| 3 ++- > replay/replay.c

Re: [PATCH v2 01/18] migration: Fix file migration with fdset

2024-05-24 Thread Fabiano Rosas
Prasad Pandit writes: > On Fri, 24 May 2024 at 00:38, Fabiano Rosas wrote: >> This is further indicated by the presence of the 'offset' >> argument, which indicates the start of the region where QEMU is >> allowed to write. >> >> Fix the issue by r

Re: [PATCH] tests/qtest/migration-test: Run some basic tests on s390x and ppc64 with TCG, too

2024-05-24 Thread Fabiano Rosas
Thomas Huth writes: > On 24/05/2024 02.05, Nicholas Piggin wrote: >> On Wed May 22, 2024 at 7:12 PM AEST, Thomas Huth wrote: >>> On s390x, we recently had a regression that broke migration / savevm >>> (see commit bebe9603fc ("hw/intc/s390_flic: Fix crash that occurs when >>> saving the machine

[RFC PATCH 3/4] tests/qtest/migration: Add support for simple device tests

2024-05-23 Thread Fabiano Rosas
$ cd build $ QTEST_DEVICE_OPTS='-device virtio-net' \ QTEST_QEMU_BINARY=./qemu-system-x86_64 \ QTEST_QEMU_BINARY_DST=../build-previous/qemu-system-x86_64 \ ./tests/qtest/migration-test -p /x86_64/migration/precopy/tcp/plain Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 19

[RFC PATCH 0/4] migration-test: Device migration smoke tests

2024-05-23 Thread Fabiano Rosas
, either a hardcoded one provided by us, or a custom one provided by whoever is experimenting with this code. This also give us the ability to quickly check what happens when a new (to the tests) device is added. 1- https://lore.kernel.org/r/87wmo5l58z@suse.de Fabiano Rosas (4): tests/qtest

[RFC PATCH 2/4] tests/qtest/migration: Add a test that runs vmstate-static-checker

2024-05-23 Thread Fabiano Rosas
\ ./tests/qtest/migration-test -p /x86_64/migration/vmstate-checker-script Signed-off-by: Fabiano Rosas --- some code duplication for now, just so we can reason about this without too much noise --- tests/qtest/migration-test.c | 82 1 file changed, 82 insertions

[RFC PATCH 1/4] tests/qtest/libqtest: Introduce another qtest_init version with no handshake

2024-05-23 Thread Fabiano Rosas
this time we want to pass along the special QTEST_QEMU_BINARY_SRC|DST environment variables. Signed-off-by: Fabiano Rosas --- tests/qtest/libqtest.c | 14 +- tests/qtest/libqtest.h | 13 + 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/tests/qtest

[RFC PATCH 4/4] ci: Add the new migration device tests

2024-05-23 Thread Fabiano Rosas
tests to the migration-compat-common job. Signed-off-by: Fabiano Rosas --- .gitlab-ci.d/buildtest.yml | 43 +++--- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 91c57efded..bc7ac35983

[PATCH v2 03/18] tests/qtest/migration: Add a precopy file test with fdset

2024-05-23 Thread Fabiano Rosas
providing a non empty migration file. Note that fd passing is not available on Windows, so anything that uses add-fd needs to exclude that platform. Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 44 1 file changed, 44 insertions(+) diff --git

[PATCH v2 08/18] monitor: Report errors from monitor_fdset_dup_fd_add

2024-05-23 Thread Fabiano Rosas
I'm keeping the EACCES because callers expect to be able to look at errno. Signed-off-by: Fabiano Rosas --- include/monitor/monitor.h | 2 +- monitor/fds.c | 10 +- stubs/fdset.c | 2 +- util/osdep.c | 10 +- 4 files changed, 12 insertions

[PATCH v2 12/18] tests/qtest/migration: Add tests for file migration with direct-io

2024-05-23 Thread Fabiano Rosas
The tests are only allowed to run in systems that know about the O_DIRECT flag and in filesystems which support it. Note: this also brings back migrate_set_parameter_bool() which went away when we removed the compression tests. I copied it verbatim. Signed-off-by: Fabiano Rosas Reviewed

[PATCH v2 07/18] monitor: Simplify fdset and fd removal

2024-05-23 Thread Fabiano Rosas
Remove fds right away instead of setting the ->removed flag. We don't need the extra complexity of having a cleanup function reap the removed entries at a later time. Signed-off-by: Fabiano Rosas --- monitor/fds.c | 27 ++- 1 file changed, 6 insertions(+), 21 deleti

[PATCH v2 09/18] io: Stop using qemu_open_old in channel-file

2024-05-23 Thread Fabiano Rosas
We want to make use of the Error object to report fdset errors from qemu_open_internal() and passing the error pointer to qemu_open_old() would require changing all callers. Move the file channel to the new API instead. Reviewed-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas --- io

[PATCH v2 06/18] monitor: Stop removing non-duplicated fds

2024-05-23 Thread Fabiano Rosas
t") Signed-off-by: Fabiano Rosas --- monitor/fds.c | 15 --- monitor/hmp.c | 2 -- monitor/monitor-internal.h | 1 - monitor/qmp.c | 2 -- 4 files changed, 8 insertions(+), 12 deletions(-) diff --git a/monitor/fds.c b/monitor/fds.c index 10

[PATCH v2 04/18] monitor: Drop monitor_fdset_dup_fd_add()

2024-05-23 Thread Fabiano Rosas
that situation so far. Cc: Dr. David Alan Gilbert Cc: Markus Armbruster Cc: Philippe Mathieu-Daudé Cc: Paolo Bonzini Cc: Daniel P. Berrangé Signed-off-by: Peter Xu [add missing return statement, removal during traversal is not safe] Signed-off-by: Fabiano Rosas --- include/monitor/monitor.h

[PATCH v2 17/18] migration: Add direct-io helpers

2024-05-23 Thread Fabiano Rosas
/disable direct-io around the aligned parts. Signed-off-by: Fabiano Rosas --- migration/migration.c | 31 +++ migration/migration.h | 2 ++ migration/qemu-file.c | 29 + migration/qemu-file.h | 2 +- 4 files changed, 63 insertions(+), 1

[PATCH v2 15/18] tests/qtest/migration: Add a test for mapped-ram with passing of fds

2024-05-23 Thread Fabiano Rosas
that fdset code goes under _WIN32 because fd passing is not supported on Windows. Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 101 +-- 1 file changed, 98 insertions(+), 3 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration

[PATCH v2 16/18] io/channel-file: Add direct-io support

2024-05-23 Thread Fabiano Rosas
Add support for setting/clearing the O_DIRECT flag on a file descriptor. This will be used for enabling O_DIRECT in the main migration channel when multifd is not in use. Signed-off-by: Fabiano Rosas --- include/io/channel-file.h | 8 io/channel-file.c | 25

[PATCH v2 11/18] migration/multifd: Add direct-io support

2024-05-23 Thread Fabiano Rosas
descriptor to be used exclusively for direct I/O. The second file descriptor can be obtained by QEMU by re-opening the migration file (already possible), or by being provided by the user or management application (support to be added in future patches). Signed-off-by: Fabiano Rosas --- migration

[PATCH v2 18/18] migration/ram: Add direct-io support to precopy file migration

2024-05-23 Thread Fabiano Rosas
sent. Fortunately, due to fixed-ram migration there is a discernible moment where only RAM pages are written to the migration file. Enable direct-io during that moment. Signed-off-by: Fabiano Rosas --- migration/ram.c | 40 tests/qtest/migration-te

[PATCH v2 05/18] monitor: Introduce monitor_fdset_*free

2024-05-23 Thread Fabiano Rosas
and when an fd is removed to cleanup any fdset that is now empty. - monitor_fdset_fd_free() will be used to remove one or more fds that have been explicitly targeted by qmp_remove_fd(). Signed-off-by: Fabiano Rosas --- monitor/fds.c | 26 ++ 1 file changed, 18 insertions(+), 8

[PATCH v2 02/18] tests/qtest/migration: Fix file migration offset check

2024-05-23 Thread Fabiano Rosas
, switch to using g_get_file_contents() which is more portable than mmap(). Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 79 ++-- 1 file changed, 48 insertions(+), 31 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration

[PATCH v2 00/18] migration/mapped-ram: Add direct-io support

2024-05-23 Thread Fabiano Rosas
arosas/qemu/-/pipelines/1269352083 Fabiano Rosas (17): migration: Fix file migration with fdset tests/qtest/migration: Fix file migration offset check tests/qtest/migration: Add a precopy file test with fdset monitor: Introduce monitor_fdset_*free monitor: Stop removing non-duplicated fds

[PATCH v2 10/18] migration: Add direct-io parameter

2024-05-23 Thread Fabiano Rosas
Signed-off-by: Fabiano Rosas --- include/qemu/osdep.h | 2 ++ migration/migration-hmp-cmds.c | 11 +++ migration/options.c| 28 migration/options.h| 1 + qapi/migration.json| 21 ++--- util/osdep.c

[PATCH v2 14/18] migration: Add documentation for fdset with multifd + file

2024-05-23 Thread Fabiano Rosas
ith RDONLY on the migration destination side. Signed-off-by: Fabiano Rosas --- docs/devel/migration/main.rst | 24 +++- docs/devel/migration/mapped-ram.rst | 6 +- 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/docs/devel/migration/main.rst b/docs/devel

[PATCH v2 13/18] monitor: fdset: Match against O_DIRECT

2024-05-23 Thread Fabiano Rosas
flag. Signed-off-by: Fabiano Rosas --- monitor/fds.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/monitor/fds.c b/monitor/fds.c index d93d2e695b..a6580e215e 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -419,6 +419,11 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id

[PATCH v2 01/18] migration: Fix file migration with fdset

2024-05-23 Thread Fabiano Rosas
the region where QEMU is allowed to write. Fix the issue by replacing the O_TRUNC flag on open by an ftruncate call, which will take the offset into consideration. Fixes: 385f510df5 ("migration: file URI offset") Suggested-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas --- m

[PULL 6/9] virtio-gpu: fix v2 migration

2024-05-22 Thread Fabiano Rosas
te-version and a field test to save/load appropriately according to the machine version. Fixes: dfcf74fa ("virtio-gpu: fix scanout migration post-load") Signed-off-by: Marc-André Lureau Signed-off-by: Peter Xu Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner [fixed long lines] Signed-off-by: Fa

[PULL 7/9] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

2024-05-22 Thread Fabiano Rosas
SO features") Signed-off-by: Fiona Ebner Reviewed-by: Fabiano Rosas Acked-by: Jason Wang Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- hw/core/machine.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index 8d6d

[PULL 5/9] migration: fix a typo

2024-05-22 Thread Fabiano Rosas
From: Marc-André Lureau Signed-off-by: Marc-André Lureau Reviewed-by: Peter Xu Reviewed-by: Fabiano Rosas Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner Signed-off-by: Fabiano Rosas --- migration/vmstate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration

[PULL 9/9] tests/qtest/migration-test: Fix the check for a successful run of analyze-migration.py

2024-05-22 Thread Fabiano Rosas
emu: Assertion `pid == s->qemu_pid' failed. ok 2 /x86_64/migration/analyze-script ... Let's better fail the test in case the child did not exit properly, too. Signed-off-by: Thomas Huth Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- tests/qtest/migration-

[PULL 1/9] migration/colo: Minor fix for colo error message

2024-05-22 Thread Fabiano Rosas
From: Li Zhijian - Explicitly show the missing module name: replication - Fix capability name to x-colo Reviewed-by: Peter Xu Reviewed-by: Zhang Chen Signed-off-by: Li Zhijian Suggested-by: Michael Tokarev [fixed mangled author email address] Signed-off-by: Fabiano Rosas --- migration

[PULL 2/9] migration/colo: make colo_incoming_co() return void

2024-05-22 Thread Fabiano Rosas
exits. Cc: Fabiano Rosas Reviewed-by: Peter Xu Reviewed-by: Zhang Chen Signed-off-by: Li Zhijian [fixed mangled author email address] Signed-off-by: Fabiano Rosas --- include/migration/colo.h | 2 +- migration/colo-stubs.c | 3 +-- migration/colo.c | 9 ++--- migration/migration.c

[PULL 8/9] tests/qtest/migration-test: Run some basic tests on s390x and ppc64 with TCG, too

2024-05-22 Thread Fabiano Rosas
ot;bad_dest" tests before checking for KVM on s390x or ppc64 (this also fixes the problem that the "analyze-script" test was not run on s390x at all anymore since it got disabled again by accident in a previous refactoring of the code). Signed-off-by: Thomas Huth Reviewed-by: Fabiano

[PULL 3/9] migration/colo: Tidy up bql_unlock() around bdrv_activate_all()

2024-05-22 Thread Fabiano Rosas
From: Li Zhijian Make the code more tight. Suggested-by: Michael Tokarev Reviewed-by: Peter Xu Reviewed-by: Zhang Chen Signed-off-by: Li Zhijian Signed-off-by: Michael Tokarev [fixed mangled author email address] Signed-off-by: Fabiano Rosas --- migration/colo.c | 3 +-- 1 file changed

[PULL 0/9] Migration patches for 2024-05-22

2024-05-22 Thread Fabiano Rosas
The following changes since commit 01782d6b294f95bcde334386f0aaac593cd28c0d: Merge tag 'hw-misc-20240517' of https://github.com/philmd/qemu into staging (2024-05-18 11:49:01 +0200) are available in the Git repository at: https://gitlab.com/farosas/qemu.git

[PULL 4/9] migration: add "exists" info to load-state-field trace

2024-05-22 Thread Fabiano Rosas
From: Marc-André Lureau Signed-off-by: Marc-André Lureau Reviewed-by: Peter Xu Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner Signed-off-by: Fabiano Rosas --- migration/trace-events | 2 +- migration/vmstate.c| 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git

Re: [PATCH] tests/qtest/migration-test: Fix the check for a successful run of analyze-migration.py

2024-05-22 Thread Fabiano Rosas
On Wed, 22 May 2024 11:23:01 +0200, Thomas Huth wrote: > If analyze-migration.py cannot be run or crashes, the error is currently > ignored since the code only checks for nonzero values in case the child > exited properly. For example, if you run the test with a non-existing > Python interpreter,

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

2024-05-22 Thread Fabiano Rosas
On Fri, 17 May 2024 09:53:36 +0200, Fiona Ebner wrote: > Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine > version 8.1 can fail with: > > > kvm: Features 0x1c0010130afffa7 unsupported. Allowed features: 0x10179bfffe7 > > kvm: Failed to load virtio-net:virtio > > kvm: error while

Re: [PATCH v4 0/3] Fix "virtio-gpu: fix scanout migration post-load"

2024-05-22 Thread Fabiano Rosas
On Thu, 16 May 2024 12:40:19 +0400, marcandre.lur...@redhat.com wrote: > The aforementioned patch breaks virtio-gpu device migrations for versions > pre-9.0/9.0, both forwards and backwards. Versioning of `VMS_STRUCT` is more > complex than it may initially appear, as evidenced in the problematic

Re: [PATCH] tests/qtest/migration-test: Run some basic tests on s390x and ppc64 with TCG, too

2024-05-22 Thread Fabiano Rosas
On Wed, 22 May 2024 11:12:55 +0200, Thomas Huth wrote: > On s390x, we recently had a regression that broke migration / savevm > (see commit bebe9603fc ("hw/intc/s390_flic: Fix crash that occurs when > saving the machine state"). The problem was merged without being noticed > since we currently do

Re: [PATCH v2 1/3] migration/colo: Minor fix for colo error message

2024-05-22 Thread Fabiano Rosas
On Thu, 16 May 2024 11:45:15 +0800, Li Zhijian via wrote: > - Explicitly show the missing module name: replication > - Fix capability name to x-colo > > Queued, thanks!

Re: [PATCH] tests/qtest/migration-test: Fix the check for a successful run of analyze-migration.py

2024-05-22 Thread Fabiano Rosas
ATUS(wstatus) != 0) { > +if (!WIFEXITED(wstatus) || WEXITSTATUS(wstatus) != 0) { > g_test_message("Failed to analyze the migration stream"); > g_test_fail(); > } Reviewed-by: Fabiano Rosas

Re: [PATCH] tests/qtest/migration-test: Run some basic tests on s390x and ppc64 with TCG, too

2024-05-22 Thread Fabiano Rosas
> Thus enable the "analyze-script" and the "bad_dest" tests before checking > for KVM on s390x or ppc64 (this also fixes the problem that the > "analyze-script" test was not run on s390x at all anymore since it got > disabled again by accident in a previous refactori

Re: [PULL 10/38] tests/qtest/migration: Add a test for the analyze-migration script

2024-05-22 Thread Fabiano Rosas
Thomas Huth writes: > On 21/05/2024 14.46, Fabiano Rosas wrote: >> Alex Bennée writes: >> >>> Juan Quintela writes: >>> >>>> From: Fabiano Rosas >>>> >>>> Add a smoke test that migrates to a file and gives it to the

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

2024-05-21 Thread Fabiano Rosas
Fiona Ebner writes: > Am 21.05.24 um 00:22 schrieb Fabiano Rosas: >> Fiona Ebner writes: >> >>> Migration from an 8.2 or 9.0 binary to an 8.1 binary with machine >>> version 8.1 can fail with: >>> >>>> kvm: Features 0x1c0010130afffa7 unsup

Re: [PULL 10/38] tests/qtest/migration: Add a test for the analyze-migration script

2024-05-21 Thread Fabiano Rosas
Alex Bennée writes: > Juan Quintela writes: > >> From: Fabiano Rosas >> >> Add a smoke test that migrates to a file and gives it to the >> script. It should catch the most annoying errors such as changes in >> the ram flags. >> >> After code has

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-20 Thread Fabiano Rosas
Steven Sistare writes: > Hi Peter, Hi Fabiano, >Will you have time to review the migration guts of this series any time > soon? > In particular: > > [PATCH V1 05/26] migration: precreate vmstate > [PATCH V1 06/26] migration: precreate vmstate for exec > [PATCH V1 12/26] migration: vmstate

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

2024-05-20 Thread Fabiano Rosas
t this patch to a binary with this > patch. > > Fixes: 53da8b5a99 ("virtio-net: Add support for USO features") > Signed-off-by: Fiona Ebner Reviewed-by: Fabiano Rosas I'll get to it eventually, but is this another one where just having -device virtio-net in the command line when testing cross-version migration would already have caught the issue?

Re: [PATCH 8/9] migration: Add support for fdset with multifd + file

2024-05-17 Thread Fabiano Rosas
Daniel P. Berrangé writes: > On Wed, May 08, 2024 at 05:39:53PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Wed, May 08, 2024 at 09:53:48AM +0100, Daniel P. Berrangé wrote: >> >> On Fri, Apr 26, 2024 at 11:20:41AM -0300, Fabiano Rosas wrote: &

Re: [PATCH 1/9] monitor: Honor QMP request for fd removal immediately

2024-05-16 Thread Fabiano Rosas
Daniel P. Berrangé writes: > On Fri, Apr 26, 2024 at 11:20:34AM -0300, Fabiano Rosas wrote: >> We're enabling using the fdset interface to pass file descriptors for >> use in the migration code. Since migrations can happen more than once >> during the VMs lifetime, we need

Re: [PATCH 1/9] monitor: Honor QMP request for fd removal immediately

2024-05-16 Thread Fabiano Rosas
Hi all, sorry to have been away from this thread for a while, I was trying to catch up on my reviews queue. Peter Xu writes: > On Fri, Apr 26, 2024 at 11:20:34AM -0300, Fabiano Rosas wrote: >> We're enabling using the fdset interface to pass file descriptors for >> use in the

Re: [PATCH 2/3] migration/colo: make colo_incoming_co() return void

2024-05-15 Thread Fabiano Rosas
Li Zhijian via writes: > Currently, it always returns 0, no need to check the return value at all. > In addition, enter colo coroutine only if migration_incoming_colo_enabled() > is true. > Once the destination side enters the COLO* state, the COLO process will > take over the remaining

Re: [PATCH 4/9] migration: Add direct-io parameter

2024-05-15 Thread Fabiano Rosas
Markus Armbruster writes: > Fabiano Rosas writes: > >> Markus Armbruster writes: >> >>> Peter Xu writes: >>> >>>> On Fri, May 03, 2024 at 05:49:32PM -0300, Fabiano Rosas wrote: >>>>> Peter Xu writes: >>>>> >&

Re: [PATCH 4/9] migration: Add direct-io parameter

2024-05-14 Thread Fabiano Rosas
Markus Armbruster writes: > Peter Xu writes: > >> On Fri, May 03, 2024 at 05:49:32PM -0300, Fabiano Rosas wrote: >>> Peter Xu writes: >>> >>> > On Fri, Apr 26, 2024 at 11:20:37AM -0300, Fabiano Rosas wrote: >>> >> Add the di

RE: [PATCH v6 6/7] migration/multifd: implement qpl compression and decompression

2024-05-14 Thread Fabiano Rosas
"Liu, Yuan1" writes: >> -Original Message- >> From: Fabiano Rosas >> Sent: Monday, May 13, 2024 11:14 PM >> To: Liu, Yuan1 ; pet...@redhat.com >> Cc: qemu-devel@nongnu.org; Liu, Yuan1 ; Zou, Nanhai >> >> Subject: Re: [PATCH

Re: [PATCH v2 2/4] migration: fix a typo

2024-05-13 Thread Fabiano Rosas
; -/* subsection name has be be "section_name/a" */ > +/* subsection name has to be "section_name/a" */ > trace_vmstate_subsection_load_bad(vmsd->name, "(short)", ""); > return 0; > } Reviewed-by: Fabiano Rosas

Re: [PATCH v5 03/10] vfio: Extend migration_file_set_error() with Error** argument

2024-05-13 Thread Fabiano Rosas
Cédric Le Goater writes: > Use it to update the current error of the migration stream if > available and if not, simply print out the error. Next changes will > update with an error to report. > > Signed-off-by: Cédric Le Goater Acked-by: Fabiano Rosas

Re: [PATCH V1 26/26] migration: only-migratable-modes

2024-05-13 Thread Fabiano Rosas
Steven Sistare writes: > On 5/9/2024 3:14 PM, Fabiano Rosas wrote: >> Steve Sistare writes: >> >>> Add the only-migratable-modes option as a generalization of only-migratable. >>> Only devices that support all requested modes are allowed. >

Re: [PATCH V1 06/26] migration: precreate vmstate for exec

2024-05-13 Thread Fabiano Rosas
Steven Sistare writes: > On 5/6/2024 7:34 PM, Fabiano Rosas wrote: >> Steve Sistare writes: >> >>> Provide migration_precreate_save for saving precreate vmstate across exec. >>> Create a memfd, save its value in the environment, and serialize stat

  1   2   3   4   5   6   7   8   9   10   >