Re: guest Linux Kernel hangs and reports CPU lockup/stuck gitlab bug

2022-11-25 Thread Fabiano Rosas
Gerd Hoffmann writes: > On Wed, Sep 21, 2022 at 11:55:01AM +0200, Claudio Fontana wrote: >> Hi, >> >> I think this bug report warrants some attention, >> >> can Gerd take a look here? >> >> The GTK Clipboard commit seems involved: >> >> https://gitlab.com/qemu-project/qemu/-/issues/1150 > > H

Re: [PATCH-for-8.0 1/2] typedefs: Forward-declare AccelState

2022-11-30 Thread Fabiano Rosas
Philippe Mathieu-Daudé writes: > Forward-declare AccelState in "qemu/typedefs.h" so structures > using a reference of it (like MachineState in "hw/boards.h") > don't have to include "qemu/accel.h". > > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Fabiano Rosas

Re: [PATCH-for-8.0 2/2] hw: Reduce "qemu/accel.h" inclusion

2022-11-30 Thread Fabiano Rosas
explicitly. Anyway, Reviewed-by: Fabiano Rosas > Signed-off-by: Philippe Mathieu-Daudé > --- > hw/core/machine.c | 1 + > include/hw/boards.h | 1 - > 2 files changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/core/machine.c b/hw/core/machine.c > index 8d34caa3

Re: [PATCH v2 1/1] tcg: convert tcg/README to rst

2022-11-30 Thread Fabiano Rosas
the Atomic Operations document to use a proper > reference > > - Replace tcg/README in tcg.rst with a proper reference to the new document > > Signed-off-by: Mark Cave-Ayland Reviewed-by: Fabiano Rosas I have a couple of suggestions for a small restructuring. But we could

Re: [PATCH 1/3] block: mention 'password-secret' option for -iscsi

2022-12-01 Thread Fabiano Rosas
Daniel P. Berrangé writes: > The 'password-secret' option was added > > commit b189346eb1784df95ed6fed610411dbf23d19e1f > Author: Daniel P. Berrangé > Date: Thu Jan 21 14:19:21 2016 + > > iscsi: add support for getting CHAP password via QCryptoSecret API > > but was not mentioned

Re: [PATCH v10 2/9] KVM: Introduce per-page memory attributes

2022-12-06 Thread Fabiano Rosas
Chao Peng writes: > In confidential computing usages, whether a page is private or shared is > necessary information for KVM to perform operations like page fault > handling, page zapping etc. There are other potential use cases for > per-page memory attributes, e.g. to make memory read-only (or

Re: [PULL 10/66] tests/qtest: enable tests for virtio-scmi

2023-07-19 Thread Fabiano Rosas
Thomas Huth writes: > On 18/07/2023 14.55, Milan Zamazal wrote: >> Thomas Huth writes: >> >>> On 11/07/2023 01.02, Michael S. Tsirkin wrote: From: Milan Zamazal We don't have a virtio-scmi implementation in QEMU and only support >>> a vhost-user backend. This is very simil

Re: [PATCH] hw/virtio: Add a protection against duplicate vu_scmi_stop calls

2023-07-20 Thread Fabiano Rosas
t_vector_notifiers: Assertion `dev->msix_vector_use_notifier && > dev->msix_vector_release_notifier' failed. > ../../devel/qemu/tests/qtest/libqtest.c:200: kill_qemu() detected QEMU > death from signal 6 (Aborted) (core dumped) > > As discovered by F

Re: [PATCH 02/26] migration/multifd: Protect accesses to migration_threads

2023-07-24 Thread Fabiano Rosas
Juan Quintela writes: > From: Fabiano Rosas > > This doubly linked list is common for all the multifd and migration > threads so we need to avoid concurrent access. > > Add a mutex to protect the data from concurrent access. This fixes a > crash when removing two Migrati

Re: [PATCH v2 0/7] migration: Better error handling in return path thread

2023-07-24 Thread Fabiano Rosas
Peter Xu writes: > v2: > - Patch "migration: Provide explicit error message for file shutdowns" > - Touched up qapi doc [Fabiano] > - Added Bugzilla link to commit which I didn't even notice that I was > fixing a bug.. but rightfully pointed out by Laszlo. > - Moved it to the 1st patch

[PATCH] migration/ram: Refactor precopy ram loading code

2023-07-25 Thread Fabiano Rosas
From: Nikolay Borisov Extract the ramblock parsing code into a routine that operates on the sequence of headers from the stream and another the parses the individual ramblock. This makes ram_load_precopy() easier to comprehend. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas

Re: [PATCH v2 0/7] migration: Better error handling in return path thread

2023-07-25 Thread Fabiano Rosas
Fabiano Rosas writes: > Peter Xu writes: > >> v2: >> - Patch "migration: Provide explicit error message for file shutdowns" >> - Touched up qapi doc [Fabiano] >> - Added Bugzilla link to commit which I didn't even notice that I was >>

[PATCH 3/3] migration: Replace the return path retry logic

2023-07-28 Thread Fabiano Rosas
ish before creating a new one with the updated 'from_dst_file'. Signed-off-by: Fabiano Rosas --- migration/migration.c | 72 +++ migration/migration.h | 1 - 2 files changed, 25 insertions(+), 48 deletions(-) diff --git a/migration/migration.c b/mig

[PATCH 2/3] migration: Simplify calling of await_return_path_close_on_source

2023-07-28 Thread Fabiano Rosas
We're about to reuse this function so move the 'rp_thread_created' check into it and remove the redundant tracing and comment. Add a new tracepoint akin to what is already done at migration_completion(). Signed-off-by: Fabiano Rosas --- migration/mi

[PATCH 1/3] migration: Stop marking RP bad after shutdown

2023-07-28 Thread Fabiano Rosas
patches to wait on the thread during a paused migration. Signed-off-by: Fabiano Rosas --- migration/migration.c | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 91bba630a8..051067f8c5 100644 --- a/migration/migration.c +++ b/migration/m

[PATCH 0/3] Fix segfault on migration return path

2023-07-28 Thread Fabiano Rosas
CI run: https://gitlab.com/farosas/qemu/-/pipelines/947875609 Fabiano Rosas (3): migration: Stop marking RP bad after shutdown migration: Simplify calling of await_return_path_close_on_source migration: Replace the return path retry logic migration/mi

Re: [PATCH 2/3] migration: Simplify calling of await_return_path_close_on_source

2023-07-31 Thread Fabiano Rosas
Peter Xu writes: > On Fri, Jul 28, 2023 at 09:15:15AM -0300, Fabiano Rosas wrote: >> We're about to reuse this function so move the 'rp_thread_created' >> check into it and remove the redundant tracing and comment. >> >> Add a new

Re: [PATCH 3/3] migration: Replace the return path retry logic

2023-07-31 Thread Fabiano Rosas
Peter Xu writes: > On Fri, Jul 28, 2023 at 09:15:16AM -0300, Fabiano Rosas wrote: >> Replace the return path retry logic with finishing and restarting the >> thread. This fixes a race when resuming the migration that leads to a >> segfault. >> >> Currently whe

[PATCH] block-migration: Ensure we don't crash during migration cleanup

2023-07-31 Thread Fabiano Rosas
095 #2 0x55bbae13 in block_migration_cleanup_bmds () at ../migration/block.c:690 Signed-off-by: Fabiano Rosas --- migration/block.c | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/block.c b/migration/block.c index b9580a6c7e..86c2256a2b 100644 --- a/mig

Re: [PATCH 1/3] migration: Stop marking RP bad after shutdown

2023-07-31 Thread Fabiano Rosas
Peter Xu writes: > On Fri, Jul 28, 2023 at 09:15:14AM -0300, Fabiano Rosas wrote: >> When waiting for the return path (RP) thread to finish, there is >> really nothing wrong in the RP if the destination end of the migration >> stops responding, leaving it stuck. >> &g

Re: [PATCH 1/3] migration: Stop marking RP bad after shutdown

2023-07-31 Thread Fabiano Rosas
Fabiano Rosas writes: > Peter Xu writes: > >> On Fri, Jul 28, 2023 at 09:15:14AM -0300, Fabiano Rosas wrote: >>> When waiting for the return path (RP) thread to finish, there is >>> really nothing wrong in the RP if the destination end of the migration >>

[PATCH v2 2/2] migration: Replace the return path retry logic

2023-08-02 Thread Fabiano Rosas
ish before creating a new one with the updated 'from_dst_file'. Signed-off-by: Fabiano Rosas --- migration/migration.c | 64 +++--- migration/migration.h | 1 - migration/trace-events | 1 + 3 files changed, 17 insertions(+), 49 deletions(-) diff -

[PATCH v2 0/2] Fix segfault on migration return path

2023-08-02 Thread Fabiano Rosas
, but I find it more straight-forward to move the retry logic outside of the thread by letting it finish and starting a new thread when resuming the migration. More details on the commit message. CI run: https://gitlab.com/farosas/qemu/-/pipelines/947875609 Fabiano Rosas (2)

[PATCH v2 1/2] migration: Split await_return_path_close_on_source

2023-08-02 Thread Fabiano Rosas
ove this more complicated part of the code to a separate routine so we can wait on the thread without all of this baggage. Signed-off-by: Fabiano Rosas --- migration/migration.c | 46 +-- 1 file changed, 27 insertions(+), 19 deletions(-) diff --gi

Re: [PATCH v2 1/2] migration: Split await_return_path_close_on_source

2023-08-02 Thread Fabiano Rosas
Peter Xu writes: > On Wed, Aug 02, 2023 at 11:36:43AM -0300, Fabiano Rosas wrote: >> This function currently has a straight-forward part which is waiting >> for the thread to join and a complicated part which is doing a >> qemu_file_shutdown() on the return path file. &

Re: [PATCH v2 2/2] migration: Replace the return path retry logic

2023-08-02 Thread Fabiano Rosas
Peter Xu writes: >> +if (await_return_path_close_on_source(s)) { >> +trace_migration_return_path_pause_err(); >> +return MIG_THR_ERR_FATAL; >> +} > > I see that here on return path failures we'll bail out, and actually it's > against the instinction (that w

Re: [PATCH v2 1/2] migration: Split await_return_path_close_on_source

2023-08-03 Thread Fabiano Rosas
Peter Xu writes: > On Wed, Aug 02, 2023 at 04:58:38PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Wed, Aug 02, 2023 at 11:36:43AM -0300, Fabiano Rosas wrote: >> >> This function currently has a straight-forward part which is waiting >> &

Re: [PATCH v2 2/2] migration: Replace the return path retry logic

2023-08-03 Thread Fabiano Rosas
Peter Xu writes: > On Wed, Aug 02, 2023 at 05:04:45PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> >> +if (await_return_path_close_on_source(s)) { >> >> +trace_migration_return_path_pause_err(); >

Re: [PATCH v4 19/34] migration/multifd: Allow receiving pages without packets

2024-02-26 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Feb 20, 2024 at 07:41:23PM -0300, Fabiano Rosas wrote: >> Currently multifd does not need to have knowledge of pages on the >> receiving side because all the information needed is within the >> packets that come in the stream. >> >> We

[PATCH 1/2] migration: Join the return path thread before releasing to_dst_file

2024-02-26 Thread Fabiano Rosas
Goater Tested-by: Cédric Le Goater Signed-off-by: Fabiano Rosas --- migration/migration.c | 22 +- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index ab21de2cad..3161be7cde 100644 --- a/migration/migration.c +++ b

[PATCH 0/2] migration: Fix RP shutdown order

2024-02-26 Thread Fabiano Rosas
_error() in close_return_path_on_source() Fabiano Rosas (1): migration: Join the return path thread before releasing to_dst_file migration/migration.c | 25 ++--- 1 file changed, 10 insertions(+), 15 deletions(-) -- 2.35.3

[PATCH 2/2] migration: Use migrate_has_error() in close_return_path_on_source()

2024-02-26 Thread Fabiano Rosas
ter Reviewed-by: Peter Xu [added some words about the motivation for this patch] Signed-off-by: Fabiano Rosas --- migration/migration.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 3161be7cde..5316bbe670 100644 --- a/m

Re: [PATCH v4 19/34] migration/multifd: Allow receiving pages without packets

2024-02-26 Thread Fabiano Rosas
Fabiano Rosas writes: > Peter Xu writes: > >> On Tue, Feb 20, 2024 at 07:41:23PM -0300, Fabiano Rosas wrote: >>> Currently multifd does not need to have knowledge of pages on the >>> receiving side because all the information needed is within the >&

Re: [PATCH v4 22/34] migration/multifd: Prepare multifd sync for fixed-ram migration

2024-02-26 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Feb 20, 2024 at 07:41:26PM -0300, Fabiano Rosas wrote: >> The fixed-ram migration can be performed live or non-live, but it is >> always asynchronous, i.e. the source machine and the destination >> machine are not migrating at the same time. We

Re: [PATCH v4 22/34] migration/multifd: Prepare multifd sync for fixed-ram migration

2024-02-27 Thread Fabiano Rosas
Peter Xu writes: > On Mon, Feb 26, 2024 at 07:52:20PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Tue, Feb 20, 2024 at 07:41:26PM -0300, Fabiano Rosas wrote: >> >> The fixed-ram migration can be performed live or non-live, but it is >&g

[PATCH v5 02/23] io: add and implement QIO_CHANNEL_FEATURE_SEEKABLE for channel file

2024-02-28 Thread Fabiano Rosas
From: Nikolay Borisov Add a generic QIOChannel feature SEEKABLE which would be used by the qemu_file* apis. For the time being this will be only implemented for file channels. Signed-off-by: Nikolay Borisov Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas

[PATCH v5 07/23] migration/ram: Introduce 'mapped-ram' migration capability

2024-02-28 Thread Fabiano Rosas
n parallel. For now, enabling the capability has no effect. The next couple of patches implement the core functionality. Acked-by: Markus Armbruster Signed-off-by: Fabiano Rosas --- - s/fixed-ram/mapped-ram/ - mentioned VRAM in docs --- docs/devel/migration/features.rst | 1 + docs/devel/migration/

[PATCH v5 10/23] migration/ram: Add incoming 'mapped-ram' migration

2024-02-28 Thread Fabiano Rosas
ay Borisov Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- - s/mismatch/not supported/ - moved Error declaration to the top --- migration/ram.c | 143 +++- 1 file changed, 141 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/m

[PATCH v5 16/23] migration/multifd: Add a wrapper for channels_created

2024-02-28 Thread Fabiano Rosas
We'll need to access multifd_send_state->channels_created from outside multifd.c, so introduce a helper for that. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 7 ++- migration/multifd.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/migration/mul

[PATCH v5 01/23] migration/multifd: Cleanup multifd_recv_sync_main

2024-02-28 Thread Fabiano Rosas
Some minor cleanups and documentation for multifd_recv_sync_main. Use thread_count as done in other parts of the code. Remove p->id from the multifd_recv_state sync, since that is global and not tied to a channel. Add documentation for the sync steps. Signed-off-by: Fabiano Rosas --- migrat

[PATCH v5 23/23] tests/qtest/migration: Add a multifd + mapped-ram migration test

2024-02-28 Thread Fabiano Rosas
Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 68 1 file changed, 68 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 64a26009e9..a71504b262 100644 --- a/tests/qtest

[PATCH v5 21/23] migration/multifd: Support incoming mapped-ram stream format

2024-02-28 Thread Fabiano Rosas
) migrate_incoming file:migfile (the ram.h include needs to move because we've been previously relying on it being included from migration.c. Now file.h will start including multifd.h before migration.o is processed) Signed-off-by: Fabiano Rosas --- - dropped NOCOMP flag handling - dropped co

[PATCH v5 05/23] io: fsync before closing a file channel

2024-02-28 Thread Fabiano Rosas
. Reviewed-by: Daniel P. Berrangé Acked-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas --- io/channel-file.c | 5 + 1 file changed, 5 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index a6ad7770c6..d4706fa592 100644 --- a/io/channel-file.c +++ b/io/channel-file.c

[PATCH v5 04/23] io: implement io_pwritev/preadv for QIOChannelFile

2024-02-28 Thread Fabiano Rosas
ay Borisov Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- io/channel-file.c | 56 +++ 1 file changed, 56 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index f91bf6db1c..a6ad7770c6 100644 --- a/

[PATCH v5 11/23] tests/qtest/migration: Add tests for mapped-ram file-based migration

2024-02-28 Thread Fabiano Rosas
Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 59 1 file changed, 59 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 83512bce85..64a26009e9 100644 --- a/tests/qtest

[PATCH v5 00/23] migration: File based migration with multifd and mapped-ram

2024-02-28 Thread Fabiano Rosas
e.kernel.org/r/20230330180336.2791-1-faro...@suse.de Fabiano Rosas (20): migration/multifd: Cleanup multifd_recv_sync_main io: fsync before closing a file channel migration/qemu-file: add utility methods for working with seekable channels migration/ram: Introduce 'mapped-ram&#x

[PATCH v5 12/23] migration/multifd: Rename MultiFDSend|RecvParams::data to compress_data

2024-02-28 Thread Fabiano Rosas
Use a more specific name for the compression data so we can use the generic for the multifd core code. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-zlib.c | 20 ++-- migration/multifd-zstd.c | 20 ++-- migration/multifd.h | 4

[PATCH v5 20/23] migration/multifd: Support outgoing mapped-ram stream format

2024-02-28 Thread Fabiano Rosas
) migrate_set_capability multifd on (qemu) migrate_set_capability mapped-ram on (qemu) migrate_set_parameter max-bandwidth 0 (qemu) migrate_set_parameter multifd-channels 8 (qemu) migrate file:migfile Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- - added used_length sanity check - allowed

[PATCH v5 22/23] migration/multifd: Add mapped-ram support to fd: URI

2024-02-28 Thread Fabiano Rosas
hread effects when doing cleanup (i.e. close(fd)). The original fd doesn't need to be duplicated because monitor_get_fd() transfers ownership to the caller. Signed-off-by: Fabiano Rosas --- - dup() the fds that are passed to multifd. - update the commit message --- migration/fd.c

[PATCH v5 13/23] migration/multifd: Decouple recv method from pages

2024-02-28 Thread Fabiano Rosas
Next patches will abstract the type of data being received by the channels, so do some cleanup now to remove references to pages and dependency on 'normal_num'. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-zlib.c | 6 +++--- migration/multifd-z

[PATCH v5 19/23] migration/multifd: Prepare multifd sync for mapped-ram migration

2024-02-28 Thread Fabiano Rosas
any other unexpected flags are found on the stream. Signed-off-by: Fabiano Rosas --- - skipped all FLUSH flags - added invalid flags - skipped EOS --- migration/ram.c | 26 ++ 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/migration/ram.c b/migration/r

[PATCH v5 18/23] migration/multifd: Add incoming QIOChannelFile support

2024-02-28 Thread Fabiano Rosas
On the receiving side we don't need to differentiate between main channel and threads, so whichever channel is defined first gets to be the main one. And since there are no packets, use the atomic channel count to index into the params array. Signed-off-by: Fabiano Rosas --- - use

[PATCH v5 14/23] migration/multifd: Allow multifd without packets

2024-02-28 Thread Fabiano Rosas
taken from the standard migration headers with some (future) changes to multifd code. Use the presence of the mapped-ram capability to decide whether to send packets. This only moves code under multifd_use_packets(), it has no effect for now as mapped-ram cannot yet be enabled with multifd. Signed-

[PATCH v5 06/23] migration/qemu-file: add utility methods for working with seekable channels

2024-02-28 Thread Fabiano Rosas
Add utility methods that will be needed when implementing 'mapped-ram' migration capability. Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- include/migration/qemu-file-types.h | 2 + migration/qemu-file.c | 106 migr

[PATCH v5 17/23] migration/multifd: Add outgoing QIOChannelFile support

2024-02-28 Thread Fabiano Rosas
e channels. Signed-off-by: Fabiano Rosas --- - moved flags change to another patch - removed channels_created assert --- migration/file.c| 41 +++-- migration/file.h| 4 migration/multifd.c | 18 +++--- migration/multifd.h | 1 + 4 files c

[PATCH v5 08/23] migration: Add mapped-ram URI compatibility check

2024-02-28 Thread Fabiano Rosas
The mapped-ram migration format needs a channel that supports seeking to be able to write each page to an arbitrary offset in the migration stream. Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 29

[PATCH v5 15/23] migration/multifd: Allow receiving pages without packets

2024-02-28 Thread Fabiano Rosas
receiving. b) the receiving side has to be stopped to load the pages, which means we can experiment with larger granularities than page size when transferring data. Signed-off-by: Fabiano Rosas --- - added barriers similar to send side - changed recv_sync to allow reentrancy --- migra

[PATCH v5 03/23] io: Add generic pwritev/preadv interface

2024-02-28 Thread Fabiano Rosas
P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- include/io/channel.h | 82 io/channel.c | 58 +++ 2 files changed, 140 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h

[PATCH v5 09/23] migration/ram: Add outgoing 'mapped-ram' migration

2024-02-28 Thread Fabiano Rosas
Signed-off-by: Nikolay Borisov Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- include/exec/ramblock.h | 13 migration/ram.c | 131 +--- 2 files changed, 135 insertions(+), 9 deletions(-) diff --git a/include/exec/ramblock.h b/include/exec/r

Re: [PATCH v3 2/7] migration/multifd: Implement zero page transmission on the multifd thread.

2024-02-28 Thread Fabiano Rosas
Hao Xiang writes: > 1. Add zero_pages field in MultiFDPacket_t. > 2. Implements the zero page detection and handling on the multifd > threads for non-compression, zlib and zstd compression backends. > 3. Added a new value 'multifd' in ZeroPageDetection enumeration. > 4. Handle migration QEMU9.0 -

Re: [PATCH v3 3/7] migration/multifd: Implement ram_save_target_page_multifd to handle multifd version of MigrationOps::ram_save_target_page.

2024-02-28 Thread Fabiano Rosas
Hao Xiang Reviewed-by: Fabiano Rosas

Re: [PATCH v2 1/3] qtest: migration: Enhance qtest migration functions to support 'channels' argument

2024-02-28 Thread Fabiano Rosas
Het Gala writes: > On 27/02/24 1:04 am, Het Gala wrote: >> >> >> On 26/02/24 6:31 pm, Fabiano Rosas wrote: >>> Het Gala writes: >>> >>>> On 24/02/24 1:42 am, Fabiano Rosas wrote: >>>> this was the same first approach that I atte

Re: [PATCH v2 02/21] migration: Remove SaveStateHandler and LoadStateHandler typedefs

2024-02-28 Thread Fabiano Rosas
Cédric Le Goater writes: > They are only used once. > > Signed-off-by: Cédric Le Goater Reviewed-by: Fabiano Rosas

Re: [PATCH v5 19/23] migration/multifd: Prepare multifd sync for mapped-ram migration

2024-02-29 Thread Fabiano Rosas
Peter Xu writes: > On Wed, Feb 28, 2024 at 12:21:23PM -0300, Fabiano Rosas wrote: >> The mapped-ram migration can be performed live or non-live, but it is >> always asynchronous, i.e. the source machine and the destination >> machine are not migrating at the same time. We

Re: [PATCH v5 17/23] migration/multifd: Add outgoing QIOChannelFile support

2024-02-29 Thread Fabiano Rosas
Peter Xu writes: > On Thu, Feb 29, 2024 at 10:44:21AM +0800, Peter Xu wrote: >> On Wed, Feb 28, 2024 at 12:21:21PM -0300, Fabiano Rosas wrote: >> > Allow multifd to open file-backed channels. This will be used when >> > enabling the mapped-ram migration st

Re: [PATCH 14/17] migration/option: Fix missing ERRP_GUARD() for error_prepend()

2024-02-29 Thread Fabiano Rosas
he above issue. > > To fix this, add missing ERRP_GUARD() at the beginning of this function. > > [1]: Issue description in the commit message of commit ae7c80a7bd73 > ("error: New macro ERRP_GUARD()"). > > Cc: Peter Xu > Cc: Fabiano Rosas > Signed-off-by: Zhao Liu Reviewed-by: Fabiano Rosas

[PATCH v6 06/23] migration/qemu-file: add utility methods for working with seekable channels

2024-02-29 Thread Fabiano Rosas
Add utility methods that will be needed when implementing 'mapped-ram' migration capability. Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- include/migration/qemu-file-types.h | 2 + migration/qemu-file.c | 106 migr

[PATCH v6 02/23] io: add and implement QIO_CHANNEL_FEATURE_SEEKABLE for channel file

2024-02-29 Thread Fabiano Rosas
From: Nikolay Borisov Add a generic QIOChannel feature SEEKABLE which would be used by the qemu_file* apis. For the time being this will be only implemented for file channels. Signed-off-by: Nikolay Borisov Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas

[PATCH v6 20/23] migration/multifd: Support outgoing mapped-ram stream format

2024-02-29 Thread Fabiano Rosas
) migrate_set_capability multifd on (qemu) migrate_set_capability mapped-ram on (qemu) migrate_set_parameter max-bandwidth 0 (qemu) migrate_set_parameter multifd-channels 8 (qemu) migrate file:migfile Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- include/qemu/bitops.h | 13

[PATCH v6 14/23] migration/multifd: Allow multifd without packets

2024-02-29 Thread Fabiano Rosas
ed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 175 +--- 1 file changed, 114 insertions(+), 61 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 3a8520097b..8c43424c81 100644 --- a/migration/multifd.c +++ b/migr

[PATCH v6 08/23] migration: Add mapped-ram URI compatibility check

2024-02-29 Thread Fabiano Rosas
The mapped-ram migration format needs a channel that supports seeking to be able to write each page to an arbitrary offset in the migration stream. Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 29

[PATCH v6 07/23] migration/ram: Introduce 'mapped-ram' migration capability

2024-02-29 Thread Fabiano Rosas
n parallel. For now, enabling the capability has no effect. The next couple of patches implement the core functionality. Acked-by: Markus Armbruster Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- docs/devel/migration/features.rst | 1 + docs/devel/migration/

[PATCH v6 10/23] migration/ram: Add incoming 'mapped-ram' migration

2024-02-29 Thread Fabiano Rosas
ay Borisov Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/ram.c | 143 +++- 1 file changed, 141 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f807824d49..18620784c6 100644 --- a/migration/ram.c +

[PATCH v6 23/23] tests/qtest/migration: Add a multifd + mapped-ram migration test

2024-02-29 Thread Fabiano Rosas
Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 68 1 file changed, 68 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 64a26009e9..a71504b262 100644 --- a/tests/qtest

[PATCH v6 09/23] migration/ram: Add outgoing 'mapped-ram' migration

2024-02-29 Thread Fabiano Rosas
Signed-off-by: Nikolay Borisov Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- include/exec/ramblock.h | 13 migration/ram.c | 131 +--- 2 files changed, 135 insertions(+), 9 deletions(-) diff --git a/include/exec/ramblock.h b/include/exec/r

[PATCH v6 05/23] io: fsync before closing a file channel

2024-02-29 Thread Fabiano Rosas
. Reviewed-by: Daniel P. Berrangé Acked-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas --- io/channel-file.c | 5 + 1 file changed, 5 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index a6ad7770c6..d4706fa592 100644 --- a/io/channel-file.c +++ b/io/channel-file.c

[PATCH v6 18/23] migration/multifd: Add incoming QIOChannelFile support

2024-02-29 Thread Fabiano Rosas
On the receiving side we don't need to differentiate between main channel and threads, so whichever channel is defined first gets to be the main one. And since there are no packets, use the atomic channel count to index into the params array. Reviewed-by: Peter Xu Signed-off-by: Fabiano

[PATCH v6 22/23] migration/multifd: Add mapped-ram support to fd: URI

2024-02-29 Thread Fabiano Rosas
hread effects when doing cleanup (i.e. close(fd)). The original fd doesn't need to be duplicated because monitor_get_fd() transfers ownership to the caller. Signed-off-by: Fabiano Rosas --- migration/fd.c| 44 +++ migration/fd.h| 2 ++

[PATCH v6 16/23] migration/multifd: Add a wrapper for channels_created

2024-02-29 Thread Fabiano Rosas
We'll need to access multifd_send_state->channels_created from outside multifd.c, so introduce a helper for that. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 7 ++- migration/multifd.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff

[PATCH v6 15/23] migration/multifd: Allow receiving pages without packets

2024-02-29 Thread Fabiano Rosas
receiving. b) the receiving side has to be stopped to load the pages, which means we can experiment with larger granularities than page size when transferring data. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/file.c| 1 + migra

[PATCH v6 13/23] migration/multifd: Decouple recv method from pages

2024-02-29 Thread Fabiano Rosas
Next patches will abstract the type of data being received by the channels, so do some cleanup now to remove references to pages and dependency on 'normal_num'. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-zlib.c | 6 +++--- migration/multifd-z

[PATCH v6 17/23] migration/multifd: Add outgoing QIOChannelFile support

2024-02-29 Thread Fabiano Rosas
e channels. Signed-off-by: Fabiano Rosas --- - fix spurious hunk - use error_abort in qio_channel_close --- migration/file.c| 37 + migration/file.h| 4 migration/multifd.c | 18 +++--- migration/multifd.h | 1 + 4 files changed, 57 insert

[PATCH v6 21/23] migration/multifd: Support incoming mapped-ram stream format

2024-02-29 Thread Fabiano Rosas
) migrate_incoming file:migfile (the ram.h include needs to move because we've been previously relying on it being included from migration.c. Now file.h will start including multifd.h before migration.o is processed) Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/file.c

[PATCH v6 01/23] migration/multifd: Cleanup multifd_recv_sync_main

2024-02-29 Thread Fabiano Rosas
-by: Fabiano Rosas --- migration/multifd.c| 17 + migration/trace-events | 2 +- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 6c07f19af1..c7389bf833 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1182

[PATCH v6 11/23] tests/qtest/migration: Add tests for mapped-ram file-based migration

2024-02-29 Thread Fabiano Rosas
Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 59 1 file changed, 59 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 83512bce85..64a26009e9 100644 --- a/tests/qtest

[PATCH v6 19/23] migration/multifd: Prepare multifd sync for mapped-ram migration

2024-02-29 Thread Fabiano Rosas
any other unexpected flags are found on the stream. Signed-off-by: Fabiano Rosas --- - skipped the flush at ram_save_iterate - added a mapped-ram guard at find_dirty_block - moved EOS skip to the switch statement - added flush after parse_ramblocks --- migration/

[PATCH v6 00/23] migration: File based migration with multifd and mapped-ram

2024-02-29 Thread Fabiano Rosas
...@suse.de v4: https://lore.kernel.org/r/20240220224138.24759-1-faro...@suse.de v3: https://lore.kernel.org/r/20231127202612.23012-1-faro...@suse.de v2: https://lore.kernel.org/r/20231023203608.26370-1-faro...@suse.de v1: https://lore.kernel.org/r/20230330180336.2791-1-faro...@suse.de Fab

[PATCH v6 03/23] io: Add generic pwritev/preadv interface

2024-02-29 Thread Fabiano Rosas
P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- include/io/channel.h | 82 io/channel.c | 58 +++ 2 files changed, 140 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h

[PATCH v6 04/23] io: implement io_pwritev/preadv for QIOChannelFile

2024-02-29 Thread Fabiano Rosas
ay Borisov Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- io/channel-file.c | 56 +++ 1 file changed, 56 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index f91bf6db1c..a6ad7770c6 100644 --- a/

[PATCH v6 12/23] migration/multifd: Rename MultiFDSend|RecvParams::data to compress_data

2024-02-29 Thread Fabiano Rosas
Use a more specific name for the compression data so we can use the generic for the multifd core code. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-zlib.c | 20 ++-- migration/multifd-zstd.c | 20 ++-- migration/multifd.h | 4

Re: [External] Re: [PATCH v3 2/7] migration/multifd: Implement zero page transmission on the multifd thread.

2024-02-29 Thread Fabiano Rosas
Markus Armbruster writes: > Hao Xiang writes: > >> On Wed, Feb 28, 2024 at 1:50 AM Markus Armbruster wrote: >>> >>> Hao Xiang writes: >>> >>> > 1. Add zero_pages field in MultiFDPacket_t. >>> > 2. Implements the zero page detection and handling on the multifd >>> > threads for non-compression,

Re: [PATCH v2 1/3] qtest: migration: Enhance qtest migration functions to support 'channels' argument

2024-02-29 Thread Fabiano Rosas
Fabiano Rosas writes: > Het Gala writes: > >> On 27/02/24 1:04 am, Het Gala wrote: >>> >>> >>> On 26/02/24 6:31 pm, Fabiano Rosas wrote: >>>> Het Gala writes: >>>> >>>>> On 24/02/24 1:42 am, Fabiano Rosas wrote:

Re: [PATCH] migration/multifd: Document two places for mapped-ram

2024-03-01 Thread Fabiano Rosas
pet...@redhat.com writes: > From: Peter Xu > > Add two documentations for mapped-ram migration on two spots that may not > be extremely clear. > > Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas

Re: [PATCH v2 1/3] qtest: migration: Enhance qtest migration functions to support 'channels' argument

2024-03-01 Thread Fabiano Rosas
Het Gala writes: > On 01/03/24 2:19 pm, Het Gala wrote: >> >> On 29/02/24 6:47 am, Fabiano Rosas wrote: >>> Het Gala writes: >>> >>>> On 27/02/24 1:04 am, Het Gala wrote: >>>>> >>>>> On 26/02/24 6:31 pm, Fabiano Rosas

Re: [PATCH v6 00/23] migration: File based migration with multifd and mapped-ram

2024-03-04 Thread Fabiano Rosas
Daniel P. Berrangé writes: > On Mon, Mar 04, 2024 at 08:35:36PM +0800, Peter Xu wrote: >> Fabiano, >> >> On Thu, Feb 29, 2024 at 12:29:54PM -0300, Fabiano Rosas wrote: >> > => guest: 128 GB RAM - 120 GB dirty - 1 vcpu in tight loop dirtying memory >> >

Re: [PATCH v4 2/7] migration/multifd: Implement zero page transmission on the multifd thread.

2024-03-04 Thread Fabiano Rosas
Peter Xu writes: > On Fri, Mar 01, 2024 at 02:28:24AM +, Hao Xiang wrote: >> -GlobalProperty hw_compat_8_2[] = {}; >> +GlobalProperty hw_compat_8_2[] = { >> +{ "migration", "zero-page-detection", "legacy"}, >> +}; > > I hope we can make it for 9.0, then this (and many rest places) can be

Re: [PATCH v4 2/7] migration/multifd: Implement zero page transmission on the multifd thread.

2024-03-04 Thread Fabiano Rosas
Fabiano Rosas writes: > Peter Xu writes: > >> On Fri, Mar 01, 2024 at 02:28:24AM +, Hao Xiang wrote: >>> -GlobalProperty hw_compat_8_2[] = {}; >>> +GlobalProperty hw_compat_8_2[] = { >>> +{ "migration", "zero-page-detection", &

Re: [PATCH v4 2/7] migration/multifd: Implement zero page transmission on the multifd thread.

2024-03-04 Thread Fabiano Rosas
Fabiano Rosas writes: > Fabiano Rosas writes: > >> Peter Xu writes: >> >>> On Fri, Mar 01, 2024 at 02:28:24AM +, Hao Xiang wrote: >>>> -GlobalProperty hw_compat_8_2[] = {}; >>>> +GlobalProperty hw_compat_8_2[] = { >

Re: [PATCH v4 2/7] migration/multifd: Implement zero page transmission on the multifd thread.

2024-03-04 Thread Fabiano Rosas
Fabiano Rosas writes: > Peter Xu writes: > >> On Fri, Mar 01, 2024 at 02:28:24AM +, Hao Xiang wrote: >>> -GlobalProperty hw_compat_8_2[] = {}; >>> +GlobalProperty hw_compat_8_2[] = { >>> +{ "migration", "zero-page-detection", &

Re: [PATCH v6 00/23] migration: File based migration with multifd and mapped-ram

2024-03-04 Thread Fabiano Rosas
Peter Xu writes: > On Mon, Mar 04, 2024 at 08:53:24PM +0800, Peter Xu wrote: >> On Mon, Mar 04, 2024 at 12:42:25PM +, Daniel P. Berrangé wrote: >> > On Mon, Mar 04, 2024 at 08:35:36PM +0800, Peter Xu wrote: >> > > Fabiano, >> > > >> >

  1   2   3   4   5   6   7   8   9   10   >