Re: [PATCH v3 0/6] migration: check required entries and sections are loaded

2023-11-07 Thread Juan Quintela
"Michael S. Tsirkin" wrote: > On Mon, Nov 06, 2023 at 03:35:54PM +0400, marcandre.lur...@redhat.com wrote: >> From: Marc-André Lureau >> >> Hi, >> >> Surprisingly, the migration code doesn't check that required migration >> entries >> and subsections are loaded. Either optional or required

Re: [PULL 38/40] migration: Implement MigrateChannelList to qmp migration flow.

2023-11-06 Thread Juan Quintela
Peter Maydell wrote: > On Thu, 2 Nov 2023 at 11:46, Juan Quintela wrote: >> >> From: Het Gala >> >> Integrate MigrateChannelList with all transport backends >> (socket, exec and rdma) for both src and dest migration >> endpoints for qmp migration. &

[PULL 36/40] migration: New migrate and migrate-incoming argument 'channels'

2023-11-02 Thread Juan Quintela
'MigrateChannel' to make migration QAPIs future proof. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala Acked-by: Markus Armbruster Reviewed-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-10-f

[PULL 35/40] migration: Convert the file backend to the new QAPI syntax

2023-11-02 Thread Juan Quintela
From: Fabiano Rosas Convert the file: URI to accept a FileMigrationArgs to be compatible with the new migration QAPI. Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-9-faro...@suse.de> --- migration/file.h

[PULL 24/40] cpr: reboot mode

2023-11-02 Thread Juan Quintela
-volatile across reboot, such as by backing it with a dax device, but this is not enforced. The restarted qemu arguments must match those used to initially start qemu, plus the -incoming option. Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID

[PULL 33/40] migration: convert rdma backend to accept MigrateAddress

2023-11-02 Thread Juan Quintela
-by: Aravind Retnakaran Signed-off-by: Het Gala Reviewed-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-7-faro...@suse.de> --- migration/rdma.h | 6 -- migration/migration.

[PULL 40/40] migration: modify test_multifd_tcp_none() to use new QAPI syntax.

2023-11-02 Thread Juan Quintela
From: Het Gala modify multifd tcp common test to incorporate the new QAPI syntax defined. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-15-faro...@suse

[PULL 34/40] migration: convert exec backend to accept MigrateAddress.

2023-11-02 Thread Juan Quintela
-by: Het Gala Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-8-faro...@suse.de> --- migration/exec.h | 4 +-- migration/exec.c | 73 +++ migration/migration.c | 8 ++

[PULL 39/40] migration: Implement MigrateChannelList to hmp migration flow.

2023-11-02 Thread Juan Quintela
aro...@suse.de> Signed-off-by: Juan Quintela --- migration/migration.h | 3 ++- migration/migration-hmp-cmds.c | 23 +-- migration/migration.c | 5 ++--- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/migration/migration.h b/migration/migration.h

[PULL 19/40] migration: Add tracepoints for downtime checkpoints

2023-11-02 Thread Juan Quintela
then have unified prefix for all downtime relevant tracepoints. Co-developed-by: Joao Martins Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231030163346.765724-6-pet...@redhat.com> --- migration/migration.c | 16 +++- migration/s

[PULL 38/40] migration: Implement MigrateChannelList to qmp migration flow.

2023-11-02 Thread Juan Quintela
Retnakaran Signed-off-by: Het Gala Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-13-faro...@suse.de> --- migration/migration.c | 101 +++--- 1 file changed, 55 insertions(+), 46 del

[PULL 30/40] migration: New QAPI type 'MigrateAddress'

2023-11-02 Thread Juan Quintela
. Fabiano fixed for "file" transport. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala Reviewed-by: Juan Quintela Reviewed-by: Daniel P. Berrangé Acked-by: Markus Armbruster Signed-off-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-2-faro...@sus

[PULL 37/40] migration: modify migration_channels_and_uri_compatible() for new QAPI syntax

2023-11-02 Thread Juan Quintela
nel' : migration_channels_and_uri_compatible() -> migration_channels_and_transport_compatible() passes object as argument and check for valid transport mechanism. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala Reviewed-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Sign

[PULL 31/40] migration: convert migration 'uri' into 'MigrateAddress'

2023-11-02 Thread Juan Quintela
: Het Gala Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-4-faro...@suse.de> Message-ID: <20231023182053.8711-5-faro...@suse.de> --- migration/exec.h | 4 +++ migr

[PULL 32/40] migration: convert socket backend to accept MigrateAddress

2023-11-02 Thread Juan Quintela
Retnakaran Signed-off-by: Het Gala Reviewed-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-6-faro...@suse.de> --- migration/socket.h| 7 --- migration/migration.

[PULL 29/40] migration: Change ram_dirty_bitmap_reload() retval to bool

2023-11-02 Thread Juan Quintela
Quintela Signed-off-by: Juan Quintela Message-ID: <20231017202633.296756-5-pet...@redhat.com> --- migration/ram.h | 2 +- migration/migration.c | 18 +- migration/ram.c | 24 3 files changed, 22 insertions(+), 22 deletions(-) diff

[PULL 25/40] tests/qtest: migration: add reboot mode test

2023-11-02 Thread Juan Quintela
From: Steve Sistare [ Maintainer note: I put the test as flaky because our CI has problems with shared memory. We will remove the flaky bits as soon as we get a solution. ] Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <1698263069-40697

[PULL 22/40] cpr: relax blockdev migration blockers

2023-11-02 Thread Juan Quintela
for cpr. Narrow the scope of these blockers so they only apply to normal mode. They will not block cpr modes when they are added in subsequent patches. No functional change until a new mode is added. Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message

[PULL 27/40] migration: Allow network to fail even during recovery

2023-11-02 Thread Juan Quintela
eded using migrate-pause when needed. After bouncing back to PAUSED stage, one can recover again. Reported-by: Xiaohui Li Reviewed-by: Fabiano Rosas Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2111332 Reviewed-by: Juan Quintela Signed-off-by: Peter Xu Signed-off-by: Juan Qu

[PULL 20/40] migration: mode parameter

2023-11-02 Thread Juan Quintela
that the mode is shown by the 'info migrate' command. Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <1698263069-406971-2-git-send-email-steven.sist...@oracle.com> --- qapi/migration.json | 27 --- include/h

[PULL 12/40] migration: Use vmstate_register_any() for audio

2023-11-02 Thread Juan Quintela
We can have more than one audio backend. void audio_init_audiodevs(void) { AudiodevListEntry *e; QSIMPLEQ_FOREACH(e, , next) { audio_init(e->dev, _fatal); } } Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-12-quint...@redh

[PULL 17/40] migration: Add per vmstate downtime tracepoints

2023-11-02 Thread Juan Quintela
an save() super fast, but load() super slow, vice versa. We can't figure that out without tracing both. Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231030163346.765724-4-pet...@redhat.com> --- m

[PULL 28/40] tests/migration-test: Add a test for postcopy hangs during RECOVER

2023-11-02 Thread Juan Quintela
is based on Fabiano's version here: https://lore.kernel.org/r/877cowmdu0@suse.de Signed-off-by: Fabiano Rosas [peterx: write commit message, remove case 1, fix bugs, and more] Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231017202633.29675

[PULL 21/40] migration: per-mode blockers

2023-11-02 Thread Juan Quintela
Error object may be added to multiple lists. When a block is deleted, it is removed from every list, and the Error is freed. No functional change until a new mode is added. Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <1698263069-406971-3-

[PULL 13/40] migration: Use vmstate_register_any() for eeprom93xx

2023-11-02 Thread Juan Quintela
We can have more than one eeprom93xx. For instance: e100_nic_realize() -> eeprom93xx_new() Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-13-quint...@redhat.com> --- hw/nvram/eeprom93xx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion

[PULL 26/40] migration: Refactor error handling in source return path

2023-11-02 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231017202633.296756-2-pet...@redhat.com> --- migration/migration.h | 1 - migration/qemu-file.h | 1 + migration/ram.h| 5 +- migration/migration.c

[PULL 23/40] cpr: relax vhost migration blockers

2023-11-02 Thread Juan Quintela
of these blockers so they only apply to normal mode. They will not block cpr modes when they are added in subsequent patches. No functional change until a new mode is added. Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <1698263069-406971-5-git-s

[PULL 15/40] migration: Set downtime_start even for postcopy

2023-11-02 Thread Juan Quintela
in postcopy_start() along the way. Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231030163346.765724-2-pet...@redhat.com> --- migration/migration.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff

[PULL 14/40] migration: Use vmstate_register_any() for vmware_vga

2023-11-02 Thread Juan Quintela
I have no idea if we can have more than one vmware_vga device, so play it safe. Reviewed-by: Stefan Berger Reviewed-by: Thomas Huth Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-14-quint...@redhat.com> --- hw/display/vmware_vga.c | 2 +- 1 file changed, 1 insertion

[PULL 10/40] migration: Check in savevm_state_handler_insert for dups

2023-11-02 Thread Juan Quintela
Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-10-quint...@redhat.com> --- migration/savevm.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/migration/savevm.c b/migratio

[PULL 18/40] migration: migration_stop_vm() helper

2023-11-02 Thread Juan Quintela
From: Peter Xu Provide a helper for non-COLO use case of migration to stop a VM. This prepares for adding some downtime relevant tracepoints to migration, where they may or may not apply to COLO. Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID

[PULL 03/40] hw/s390x/s390-stattrib: Simplify handling of the "migration-enabled" property

2023-11-02 Thread Juan Quintela
From: Thomas Huth There's no need for dedicated handlers here if they don't do anything special. Acked-by: David Hildenbrand Reviewed-by: Eric Farman Acked-by: Juan Quintela Signed-off-by: Thomas Huth Signed-off-by: Juan Quintela Message-ID: <20231020150554.664422-3-th...@redhat.

[PULL 11/40] migration: Improve example and documentation of vmstate_register()

2023-11-02 Thread Juan Quintela
Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-11-quint...@redhat.com> --- docs/devel/migration.rst | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index be91

[PULL 16/40] migration: Add migration_downtime_start|end() helpers

2023-11-02 Thread Juan Quintela
From: Peter Xu Unify the three users on recording downtimes with the same pair of helpers. Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231030163346.765724-3-pet...@redhat.com> --- migration/migration.

[PULL 09/40] migration: Hack to maintain backwards compatibility for ppc

2023-11-02 Thread Juan Quintela
k_for_ppc() with warnings left and right that it is a hack. CC: Cedric Le Goater CC: Daniel Henrique Barboza CC: David Gibson CC: Greg Kurz Reviewed-by: Nicholas Piggin Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-8-quint...@redhat.com> --- include/migration/vmstat

[PULL 00/40] Migration 20231102 patches

2023-11-02 Thread Juan Quintela
MigrateChannelList to hmp migration flow. migration: modify test_multifd_tcp_none() to use new QAPI syntax. Juan Quintela (9): migration: Create vmstate_register_any() migration: Use vmstate_register_any() migration: Use vmstate_register_any() for isa-ide migration: Use VMSTATE_INSTANCE_ID_ANY

[PULL 08/40] migration: Use VMSTATE_INSTANCE_ID_ANY for slirp

2023-11-02 Thread Juan Quintela
/qtest/libqtest.c:195: kill_qemu() tried to terminate QEMU process but encountered exit status 1 (expected 0) Aborted (core dumped) Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-6-quint...@redhat.com> --- net/slirp.c | 5 +++-- 1 file chan

[PULL 07/40] migration: Use vmstate_register_any() for isa-ide

2023-11-02 Thread Juan Quintela
exit status 1 (expected 0) Aborted (core dumped) $ Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-4-quint...@redhat.com> --- hw/ide/isa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/ide/isa.c b/hw/ide/isa.c index 9505

[PULL 05/40] migration: Create vmstate_register_any()

2023-11-02 Thread Juan Quintela
-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-2-quint...@redhat.com> --- include/migration/vmstate.h | 17 + 1 file changed, 17 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 1af181877c..1ea9

[PULL 06/40] migration: Use vmstate_register_any()

2023-11-02 Thread Juan Quintela
This are the easiest cases, where we were already using VMSTATE_INSTANCE_ID_ANY. Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-3-quint...@redhat.com> --- backends/dbus-vmstate.c | 3 +-- backends/tpm/tpm_emulator.c | 3 +-- hw/i2c/

[PULL 04/40] hw/s390x/s390-stattrib: Don't call register_savevm_live() during instance_init()

2023-11-02 Thread Juan Quintela
From: Thomas Huth We must not call register_savevm_live() from an instance_init() function (since this could be called multiple times during device introspection). Move this to the realize() function instead. Acked-by: David Hildenbrand Reviewed-by: Eric Farman Signed-off-by: Juan Quintela

[PULL 02/40] hw/s390x/s390-skeys: Don't call register_savevm_live() during instance_init()

2023-11-02 Thread Juan Quintela
quite a bit. Acked-by: David Hildenbrand Reviewed-by: Eric Farman Acked-by: Juan Quintela Signed-off-by: Thomas Huth Signed-off-by: Juan Quintela Message-ID: <20231020150554.664422-2-th...@redhat.com> --- hw/s390x/s390-skeys.c | 36 +--- 1 file ch

[PULL 01/40] hw/ipmi: Don't call vmstate_register() from instance_init() functions

2023-11-02 Thread Juan Quintela
From: Thomas Huth instance_init() can be called multiple times, e.g. during introspection of the device. We should not install the vmstate handlers here. Do it in the realize() function instead. Signed-off-by: Thomas Huth Reviewed-by: Juan Quintela Acked-by: Corey Minyard Signed-off-by: Juan

[PULL 25/38] migration: Deprecate old compression method

2023-10-31 Thread Juan Quintela
Acked-by: Stefan Hajnoczi Acked-by: Peter Xu Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-6-quint...@redhat.com> --- docs/about/deprecated.rst | 8 + qapi/migration.json | 63 ++- mig

[PULL 24/38] migration: Deprecate block migration

2023-10-31 Thread Juan Quintela
It is obsolete. It is better to use driver-mirror with NBD instead. CC: Kevin Wolf CC: Eric Blake CC: Stefan Hajnoczi CC: Hanna Czenczek Acked-by: Stefan Hajnoczi Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-5-quint...@redhat.com> ---

[PULL 13/38] migration: Create compress_update_rates()

2023-10-31 Thread Juan Quintela
So we can move more compression_counters stuff to ram-compress.c. Create compression_counters struct to add the stuff that was on MigrationState. Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-8-quint...@redhat.

[PULL 16/38] migration: Merge flush_compressed_data() and compress_flush_data()

2023-10-31 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-11-quint...@redhat.com> --- migration/ram-compress.h | 1 - migration/ram-compress.c | 17 ++--- 2 files changed, 6 insertions(+), 12 deletions(-) diff --git a/migration/ram-compre

[PULL 31/38] qemu_file: Remove unused qemu_file_transferred()

2023-10-31 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-6-quint...@redhat.com> --- migration/qemu-file.h | 18 -- migration/qemu-file.c | 7 --- 2 files changed, 25 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-

[PULL 38/38] qemu-file: Make qemu_fflush() return errors

2023-10-31 Thread Juan Quintela
for f->last_error because qemu_fflush() returns it at the beggining of the function. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-13-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-fi

[PULL 30/38] migration: Use the number of transferred bytes directly

2023-10-31 Thread Juan Quintela
-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-5-quint...@redhat.com> --- migration/migration-stats.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 4cc989d975..1d9197b4c3

[PULL 26/38] migration: Stop migration immediately in RDMA error paths

2023-10-31 Thread Juan Quintela
(Fujitsu) Cc: Markus Armbruster Cc: Juan Quintela Cc: Fabiano Rosas Reported-by: Thomas Huth Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231024163933.516546-1-pet...@redhat.com> --- migration/ram.

[PULL 28/38] qemu_file: Use a stat64 for qemu_file_transferred

2023-10-31 Thread Juan Quintela
This way we can read it from any thread. I checked that it gives the same value as the current one. We never use two qemu_files at the same time. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-3-quint...@redhat.com> --- migration/migration-stats

[PULL 22/38] migration: migrate 'inc' command option is deprecated.

2023-10-31 Thread Juan Quintela
Use blockdev-mirror with NBD instead. Reviewed-by: Thomas Huth Acked-by: Stefan Hajnoczi Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-3-quint...@redhat.com> --- docs/about/deprecated.rst | 8 qapi/migration.json

[PULL 14/38] migration: Export send_queued_data()

2023-10-31 Thread Juan Quintela
This function is only used for compression. So we rename it as compress_send_queued_data(). We put it on ram-compress.h because we are moving it later to ram-compress.c. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-9-quint...@redhat.

[PULL 35/38] qemu-file: Simplify qemu_file_get_error()

2023-10-31 Thread Juan Quintela
If we pass a NULL error is the same that returning directly the value. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-10-quint...@redhat.com> --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migratio

[PULL 37/38] migration: Remove transferred atomic counter

2023-10-31 Thread Juan Quintela
After last commit, it is a write only variable. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-12-quint...@redhat.com> --- migration/migration-stats.h | 4 migration/multifd.c | 3 --- migration/ram.c | 1 - 3 files chan

[PULL 18/38] migration/ram: Fix compilation with -Wshadow=local

2023-10-31 Thread Juan Quintela
From: Thomas Huth Rename the variable here to avoid that it shadows a variable from the beginning of the function scope. With this change the code now successfully compiles with -Wshadow=local. Signed-off-by: Thomas Huth Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID

[PULL 15/38] migration: Move ram_flush_compressed_data() to ram-compress.c

2023-10-31 Thread Juan Quintela
As we export it, rename it compress_flush_data(). Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-10-quint...@redhat.com> --- migration/ram-compress.h | 1 + migration/ram-compress.c | 9 + migration/ram.c | 17 --

[PULL 33/38] migration: migration_transferred_bytes() don't need the QEMUFile

2023-10-31 Thread Juan Quintela
Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-8-quint...@redhat.com> --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 6 +++--- migration/migration.c | 6 +++--- 3 files changed, 7 insertions(+), 9 del

[PULL 27/38] qemu-file: Don't increment qemu_file_transferred at qemu_file_fill_buffer

2023-10-31 Thread Juan Quintela
We only call qemu_file_transferred_* on the sending side. Remove the increment at qemu_file_fill_buffer() and add asserts to qemu_file_transferred* functions. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-2-quint...@redhat.com> --- migratio

[PULL 36/38] migration: Use migration_transferred_bytes()

2023-10-31 Thread Juan Quintela
values are used to: a - present to the user b - calculate the rate_limit So a few KB here and there is not going to make a difference. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-11-quint...@redhat.com> --- migration/migration.c | 2 +- migration

[PULL 32/38] qemu-file: Remove _noflush from qemu_file_transferred_noflush()

2023-10-31 Thread Juan Quintela
qemu_file_transferred() don't exist anymore, so we can reuse the name. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-7-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 9 - migration/block.c

[PULL 34/38] migration: migration_rate_limit_reset() don't need the QEMUFile

2023-10-31 Thread Juan Quintela
Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-9-quint...@redhat.com> --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 2 +- migration/migration.c | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) diff

[PULL 06/38] migration: Rename ram_handle_compressed() to ram_handle_zero()

2023-10-31 Thread Juan Quintela
Now that we know it only handles zero, we can remove the ch parameter. Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-ID: <20231019085259.13307-3-quint...@redhat.com> --- migration/ram.h | 2 +- migration/ram.c | 10 +- migration/rdma.

[PULL 12/38] migration: Move busy++ to migrate_with_multithread

2023-10-31 Thread Juan Quintela
And now we can simplify save_compress_page(). Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-7-quint...@redhat.com> --- migration/ram-compress.c | 1 + migration/ram.c | 8 ++-- 2 files changed, 3 inse

[PULL 20/38] migration: set file error on subsection loading

2023-10-31 Thread Juan Quintela
From: Marc-André Lureau commit 13cde50889237 ("vmstate: Return error in case of error") sets QemuFile error to stop reading from it and report to the caller (checked by unit tests). We should do the same on subsection loading error. Signed-off-by: Marc-André Lureau Reviewed-by: Jua

[PULL 07/38] migration: Give one error if trying to set MULTIFD and XBZRLE

2023-10-31 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-2-quint...@redhat.com> --- migration/options.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/options.c b/migration/options.c index 42fb818956..b8c3c3218d 100644 --- a/mig

[PULL 08/38] migration: Give one error if trying to set COMPRESSION and XBZRLE

2023-10-31 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-3-quint...@redhat.com> --- migration/options.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/options.c b/migration/options.c index b8c3c3218d..37fa1cfe74 100644 --- a/mig

[PULL 17/38] migration: Rename ram_compressed_pages() to compress_ram_pages()

2023-10-31 Thread Juan Quintela
We are moving to have all functions exported from ram-compress.c to start with compress_. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-12-quint...@redhat.com> --- migration/ram-compress.h | 2 +- migration/ram-compress.c | 2 +- migration

[PULL 04/38] migration/doc: We broke backwards compatibility

2023-10-31 Thread Juan Quintela
oes a hard reboot. Acked-by: Peter Xu Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-5-quint...@redhat.com> --- docs/devel/migration.rst | 202 +++ 1 file changed, 202 insertions(+) diff --git a/docs/devel/migr

[PULL 29/38] qemu_file: total_transferred is not used anymore

2023-10-31 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-4-quint...@redhat.com> --- migration/qemu-file.c | 4 1 file changed, 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 384985f534..641ab703cc 100644 --- a/migratio

[PULL 19/38] migration: rename vmstate_save_needed->vmstate_section_needed

2023-10-31 Thread Juan Quintela
From: Marc-André Lureau The function is used on save at this point. The following commits will use it on load. Signed-off-by: Marc-André Lureau Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231024084043.2926316-5-marcandre.lur...@redhat.com> --- include/mig

[PULL 23/38] migration: migrate 'blk' command option is deprecated.

2023-10-31 Thread Juan Quintela
Use blocked-mirror with NBD instead. Acked-by: Stefan Hajnoczi Reviewed-by: Thomas Huth Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-4-quint...@redhat.com> --- docs/about/deprecated.rst | 9 + qapi/migration.json

[PULL 11/38] migration: Simplify compress_page_with_multithread()

2023-10-31 Thread Juan Quintela
Move the goto to a while true. Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-6-quint...@redhat.com> --- migration/ram-compress.c | 48 1 file changed, 24 insertions(+), 24 del

[PULL 21/38] qemu-iotests: Filter warnings about block migration being deprecated

2023-10-31 Thread Juan Quintela
Create a new filter that removes the two warnings for test 183. Reviewed-by: Hanna Czenczek Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-2-quint...@redhat.com> --- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/common.filter | 7 +++ 2 files chan

[PULL 05/38] migration: Receiving a zero page non zero is an error

2023-10-31 Thread Juan Quintela
a1febc4950f2c6232c002f401d7cd409f6fa6a88 Author: Richard Henderson Date: Mon Aug 29 11:46:14 2016 -0700 cutils: Export only buffer_is_zero Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-ID: <20231019085259.13307-2-quint...@redhat.com> --- migration

[PULL 09/38] migration: Remove save_page_use_compression()

2023-10-31 Thread Juan Quintela
After previous patch, we disable the posiblity that we use compression together with xbzrle. So we can use directly migrate_compress(). Once there, now we don't need the rs parameter, so remove it. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.1532

[PULL 10/38] migration: Make compress_data_with_multithreads return bool

2023-10-31 Thread Juan Quintela
Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-5-quint...@redhat.com> --- migration/ram-compress.h | 4 ++-- migration/ram-compress.c | 17 ++--- migration/ram.c | 3 +-- 3 files changed, 13 inse

[PULL 02/38] migration/doc: Add documentation for backwards compatiblity

2023-10-31 Thread Juan Quintela
State what are the requeriments to get migration working between qemu versions. And once there explain how one is supposed to implement a new feature/default value and not break migration. Reviewed-by: Vladimir Sementsov-Ogievskiy Acked-by: Peter Xu Signed-off-by: Juan Quintela Message-ID

[PULL 00/38] Migration 20231031 patches

2023-10-31 Thread Juan Quintela
for 20231020. - I removed vmstate_register(big problems with s390x) - I added yet more countes (juan) CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1055797950 Please apply. Thanks, Juan. Juan Quintela (34): migration/doc

[PULL 01/38] migration/doc: Add contents

2023-10-31 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-2-quint...@redhat.com> --- docs/devel/migration.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index c3e1400c0c..4d6a98ae58 100644 ---

[PULL 03/38] migration/doc: How to migrate when hosts have different features

2023-10-31 Thread Juan Quintela
Sometimes devices have different features depending of things outside of qemu. For instance the kernel. Document how to handle that cases. Acked-by: Peter Xu Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-4-quint...@redhat.com> --- docs

Re: [PULL 00/39] Migration 20231024 patches

2023-10-26 Thread Juan Quintela
Stefan Hajnoczi wrote: > On Tue, 24 Oct 2023 at 23:45, Juan Quintela wrote: >> >> The following changes since commit a95260486aa7e78d7c7194eba65cf03311ad94ad: >> >> Merge tag 'pull-tcg-20231023' of https://gitlab.com/rth7680/qemu into >> staging (2023-10-23 14

[PATCH v2 07/12] migration: migration_transferred_bytes() don't need the QEMUFile

2023-10-25 Thread Juan Quintela
Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 6 +++--- migration/migration.c | 6 +++--- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/migration/migration-stats.h b/migration

[PATCH v2 05/12] qemu_file: Remove unused qemu_file_transferred()

2023-10-25 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/qemu-file.h | 18 -- migration/qemu-file.c | 7 --- 2 files changed, 25 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index a29c37b0d0..8b71152754 100644 --- a/migration/qemu

[PATCH v2 03/12] qemu_file: total_transferred is not used anymore

2023-10-25 Thread Juan Quintela
Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/qemu-file.c | 4 1 file changed, 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 384985f534..641ab703cc 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -41,9 +41,6

[PATCH v2 01/12] qemu-file: Don't increment qemu_file_transferred at qemu_file_fill_buffer

2023-10-25 Thread Juan Quintela
We only call qemu_file_transferred_* on the sending side. Remove the increment at qemu_file_fill_buffer() and add asserts to qemu_file_transferred* functions. Signed-off-by: Juan Quintela --- migration/qemu-file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration

[PATCH v2 04/12] migration: Use the number of transferred bytes directly

2023-10-25 Thread Juan Quintela
-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/migration-stats.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 4cc989d975..1d9197b4c3 100644 --- a/migration/migration-stats.c +++ b/migration

[PATCH v2 06/12] qemu-file: Remove _noflush from qemu_file_transferred_noflush()

2023-10-25 Thread Juan Quintela
qemu_file_transferred() don't exist anymore, so we can reuse the name. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- v2: Update the documentation (thanks fabiano) --- migration/qemu-file.h | 9 - migration/block.c | 4 ++-- migration/qemu-file.c | 2 +- migration

[PATCH v2 02/12] qemu_file: Use a stat64 for qemu_file_transferred

2023-10-25 Thread Juan Quintela
This way we can read it from any thread. I checked that it gives the same value as the current one. We never use two qemu_files at the same time. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 migration/qemu-file.c | 5 +++-- 2 files

[PATCH v2 08/12] migration: migration_rate_limit_reset() don't need the QEMUFile

2023-10-25 Thread Juan Quintela
Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 2 +- migration/migration.c | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration

[PATCH v2 09/12] qemu-file: Simplify qemu_file_get_error()

2023-10-25 Thread Juan Quintela
If we pass a NULL error is the same that returning directly the value. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 0158db2a54

[PATCH v2 11/12] migration: Remove transferred atomic counter

2023-10-25 Thread Juan Quintela
After last commit, it is a write only variable. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 migration/multifd.c | 3 --- migration/ram.c | 1 - 3 files changed, 8 deletions(-) diff --git a/migration/migration-stats.h b

[PATCH v2 12/12] qemu-file: Make qemu_fflush() return errors

2023-10-25 Thread Juan Quintela
for f->last_error because qemu_fflush() returns it at the beggining of the function. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- In v2: Now that we call always qemu_fflush() for all files, we can simplify qemu_fclose() --- migration/q

[PATCH v2 10/12] migration: Use migration_transferred_bytes()

2023-10-25 Thread Juan Quintela
values are used to: a - present to the user b - calculate the rate_limit So a few KB here and there is not going to make a difference. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela --- migration/migration.c | 2 +- migration/ram.c | 6 +++--- 2 files changed, 4 insertions(+), 4

[PATCH v2 00/12] migration: Yet another round of atomic counters

2023-10-25 Thread Juan Quintela
have three atomic counters: - multifd_bytes - rdma_bytes - qemu_file_trasferred And we only need to setup one (and only one) of these each time that we sent anything. Please review. Later, Juan. Juan Quintela (12): qemu-file: Don't increment qemu_file_transferred at qemu_file_fill_buffer

Re: [PATCH 02/12] qemu_file: Use a stat64 for qemu_file_transferred

2023-10-25 Thread Juan Quintela
Eric Blake wrote: > On Tue, Oct 24, 2023 at 05:10:32PM +0200, Juan Quintela wrote: >> This way we can read it from any thread. >> I checked that it gives the same value than the current one. We never > > s/than/as/ Done >> use to qemu_files at the same time. > > s/to/two/ Done Thanks.

Re: [PATCH 01/12] qemu-file: We only call qemu_file_transferred_* on the sending side

2023-10-25 Thread Juan Quintela
Fabiano Rosas wrote: > Juan Quintela writes: > >> Remove the increase in qemu_file_fill_buffer() and add asserts to >> qemu_file_transferred* functions. > > Patch looks ok, but I would rewrite the whole commit message like this: > > Don't increment qemu_file_transf

[PATCH 02/12] qemu_file: Use a stat64 for qemu_file_transferred

2023-10-24 Thread Juan Quintela
This way we can read it from any thread. I checked that it gives the same value than the current one. We never use to qemu_files at the same time. Signed-off-by: Juan Quintela --- migration/migration-stats.h | 4 migration/qemu-file.c | 5 +++-- 2 files changed, 7 insertions(+), 2

[PATCH 03/12] qemu_file: total_transferred is not used anymore

2023-10-24 Thread Juan Quintela
Signed-off-by: Juan Quintela --- migration/qemu-file.c | 4 1 file changed, 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 384985f534..641ab703cc 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -41,9 +41,6 @@ struct QEMUFile

  1   2   3   4   5   6   7   8   9   10   >