Re: [PULL 00/40] Migration 20231102 patches

2023-11-02 Thread Stefan Hajnoczi
Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes. signature.asc Description: PGP signature

Re: [PATCH v5 2/4] qcow2: add configurations for zoned format extension

2023-11-02 Thread Stefan Hajnoczi
On Mon, Oct 30, 2023 at 11:01:26PM +0800, Sam Li wrote: > Hi Eric, > > Eric Blake 于2023年10月30日周一 22:53写道: > > > > On Mon, Oct 30, 2023 at 08:18:45PM +0800, Sam Li wrote: > > > To configure the zoned format feature on the qcow2 driver, it > > > requires settings as: the device size, zone model,

Re: [PATCH v5 2/4] qcow2: add configurations for zoned format extension

2023-11-02 Thread Stefan Hajnoczi
On Mon, Oct 30, 2023 at 08:18:45PM +0800, Sam Li wrote: > To configure the zoned format feature on the qcow2 driver, it > requires settings as: the device size, zone model, zone size, > zone capacity, number of conventional zones, limits on zone > resources (max append bytes, max open zones, and

Re: [PATCH v5 2/4] qcow2: add configurations for zoned format extension

2023-11-02 Thread Stefan Hajnoczi
On Mon, Oct 30, 2023 at 08:18:45PM +0800, Sam Li wrote: > +typedef struct Qcow2ZoneListEntry { > +QLIST_ENTRY(Qcow2ZoneListEntry) exp_open_zone_entry; > +QLIST_ENTRY(Qcow2ZoneListEntry) imp_open_zone_entry; > +QLIST_ENTRY(Qcow2ZoneListEntry) closed_zone_entry; Where is

Re: IDE pending patches

2023-11-02 Thread Kevin Wolf
Am 02.11.2023 um 12:23 hat Niklas Cassel geschrieben: > Hello Philippe, Kevin, > > The QEMU 8.2 freeze is next week, > and the IDE maintainer (John) hasn't been replying to emails lately. > > Kevin, considering that you picked up Fiona's series: >

Re: [PATCH v2 0/2] virtio-blk: add iothread-vq-mapping parameter

2023-11-02 Thread Kevin Wolf
Am 18.09.2023 um 18:16 hat Stefan Hajnoczi geschrieben: > virtio-blk and virtio-scsi devices need a way to specify the mapping between > IOThreads and virtqueues. At the moment all virtqueues are assigned to a > single > IOThread or the main loop. This single thread can be a CPU bottleneck, so it

Re: -drive if=none: can't we make this the default?

2023-11-02 Thread Kevin Wolf
Am 02.11.2023 um 12:01 hat Peter Maydell geschrieben: > On Thu, 2 Nov 2023 at 10:43, Kevin Wolf wrote: > > > > Am 01.11.2023 um 12:21 hat Peter Maydell geschrieben: > > > On Tue, 31 Oct 2023 at 18:45, Kevin Wolf wrote: > > > > Am 16.10.2023 um 13:58 hat Michael Tokarev geschrieben: > > > > >

Re: -drive if=none: can't we make this the default?

2023-11-02 Thread Michael Tokarev
02.11.2023 17:06, Kevin Wolf: Am 02.11.2023 um 12:01 hat Peter Maydell geschrieben: Whoops, have I got the terminology wrong again? To me these are "snapshots" (they do store the whole VM state including the current state of the disk, and "qemu-img info" lists them as "snapshots"), whereas I

Re: [PATCH] block-jobs: add final flush

2023-11-02 Thread Vladimir Sementsov-Ogievskiy
On 02.11.23 15:59, Hanna Czenczek wrote: On 01.11.23 20:53, Vladimir Sementsov-Ogievskiy wrote: On 31.10.23 17:05, Hanna Czenczek wrote: On 04.10.23 15:56, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Actually block job is not completed without the final flush.

Re: [PATCH] block-jobs: add final flush

2023-11-02 Thread Hanna Czenczek
On 01.11.23 20:53, Vladimir Sementsov-Ogievskiy wrote: On 31.10.23 17:05, Hanna Czenczek wrote: On 04.10.23 15:56, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Actually block job is not completed without the final flush. It's rather unexpected to have broken target

[PULL 36/40] migration: New migrate and migrate-incoming argument 'channels'

2023-11-02 Thread Juan Quintela
From: Het Gala MigrateChannelList allows to connect accross multiple interfaces. Add MigrateChannelList struct as argument to migration QAPIs. We plan to include multiple channels in future, to connnect multiple interfaces. Hence, we choose 'MigrateChannelList' as the new argument over

[PULL 35/40] migration: Convert the file backend to the new QAPI syntax

2023-11-02 Thread Juan Quintela
From: Fabiano Rosas Convert the file: URI to accept a FileMigrationArgs to be compatible with the new migration QAPI. Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-9-faro...@suse.de> --- migration/file.h | 9

[PULL 24/40] cpr: reboot mode

2023-11-02 Thread Juan Quintela
From: Steve Sistare Add the cpr-reboot migration mode. Usage: $ qemu-system-$arch -monitor stdio ... QEMU 8.1.50 monitor - type 'help' for more information (qemu) migrate_set_capability x-ignore-shared on (qemu) migrate_set_parameter mode cpr-reboot (qemu) migrate -d file:vm.state (qemu) info

[PULL 33/40] migration: convert rdma backend to accept MigrateAddress

2023-11-02 Thread Juan Quintela
From: Het Gala RDMA based transport backend for 'migrate'/'migrate-incoming' QAPIs accept new wire protocol of MigrateAddress struct. It is achived by parsing 'uri' string and storing migration parameters required for RDMA connection into well defined InetSocketAddress struct. Suggested-by:

[PULL 40/40] migration: modify test_multifd_tcp_none() to use new QAPI syntax.

2023-11-02 Thread Juan Quintela
From: Het Gala modify multifd tcp common test to incorporate the new QAPI syntax defined. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231023182053.8711-15-faro...@suse.de> ---

[PULL 34/40] migration: convert exec backend to accept MigrateAddress.

2023-11-02 Thread Juan Quintela
From: Het Gala Exec transport backend for 'migrate'/'migrate-incoming' QAPIs accept new wire protocol of MigrateAddress struct. It is achived by parsing 'uri' string and storing migration parameters required for exec connection into strList struct. Suggested-by: Aravind Retnakaran

[PULL 39/40] migration: Implement MigrateChannelList to hmp migration flow.

2023-11-02 Thread Juan Quintela
From: Het Gala Integrate MigrateChannelList with all transport backends (socket, exec and rdma) for both src and dest migration endpoints for hmp migration. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala Signed-off-by: Fabiano Rosas Message-ID:

[PULL 19/40] migration: Add tracepoints for downtime checkpoints

2023-11-02 Thread Juan Quintela
From: Peter Xu This patch is inspired by Joao Martin's patch here: https://lore.kernel.org/r/20230926161841.98464-1-joao.m.mart...@oracle.com Add tracepoints for major downtime checkpoints on both src and dst. They share the same tracepoint with a string showing its stage. Besides the

[PULL 38/40] migration: Implement MigrateChannelList to qmp migration flow.

2023-11-02 Thread Juan Quintela
From: Het Gala Integrate MigrateChannelList with all transport backends (socket, exec and rdma) for both src and dest migration endpoints for qmp migration. For current series, limit the size of MigrateChannelList to single element (single interface) as runtime check. Suggested-by: Aravind

[PULL 30/40] migration: New QAPI type 'MigrateAddress'

2023-11-02 Thread Juan Quintela
From: Het Gala This patch introduces well defined MigrateAddress struct and its related child objects. The existing argument of 'migrate' and 'migrate-incoming' QAPI - 'uri' is of type string. The current implementation follows double encoding scheme for fetching migration parameters like 'uri'

[PULL 37/40] migration: modify migration_channels_and_uri_compatible() for new QAPI syntax

2023-11-02 Thread Juan Quintela
From: Het Gala migration_channels_and_uri_compatible() check for transport mechanism suitable for multifd migration gets executed when the caller calls old uri syntax. It needs it to be run when using the modern MigrateChannel QAPI syntax too. After URI -> 'MigrateChannel' :

[PULL 31/40] migration: convert migration 'uri' into 'MigrateAddress'

2023-11-02 Thread Juan Quintela
From: Het Gala This patch parses 'migrate' and 'migrate-incoming' QAPI's 'uri' string containing migration connection related information and stores them inside well defined 'MigrateAddress' struct. Fabiano fixed for "file" transport. Suggested-by: Aravind Retnakaran Signed-off-by: Het Gala

[PULL 32/40] migration: convert socket backend to accept MigrateAddress

2023-11-02 Thread Juan Quintela
From: Het Gala Socket transport backend for 'migrate'/'migrate-incoming' QAPIs accept new wire protocol of MigrateAddress struct. It is achived by parsing 'uri' string and storing migration parameters required for socket connection into well defined SocketAddress struct. Suggested-by: Aravind

[PULL 29/40] migration: Change ram_dirty_bitmap_reload() retval to bool

2023-11-02 Thread Juan Quintela
From: Peter Xu Now we have a Error** passed into the return path thread stack, which is even clearer than an int retval. Change ram_dirty_bitmap_reload() and the callers to use a bool instead to replace errnos. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Peter Xu Reviewed-by: Juan

[PULL 25/40] tests/qtest: migration: add reboot mode test

2023-11-02 Thread Juan Quintela
From: Steve Sistare [ Maintainer note: I put the test as flaky because our CI has problems with shared memory. We will remove the flaky bits as soon as we get a solution. ] Signed-off-by: Steve Sistare Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID:

[PULL 22/40] cpr: relax blockdev migration blockers

2023-11-02 Thread Juan Quintela
From: Steve Sistare Some blockdevs block migration because they do not support sharing across hosts and/or do not support dirty bitmaps. These prohibitions do not apply if the old and new qemu processes do not run concurrently, and if new qemu starts on the same host as old, which is the case

[PULL 27/40] migration: Allow network to fail even during recovery

2023-11-02 Thread Juan Quintela
From: Peter Xu Normally the postcopy recover phase should only exist for a super short period, that's the duration when QEMU is trying to recover from an interrupted postcopy migration, during which handshake will be carried out for continuing the procedure with state changes from PAUSED ->

[PULL 20/40] migration: mode parameter

2023-11-02 Thread Juan Quintela
From: Steve Sistare Create a mode migration parameter that can be used to select alternate migration algorithms. The default mode is normal, representing the current migration algorithm, and does not need to be explicitly set. No functional change until a new mode is added, except that the

[PULL 12/40] migration: Use vmstate_register_any() for audio

2023-11-02 Thread Juan Quintela
We can have more than one audio backend. void audio_init_audiodevs(void) { AudiodevListEntry *e; QSIMPLEQ_FOREACH(e, , next) { audio_init(e->dev, _fatal); } } Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-12-quint...@redhat.com>

[PULL 17/40] migration: Add per vmstate downtime tracepoints

2023-11-02 Thread Juan Quintela
From: Peter Xu We have a bunch of savevm_section* tracepoints, they're good to analyze migration stream, but not always suitable if someone would like to analyze the migration downtime. Two major problems: - savevm_section* tracepoints are dumping all sections, we only care about the

[PULL 28/40] tests/migration-test: Add a test for postcopy hangs during RECOVER

2023-11-02 Thread Juan Quintela
From: Fabiano Rosas To do so, create two paired sockets, but make them not providing real data. Feed those fake sockets to src/dst QEMUs for recovery to let them go into RECOVER stage without going out. Test that we can always kick it out and recover again with the right ports. This patch is

[PULL 21/40] migration: per-mode blockers

2023-11-02 Thread Juan Quintela
From: Steve Sistare Extend the blocker interface so that a blocker can be registered for one or more migration modes. The existing interfaces register a blocker for all modes, and the new interfaces take a varargs list of modes. Internally, maintain a separate blocker list per mode. The same

[PULL 13/40] migration: Use vmstate_register_any() for eeprom93xx

2023-11-02 Thread Juan Quintela
We can have more than one eeprom93xx. For instance: e100_nic_realize() -> eeprom93xx_new() Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-13-quint...@redhat.com> --- hw/nvram/eeprom93xx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[PULL 26/40] migration: Refactor error handling in source return path

2023-11-02 Thread Juan Quintela
From: Peter Xu rp_state.error was a boolean used to show error happened in return path thread. That's not only duplicating error reporting (migrate_set_error), but also not good enough in that we only do error_report() and set it to true, we never can keep a history of the exact error and show

[PULL 23/40] cpr: relax vhost migration blockers

2023-11-02 Thread Juan Quintela
From: Steve Sistare vhost blocks migration if logging is not supported to track dirty memory, and vhost-user blocks it if the log cannot be saved to a shm fd. vhost-vdpa blocks migration if both hosts do not support all the device's features using a shadow VQ, for tracking requests and dirty

[PULL 15/40] migration: Set downtime_start even for postcopy

2023-11-02 Thread Juan Quintela
From: Peter Xu Postcopy calculates its downtime separately. It always sets MigrationState.downtime properly, but not MigrationState.downtime_start. Make postcopy do the same as other modes on properly recording the timestamp when the VM is going to be stopped. Drop the temporary variable in

[PULL 14/40] migration: Use vmstate_register_any() for vmware_vga

2023-11-02 Thread Juan Quintela
I have no idea if we can have more than one vmware_vga device, so play it safe. Reviewed-by: Stefan Berger Reviewed-by: Thomas Huth Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-14-quint...@redhat.com> --- hw/display/vmware_vga.c | 2 +- 1 file changed, 1 insertion(+), 1

[PULL 10/40] migration: Check in savevm_state_handler_insert for dups

2023-11-02 Thread Juan Quintela
From: Peter Xu Before finally register one SaveStateEntry, we detect for duplicated entries. This could be helpful to notify us asap instead of get silent migration failures which could be hard to diagnose. For example, this patch will generate a message like this (if without previous fixes on

[PULL 18/40] migration: migration_stop_vm() helper

2023-11-02 Thread Juan Quintela
From: Peter Xu Provide a helper for non-COLO use case of migration to stop a VM. This prepares for adding some downtime relevant tracepoints to migration, where they may or may not apply to COLO. Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID:

[PULL 03/40] hw/s390x/s390-stattrib: Simplify handling of the "migration-enabled" property

2023-11-02 Thread Juan Quintela
From: Thomas Huth There's no need for dedicated handlers here if they don't do anything special. Acked-by: David Hildenbrand Reviewed-by: Eric Farman Acked-by: Juan Quintela Signed-off-by: Thomas Huth Signed-off-by: Juan Quintela Message-ID: <20231020150554.664422-3-th...@redhat.com> ---

[PULL 11/40] migration: Improve example and documentation of vmstate_register()

2023-11-02 Thread Juan Quintela
Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-11-quint...@redhat.com> --- docs/devel/migration.rst | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index

[PULL 16/40] migration: Add migration_downtime_start|end() helpers

2023-11-02 Thread Juan Quintela
From: Peter Xu Unify the three users on recording downtimes with the same pair of helpers. Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231030163346.765724-3-pet...@redhat.com> --- migration/migration.c | 37

[PULL 09/40] migration: Hack to maintain backwards compatibility for ppc

2023-11-02 Thread Juan Quintela
Current code does: - register pre_2_10_vmstate_dummy_icp with "icp/server" and instance dependinfg on cpu number - for newer machines, it register vmstate_icp with "icp/server" name and instance 0 - now it unregisters "icp/server" for the 1st instance. This is wrong at many levels: - we

[PULL 00/40] Migration 20231102 patches

2023-11-02 Thread Juan Quintela
The following changes since commit 6c9ae1ce82b65faa3f266fd103729878cf11e07e: Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging (2023-11-01 06:58:11 +0900) are available in the Git repository at: https://gitlab.com/juan.quintela/qemu.git tags/migration-20231102-pull

[PULL 08/40] migration: Use VMSTATE_INSTANCE_ID_ANY for slirp

2023-11-02 Thread Juan Quintela
Each user network conection create a new slirp instance. We register more than one slirp instance for number 0. qemu-system-x86_64: -netdev user,id=hs1: savevm_state_handler_insert: Detected duplicate SaveStateEntry: id=slirp, instance_id=0x0 Broken pipe

[PULL 07/40] migration: Use vmstate_register_any() for isa-ide

2023-11-02 Thread Juan Quintela
Otherwise qom-test fails. ok 4 /i386/qom/x-remote qemu-system-i386: savevm_state_handler_insert: Detected duplicate SaveStateEntry: id=isa-ide, instance_id=0x0 Broken pipe ../../../../../mnt/code/qemu/full/tests/qtest/libqtest.c:195: kill_qemu() tried to terminate QEMU process but encountered

[PULL 05/40] migration: Create vmstate_register_any()

2023-11-02 Thread Juan Quintela
We have lots of cases where we are using an instance_id==0 when we should be using VMSTATE_INSTANCE_ID_ANY (-1). Basically everything that can have more than one needs to have a proper instance_id or -1 and the system will take one for it. vmstate_register_any(): We register with -1.

[PULL 06/40] migration: Use vmstate_register_any()

2023-11-02 Thread Juan Quintela
This are the easiest cases, where we were already using VMSTATE_INSTANCE_ID_ANY. Reviewed-by: Stefan Berger Signed-off-by: Juan Quintela Message-ID: <20231020090731.28701-3-quint...@redhat.com> --- backends/dbus-vmstate.c | 3 +-- backends/tpm/tpm_emulator.c | 3 +-- hw/i2c/core.c

[PULL 04/40] hw/s390x/s390-stattrib: Don't call register_savevm_live() during instance_init()

2023-11-02 Thread Juan Quintela
From: Thomas Huth We must not call register_savevm_live() from an instance_init() function (since this could be called multiple times during device introspection). Move this to the realize() function instead. Acked-by: David Hildenbrand Reviewed-by: Eric Farman Signed-off-by: Juan Quintela

[PULL 02/40] hw/s390x/s390-skeys: Don't call register_savevm_live() during instance_init()

2023-11-02 Thread Juan Quintela
From: Thomas Huth Since the instance_init() function immediately tries to set the property to "true", the s390_skeys_set_migration_enabled() tries to register a savevm handler during instance_init(). However, instance_init() functions can be called multiple times, e.g. for introspection of

[PULL 01/40] hw/ipmi: Don't call vmstate_register() from instance_init() functions

2023-11-02 Thread Juan Quintela
From: Thomas Huth instance_init() can be called multiple times, e.g. during introspection of the device. We should not install the vmstate handlers here. Do it in the realize() function instead. Signed-off-by: Thomas Huth Reviewed-by: Juan Quintela Acked-by: Corey Minyard Signed-off-by: Juan

IDE pending patches

2023-11-02 Thread Niklas Cassel
Hello Philippe, Kevin, The QEMU 8.2 freeze is next week, and the IDE maintainer (John) hasn't been replying to emails lately. Kevin, considering that you picked up Fiona's series: https://lore.kernel.org/qemu-devel/d6286ef8-6cf0-4e72-90e9-e91cef9da...@proxmox.com/ which was sent 2023-09-06, via

Re: -drive if=none: can't we make this the default?

2023-11-02 Thread Peter Maydell
On Thu, 2 Nov 2023 at 10:43, Kevin Wolf wrote: > > Am 01.11.2023 um 12:21 hat Peter Maydell geschrieben: > > On Tue, 31 Oct 2023 at 18:45, Kevin Wolf wrote: > > > Am 16.10.2023 um 13:58 hat Michael Tokarev geschrieben: > > > > Almost everyone mentions -blockdev as a replacement for -drive. > > >

Re: [PATCH v3 1/8] qemu-img: rebase: stop when reaching EOF of old backing file

2023-11-02 Thread Kevin Wolf
Am 01.11.2023 um 18:51 hat Michael Tokarev geschrieben: > 01.11.2023 18:38, Andrey Drobyshev wrote: > > Hi Michael, > > > > Since this series is already merged in master, I'm not sure whether it's > > necessary to forward this particular patch to qemu-stable, or it should > > rather be

Re: -drive if=none: can't we make this the default?

2023-11-02 Thread Kevin Wolf
Am 01.11.2023 um 12:21 hat Peter Maydell geschrieben: > On Tue, 31 Oct 2023 at 18:45, Kevin Wolf wrote: > > Am 16.10.2023 um 13:58 hat Michael Tokarev geschrieben: > > > Almost everyone mentions -blockdev as a replacement for -drive. > > > > More specifically for -drive if=none. I honestly don't

Re: [PATCH] hw/ide/ahci: fix legacy software reset

2023-11-02 Thread Marcin Juszkiewicz
W dniu 5.10.2023 o 11:53, Niklas Cassel pisze: From: Niklas Cassel Legacy software contains a standard mechanism for generating a reset to a Serial ATA device - setting the SRST (software reset) bit in the Device Control register. Serial ATA has a more robust mechanism called COMRESET, also

Re: FreeBSD 13.2 installer does not see AHCI devices on aarch64/sbsa-ref and x86-64/q35

2023-11-02 Thread Michael Tokarev
02.11.2023 13:16, Marcin Juszkiewicz: W dniu 3.10.2023 o 22:41, Niklas Cassel pisze: On Tue, Oct 03, 2023 at 08:11:39PM +0300, Michael Tokarev wrote: Were you able to take a look at what's going on here?  I wish I were able to help here but I know right to nothing about ahci emulation.. I

Re: FreeBSD 13.2 installer does not see AHCI devices on aarch64/sbsa-ref and x86-64/q35

2023-11-02 Thread Marcin Juszkiewicz
W dniu 3.10.2023 o 22:41, Niklas Cassel pisze: On Tue, Oct 03, 2023 at 08:11:39PM +0300, Michael Tokarev wrote: Were you able to take a look at what's going on here? I wish I were able to help here but I know right to nothing about ahci emulation.. I was away on a conference all last week,

Re: [PATCH v2 0/3] ide: implement simple legacy/native mode switching for PCI IDE controllers

2023-11-02 Thread Bernhard Beschow
Am 24. Oktober 2023 22:40:53 UTC schrieb Mark Cave-Ayland : >This series adds a simple implementation of legacy/native mode switching for >PCI >IDE controllers and updates the via-ide device to use it. > >The approach I take here is to add a new pci_ide_update_mode() function which >handles

Re: -drive if=none: can't we make this the default?

2023-11-02 Thread Markus Armbruster
Peter Maydell writes: [...] > Also, you can't arbitrarily change the command-line compat > requirements because of how you've chosen to (re-)implement an > option. That doesn't mean the current syntax is set in stone, but > I'm pretty sure the command line isn't at the HMP "we can change > it