Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-08 Thread Jim Fehlig via Devel
On 8/8/24 17:46, Jim Fehlig wrote: On 8/7/24 12:39, Daniel P. Berrangé wrote: On Wed, Aug 07, 2024 at 12:04:18PM -0600, Jim Fehlig wrote: On 8/7/24 09:45, Daniel P. Berrangé wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: The QEMU mapped-ram capability currently d

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-08 Thread Jim Fehlig via Devel
On 8/7/24 12:39, Daniel P. Berrangé wrote: On Wed, Aug 07, 2024 at 12:04:18PM -0600, Jim Fehlig wrote: On 8/7/24 09:45, Daniel P. Berrangé wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: The QEMU mapped-ram capability currently does not support directio. Fabino is

[PATCH 20/20] tools: add parallel parameter to virsh restore command

2024-08-08 Thread Jim Fehlig via Devel
From: Claudio Fontana Signed-off-by: Claudio Fontana Signed-off-by: Jim Fehlig --- docs/manpages/virsh.rst | 9 +++-- tools/virsh-domain.c| 38 ++ 2 files changed, 41 insertions(+), 6 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpage

[PATCH 19/20] tools: add parallel parameter to virsh save command

2024-08-08 Thread Jim Fehlig via Devel
From: Li Zhang Signed-off-by: Claudio Fontana Signed-off-by: Jim Fehlig --- tools/virsh-domain.c | 41 + 1 file changed, 37 insertions(+), 4 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 50e80689a2..ec0e43ae7b 100644 --- a/

[PATCH 18/20] qemu: Add support for parallel save and restore

2024-08-08 Thread Jim Fehlig via Devel
Add support for parallel save and restore by mapping libvirt's "parallel-connections" parameter to QEMU's "multifd-channels" migration parameter. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 31 --- src/qemu/qemu_migration_params.c | 25 +++

[PATCH 17/20] include: Define constants for parallel save/restore

2024-08-08 Thread Jim Fehlig via Devel
From: Claudio Fontana Add a new VIR_DOMAIN_SAVE_PARALLEL flag to the save and restore APIs, which can be used to specify the use of multiple, parallel channels for saving a domain. The number of parallel channels can be set using the VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS typed parameter. Si

[PATCH 16/20] qemu: Support O_DIRECT with mapped-ram on restore

2024-08-08 Thread Jim Fehlig via Devel
When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration.c | 11 ++- src/qemu/qemu_process.c | 30

[PATCH 15/20] qemu: Support O_DIRECT with mapped-ram on save

2024-08-08 Thread Jim Fehlig via Devel
When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 9 + src/qemu/qemu_migration.c| 32

[PATCH 14/20] qemu: Add support for mapped-ram on restore

2024-08-08 Thread Jim Fehlig via Devel
Add support for the mapped-ram migration capability on restore. Using mapped-ram with QEMU to restore an image requires the same steps as saving: - The 'mapped-ram' migration capability must be set to true - The 'multifd' migration capability must be set to true and the 'multifd-channels' migrat

[PATCH 12/20] qemu: Move creation of qemuProcessIncomingDef struct

2024-08-08 Thread Jim Fehlig via Devel
qemuProcessStartWithMemoryState() is the only caller of qemuProcessStart() that uses the qemuProcessIncomingDef struct. Move creation of the struct to qemuProcessStartWithMemoryState(). Signed-off-by: Jim Fehlig --- src/qemu/qemu_process.c | 44 - src/qemu

[PATCH 13/20] qemu: Apply migration parameters in qemuMigrationDstRun

2024-08-08 Thread Jim Fehlig via Devel
Similar to qemuMigrationSrcRun, apply migration parameters in qemuMigrationDstRun. This allows callers to create customized migration parameters, but delegates their application to the function performing the migration. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration.c | 15 +--

[PATCH 11/20] qemu: Decompose qemuSaveImageOpen

2024-08-08 Thread Jim Fehlig via Devel
Split the reading of libvirt's save image metadata from the opening of the fd that will be passed to QEMU. This provides flexibility for an upcoming patch adding mapped-ram support for restore. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 37 src/qemu/qemu_saveimage

[PATCH 10/20] qemu: Add support for mapped-ram on save

2024-08-08 Thread Jim Fehlig via Devel
Introduce support for QEMU's new mapped-ram stream format [1]. mapped-ram is enabled by default if the underlying QEMU advertises the mapped-ram migration capability. It can be disabled by changing the 'save_image_version' setting in qemu.conf to version '2'. To use mapped-ram with QEMU: - The 'ma

[PATCH 09/20] qemu: Add helper function for creating save image fd

2024-08-08 Thread Jim Fehlig via Devel
Move the code in qemuSaveImageCreate that opens, labels, and wraps the save image fd to a helper function, providing more flexibility for upcoming mapped-ram support. Signed-off-by: Jim Fehlig --- src/qemu/qemu_saveimage.c | 65 +++ 1 file changed, 45 insertio

[PATCH 08/20] qemu: conf: Add setting for save image version

2024-08-08 Thread Jim Fehlig via Devel
Add a 'save_image_version' setting to qemu.conf to control the image version when saving a VM with 'virsh save' or 'virsh managedsave'. Default to the new version 3. Signed-off-by: Jim Fehlig --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 6 ++ src/qemu/

[PATCH 07/20] qemu: QEMU_SAVE_VERSION: Bump to version 3

2024-08-08 Thread Jim Fehlig via Devel
QEMU's new mapped-ram stream format [1] is incompatible with the existing sequential stream format. An older libvirt+QEMU that does not support mapped-ram must not attempt to restore a mapped-ram saved image. Currently the only way to achieve this is to bump QEMU_SAVE_VERSION. To avoid future vers

[PATCH 06/20] qemu: Add function to get migration params for save

2024-08-08 Thread Jim Fehlig via Devel
Introduce qemuMigrationParamsForSave() to create a qemuMigrationParams object initialized with appropriate migration capabilities and parameters for a save operation. Note that mapped-ram capability also requires the multifd capability. For now, the number of multifd channels is set to 1. Future w

[PATCH 05/20] qemu: Add mapped-ram migration capability

2024-08-08 Thread Jim Fehlig via Devel
Add the mapped-ram migration capability introduced in QEMU 9.0. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 1 + src/qemu/qemu_migration_params.h | 1 + 2 files changed, 2 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index d

[PATCH 04/20] qemu: Add function to get bool value from migration params

2024-08-08 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 18 ++ src/qemu/qemu_migration_params.h | 5 + 2 files changed, 23 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index c3c9120c22..daa52269f4 100644 --- a/src/qe

[PATCH 03/20] qemu: Add function to check capability in migration params

2024-08-08 Thread Jim Fehlig via Devel
Add new function qemuMigrationParamsCapEnabled() to check if a capability is set in the caller-provided migration parameters. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 16 src/qemu/qemu_migration_params.h | 4 2 files changed, 20 insertions(+) diff

[PATCH 01/20] lib: virDomainSaveParams: Ensure absolute save path

2024-08-08 Thread Jim Fehlig via Devel
When invoking virDomainSaveParams with a relative path, the image is saved to the daemon's CWD, which in most cases is '/'. Ensure a relative path is converted to absolute before invoking the driver 'domainSaveParams' function. Signed-off-by: Jim Fehlig --- src/libvirt-domain.c | 46

[PATCH 02/20] qemu_fd: Add function to retrieve fdset ID

2024-08-08 Thread Jim Fehlig via Devel
Add new function qemuFDPassGetId() for retrieving the fdset ID of provided qemuFDPass object. Signed-off-by: Jim Fehlig --- src/qemu/qemu_fd.c | 18 ++ src/qemu/qemu_fd.h | 3 +++ 2 files changed, 21 insertions(+) diff --git a/src/qemu/qemu_fd.c b/src/qemu/qemu_fd.c index e8470

[PATCH 00/20] qemu: support mapped-ram+directio+mulitfd

2024-08-08 Thread Jim Fehlig via Devel
This series is essentially V1 of a prior RFC [1] to support QEMU's mapped-ram stream format [2] and migration capability. Along with supporting mapped-ram, it implements a design approach we discussed for supporting parallel save/restore [3]. In summary, the approach is 1. Add mapped-ram migration

[PATCH 2/3] qemuxmlconftest: Don't use soon-to-be-removed machine types

2024-08-08 Thread Peter Krempa
Latest qemu will be dropping some very old machine types (2.0 - 2.3) and some of our tests use them. As in none of the cases the test actually needs given machine type, switch them to 'pc' instead. In one case 'numavcpus-topology-mismatch' this caused switch to a more modern syntax for NUMA memory

[PATCH 1/3] qemuxmlconftest: Add test for old-style NUMA memory specification

2024-08-08 Thread Peter Krempa
This excercises the old-style NUMA memory commandline used with 5.0 and older machine types: -smp 16,sockets=2,dies=1,clusters=1,cores=4,threads=2 \ -numa node,nodeid=0,cpus=0-7,mem=107 \ -numa node,nodeid=1,cpus=8-15,mem=107 \ in contrast to the modern syntax: -smp 16,sockets=2,dies=1,clus

[PATCH 0/3] tests: qemu: Update test cases and bump 9.1 caps dump

2024-08-08 Thread Peter Krempa
Peter Krempa (3): qemuxmlconftest: Add test for old-style NUMA memory specification qemuxmlconftest: Don't use soon-to-be-removed machine types qemucapabilitiesdata: Update data for qemu-9.1 dev cycle on x86_64 .../caps_9.1.0_x86_64.replies | 3796 + .../caps

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Peter Xu
On Thu, Aug 08, 2024 at 10:47:28AM -0400, Michael S. Tsirkin wrote: > On Thu, Aug 08, 2024 at 10:15:36AM -0400, Peter Xu wrote: > > On Thu, Aug 08, 2024 at 07:12:14AM -0400, Michael S. Tsirkin wrote: > > > This is too big of a hammer. People already use what you call "cross > > > migrate" and have

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Michael S. Tsirkin
On Thu, Aug 08, 2024 at 10:15:36AM -0400, Peter Xu wrote: > On Thu, Aug 08, 2024 at 07:12:14AM -0400, Michael S. Tsirkin wrote: > > This is too big of a hammer. People already use what you call "cross > > migrate" and have for years. We are not going to stop developing > > features just because som

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Michael S. Tsirkin
On Thu, Aug 08, 2024 at 09:55:49AM -0400, Peter Xu wrote: > On Thu, Aug 08, 2024 at 08:43:22PM +0900, Akihiko Odaki wrote: > > On 2024/08/07 5:41, Peter Xu wrote: > > > On Mon, Aug 05, 2024 at 04:27:43PM +0900, Akihiko Odaki wrote: > > > > On 2024/08/04 22:08, Peter Xu wrote: > > > > > On Sun, Aug

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Peter Xu
On Thu, Aug 08, 2024 at 08:32:58PM +0900, Akihiko Odaki wrote: > On 2024/08/08 20:12, Michael S. Tsirkin wrote: > > On Thu, Aug 08, 2024 at 08:03:25PM +0900, Akihiko Odaki wrote: > > > On 2024/08/08 19:54, Michael S. Tsirkin wrote: > > > > On Thu, Aug 08, 2024 at 07:52:37PM +0900, Akihiko Odaki wro

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Peter Xu
On Thu, Aug 08, 2024 at 07:12:14AM -0400, Michael S. Tsirkin wrote: > This is too big of a hammer. People already use what you call "cross > migrate" and have for years. We are not going to stop developing > features just because someone suddenly became aware of some such bit. > If you care, you wi

Re: [PATCH 0/2] qemu: Avoid false failure when resuming post-copy migration

2024-08-08 Thread Michal Prívozník
On 8/8/24 13:41, Jiri Denemark wrote: > See 2/2 for details. > > Jiri Denemark (2): > qemu: Add support for postcopy-recover-setup migration state > qemu: Avoid false failure when resuming post-copy migration > > src/conf/virdomainjob.c | 1 + > src/conf/virdomainjob.h | 1 + > s

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Peter Xu
On Thu, Aug 08, 2024 at 08:43:22PM +0900, Akihiko Odaki wrote: > On 2024/08/07 5:41, Peter Xu wrote: > > On Mon, Aug 05, 2024 at 04:27:43PM +0900, Akihiko Odaki wrote: > > > On 2024/08/04 22:08, Peter Xu wrote: > > > > On Sun, Aug 04, 2024 at 03:49:45PM +0900, Akihiko Odaki wrote: > > > > > On 2024

Re: [PATCH] network: fix crashing "modify" option for hostname

2024-08-08 Thread Michal Prívozník
On 8/6/24 09:01, Adam Julis wrote: > The original condition caused (after adding modify option) > possibly access to not allocated memory. For consistency added > new check for multiple same records. > > Resolves: https://gitlab.com/libvirt/libvirt/-/issues/654 > Signed-off-by: Adam Julis > --- >

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Akihiko Odaki
On 2024/08/07 5:41, Peter Xu wrote: On Mon, Aug 05, 2024 at 04:27:43PM +0900, Akihiko Odaki wrote: On 2024/08/04 22:08, Peter Xu wrote: On Sun, Aug 04, 2024 at 03:49:45PM +0900, Akihiko Odaki wrote: On 2024/08/03 1:26, Peter Xu wrote: On Sat, Aug 03, 2024 at 12:54:51AM +0900, Akihiko Odaki wr

[PATCH 2/2] qemu: Avoid false failure when resuming post-copy migration

2024-08-08 Thread Jiri Denemark
Depending on timing between QEMU and libvirt an attempt to resume failed post-copy migration could immediately report a failure in post-copy phase again even though the migration actually resumed and is progressing just fine. This is caused by QEMU reporting the original migration state (i.e., pos

Re: [PATCH] vsh: Allow vshReadlineInit() to be called multiple times

2024-08-08 Thread Jiri Denemark
On Thu, Aug 08, 2024 at 14:49:12 +0200, Michal Privoznik wrote: > Thing about vshReadlineInit() is - it's called multiple times. > The first time from vshInit(), when @ctl was filled only > partially (most notably, before any argv parsing is done, hence > ctl->imode is set to false). The second tim

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Akihiko Odaki
On 2024/08/08 20:12, Michael S. Tsirkin wrote: On Thu, Aug 08, 2024 at 08:03:25PM +0900, Akihiko Odaki wrote: On 2024/08/08 19:54, Michael S. Tsirkin wrote: On Thu, Aug 08, 2024 at 07:52:37PM +0900, Akihiko Odaki wrote: On 2024/08/06 22:29, Michael S. Tsirkin wrote: On Tue, Aug 06, 2024 at 04

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Michael S. Tsirkin
On Thu, Aug 08, 2024 at 08:03:25PM +0900, Akihiko Odaki wrote: > On 2024/08/08 19:54, Michael S. Tsirkin wrote: > > On Thu, Aug 08, 2024 at 07:52:37PM +0900, Akihiko Odaki wrote: > > > On 2024/08/06 22:29, Michael S. Tsirkin wrote: > > > > On Tue, Aug 06, 2024 at 04:35:44PM +0900, Akihiko Odaki wro

Re: [PATCH 1/2] qemu: Add support for postcopy-recover-setup migration state

2024-08-08 Thread Jiri Denemark
On Thu, Aug 08, 2024 at 13:41:03 +0200, Jiri Denemark wrote: > This patch adds support for recognizing the new migration state reported > by QEMU when post-copy recovery is requested. It is not actually used > for anything yet. > > Signed-off-by: Jiri Denemark > --- > src/conf/virdomainjob.c

[PATCH 1/2] qemu: Add support for postcopy-recover-setup migration state

2024-08-08 Thread Jiri Denemark
This patch adds support for recognizing the new migration state reported by QEMU when post-copy recovery is requested. It is not actually used for anything yet. Signed-off-by: Jiri Denemark --- src/conf/virdomainjob.c | 1 + src/conf/virdomainjob.h | 1 + src/qemu/qemu_domain.h |

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Michael S. Tsirkin
On Thu, Aug 08, 2024 at 07:52:37PM +0900, Akihiko Odaki wrote: > On 2024/08/06 22:29, Michael S. Tsirkin wrote: > > On Tue, Aug 06, 2024 at 04:35:44PM +0900, Akihiko Odaki wrote: > > > On 2024/08/05 19:08, Michael S. Tsirkin wrote: > > > > On Mon, Aug 05, 2024 at 06:37:58PM +0900, Akihiko Odaki wro

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Akihiko Odaki
On 2024/08/08 19:54, Michael S. Tsirkin wrote: On Thu, Aug 08, 2024 at 07:52:37PM +0900, Akihiko Odaki wrote: On 2024/08/06 22:29, Michael S. Tsirkin wrote: On Tue, Aug 06, 2024 at 04:35:44PM +0900, Akihiko Odaki wrote: On 2024/08/05 19:08, Michael S. Tsirkin wrote: On Mon, Aug 05, 2024 at 06

Re: [PATCH v2 4/4] virtio-net: Add support for USO features

2024-08-08 Thread Akihiko Odaki
On 2024/08/06 22:29, Michael S. Tsirkin wrote: On Tue, Aug 06, 2024 at 04:35:44PM +0900, Akihiko Odaki wrote: On 2024/08/05 19:08, Michael S. Tsirkin wrote: On Mon, Aug 05, 2024 at 06:37:58PM +0900, Akihiko Odaki wrote: If cross-migrate=off, QEMU can still migrate on the same host (checkpoint

Re: [PATCH] network: NULL check for "modify" DNS-txt records

2024-08-08 Thread Michal Prívozník
On 8/6/24 09:01, Adam Julis wrote: > The "modify" command allowed to replace an existing record, now > checks for the NULL string in the new value and throw error if > found. > > Resolves: https://gitlab.com/libvirt/libvirt/-/issues/655 > Signed-off-by: Adam Julis > --- > src/conf/network_conf.c

[PATCH] vsh: Allow vshReadlineInit() to be called multiple times

2024-08-08 Thread Michal Privoznik
Thing about vshReadlineInit() is - it's called multiple times. The first time from vshInit(), when @ctl was filled only partially (most notably, before any argv parsing is done, hence ctl->imode is set to false). The second time after argv parsing, from virshInit() -> vshInitReload(). In here, ctl-

[PATCH 0/2] qemu: Avoid false failure when resuming post-copy migration

2024-08-08 Thread Jiri Denemark
See 2/2 for details. Jiri Denemark (2): qemu: Add support for postcopy-recover-setup migration state qemu: Avoid false failure when resuming post-copy migration src/conf/virdomainjob.c | 1 + src/conf/virdomainjob.h | 1 + src/qemu/qemu_domain.h | 4 src/qemu/qemu_dri

[PATCH 1/2] qemu: Improve PS/2 controller detection

2024-08-08 Thread Kamil Szczęk
Up until now, we've assumed that all x86 machines have a PS/2 controller built-in. This assumption was correct until QEMU v4.2 introduced a new x86-based machine type - microvm. Due to this assumption, a pair of unnecessary PS/2 inputs are implicitly added to all microvm domains. This patch fixes

[PATCH 2/2] qemu: Introduce the ability to disable the built-in PS/2 controller

2024-08-08 Thread Kamil Szczęk
A while back QEMU introduced a new machine option for disabling the i8042 PS/2 controller (commit 4ccd5fe22feb95137d325f422016a6473) which up until then was a built-in device included by all PC machine type descendants unconditionally. This new option allowed users to disable emulation of this cont

Re: [PATCH 1/3] Makefile: `make check` now computes env variable on the fly

2024-08-08 Thread Ariel Otilibili-Anieli
Hi Jano, On Tuesday, August 06, 2024 13:00 CEST, Ján Tomko wrote:   For repos other than the main 'libvirt' repo, we use merge requests on GitLab. I'll address all your feedback in a PR, later in the day. I used patches because the guidelines advise that (https://libvirt.org/hacking.html) On a

Re: [PATCH] util: open XML files before calling libxml2

2024-08-08 Thread Peter Krempa
On Thu, Aug 08, 2024 at 09:29:41 +0200, Peter Krempa wrote: > On Wed, Aug 07, 2024 at 15:59:52 +0100, Daniel P. Berrangé wrote: [...] > diff --git a/tests/qemuxmlconfdata/broken-xml-invalid.x86_64-latest.err > b/tests/qemuxmlconfdata/broken-xml-invalid.x86_64-latest.err > index 601f547cc6..77fe2

Re: [PATCH] util: open XML files before calling libxml2

2024-08-08 Thread Peter Krempa
On Wed, Aug 07, 2024 at 15:59:52 +0100, Daniel P. Berrangé wrote: > Libxml2 has awful error reporting behaviour when reading files. When > we fail to load a file from the test driver we see: > > $ virsh -c test:///wibble.xml > I/O warning : failed to load external entity "/wibble.xml" > erro

[PATCH] qemuxmlconftest: Add test cases for broken and missing XML files

2024-08-08 Thread Peter Krempa
Add test cases for few edge cases which excercise the XML reporting from libxml2 in anticipation of upcoming changes of behaviour. 'virschematest' must skip parsing of the broken file altogether so this patch adds infrastructure to allow that. Signed-off-by: Peter Krempa --- This patch is meant