Re: [libvirt PATCH 4/8] rpm: Reduce use of with_modular_daemons

2023-07-24 Thread Martin Kletzander
On Fri, Jul 14, 2023 at 04:39:38PM +0200, Andrea Bolognani wrote: The current implementation pretty much assumes that targets where modular daemons are the default will stick with that configuration, as will targets where they're not, or that changes to these defaults will be performed by the

Re: [libvirt PATCH 3/8] rpm: Reorder scriptlets

2023-07-24 Thread Martin Kletzander
On Fri, Jul 14, 2023 at 04:39:37PM +0200, Andrea Bolognani wrote: To make things more readable, use the same order (%pre, %post, %posttrans, %preun, %postun) everywhere. Signed-off-by: Andrea Bolognani Reviewed-by: Martin Kletzander signature.asc Description: PGP signature

Re: [libvirt PATCH 2/8] rpm: Style/alignment tweaks

2023-07-24 Thread Martin Kletzander
On Fri, Jul 14, 2023 at 04:39:36PM +0200, Andrea Bolognani wrote: Signed-off-by: Andrea Bolognani Reviewed-by: Martin Kletzander signature.asc Description: PGP signature

Re: [libvirt PATCH 1/8] rpm: Bump min_fedora

2023-07-24 Thread Martin Kletzander
On Fri, Jul 14, 2023 at 04:39:35PM +0200, Andrea Bolognani wrote: According to our platform support policy, now that Fedora 38 is out we no longer target Fedora 36 and older. This allows us to simplify a few conditionals. Signed-off-by: Andrea Bolognani Reviewed-by: Martin Kletzander

Re: [libvirt PATCH v2 05/24] snapshot_conf: add new argument to virDomainSnapshotAlignDisks

2023-07-24 Thread Peter Krempa
Preferrably: virDomainSnapshotAlignDisks: Allow overriding user-configured snapshot default Reply-To: In-Reply-To: <322f30a38fa3a4e7a6ef9b1161a07da4b74f0d6e.1687877087.git.phrd...@redhat.com> On Tue, Jun 27, 2023 at 17:07:08 +0200, Pavel Hrdina wrote: > This new option will be used by

Re: [PATCH 02/26] migration/multifd: Protect accesses to migration_threads

2023-07-24 Thread Fabiano Rosas
Juan Quintela writes: > From: Fabiano Rosas > > This doubly linked list is common for all the multifd and migration > threads so we need to avoid concurrent access. > > Add a mutex to protect the data from concurrent access. This fixes a > crash when removing two MigrationThread objects from

Re: [libvirt PATCH 0/6] Sync cpu features with qemu

2023-07-24 Thread Michal Prívozník
On 7/13/23 13:45, Tim Wiederhake wrote: > This brings libvirt in sync qith qemu commit > 6f05a92ddc73ac8aa16cfd6188f907b30b0501e3. > > Tim Wiederhake (6): > cpu_map: Add missing feature "mcdt-no" > cpu_map: Add missing feature "sbdr-ssdp-no" > cpu_map: Add missing feature "fbsdp-no" >

Re: [PATCH 00/26] Migration PULL 2023-07-24

2023-07-24 Thread Thomas Huth
On 24/07/2023 15.06, Juan Quintela wrote: Hi This is the migration PULL request. Maybe it would better to use "PULL" instead of "PATCH" in the subject? Now a not on CI, thas has been really bad. After too many problems with last PULLS, I decided to learn to use qemu CI. On one hand, it is

[PATCH 23/26] qemu_file: Make qemu_file_is_writable() static

2023-07-24 Thread Juan Quintela
It is not used outside of qemu_file, and it shouldn't. Signed-off-by: Juan Quintela Message-ID: <20230530183941.7223-19-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 1 - migration/qemu-file.c | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git

[PATCH 18/26] migration: Update error description whenever migration fails

2023-07-24 Thread Juan Quintela
From: Tejus GK There are places in migration.c where the migration is marked failed with MIGRATION_STATUS_FAILED, but the failure reason is never updated. Hence libvirt doesn't know why the migration failed when it queries for it. Reviewed-by: Daniel P. Berrangé Signed-off-by: Tejus GK

[PATCH 26/26] migration/rdma: Split qemu_fopen_rdma() into input/output functions

2023-07-24 Thread Juan Quintela
This is how everything else in QEMUFile is structured. As a bonus they are three less lines of code. Reviewed-by: Peter Xu Message-ID: <20230530183941.7223-17-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 1 - migration/qemu-file.c | 12

[PATCH 13/26] migration-test: Create arch_opts

2023-07-24 Thread Juan Quintela
This will contain the options needed for both source and target. Reviewed-by: Peter Xu Message-ID: <20230608224943.3877-6-quint...@redhat.com> Signed-off-by: Juan Quintela --- tests/qtest/migration-test.c | 30 +++--- 1 file changed, 15 insertions(+), 15 deletions(-)

[PATCH 20/26] qtest/migration-tests.c: use "-incoming defer" for postcopy tests

2023-07-24 Thread Juan Quintela
From: Wei Wang The Postcopy preempt capability is expected to be set before incoming starts, so change the postcopy tests to start with deferred incoming and call migrate-incoming after the cap has been set. Why the existing tests (without this patch) didn't fail? There could be two reasons: 1)

[PATCH 15/26] migration.json: Don't use space before colon

2023-07-24 Thread Juan Quintela
So all the file is consistent. Reviewed-by: Markus Armbruster Message-ID: <20230612191604.2219-1-quint...@redhat.com> Signed-off-by: Juan Quintela --- qapi/migration.json | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/qapi/migration.json

[PATCH 19/26] migration: enforce multifd and postcopy preempt to be set before incoming

2023-07-24 Thread Juan Quintela
From: Wei Wang qemu_start_incoming_migration needs to check the number of multifd channels or postcopy ram channels to configure the backlog parameter (i.e. the maximum length to which the queue of pending connections for sockfd may grow) of listen(). So enforce the usage of postcopy-preempt and

[PATCH 09/26] migration: Implement dirty-limit convergence algo

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) Implement dirty-limit convergence algo for live migration, which is kind of like auto-converge algo but using dirty-limit instead of cpu throttle to make migration convergent. Enable dirty page limit if dirty_rate_high_cnt greater than 2 when dirty-limit capability

[PATCH 12/26] migration-test: Make machine_opts regular with other options

2023-07-24 Thread Juan Quintela
Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-ID: <20230608224943.3877-5-quint...@redhat.com> --- tests/qtest/migration-test.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index

[PATCH 04/26] qapi/migration: Introduce x-vcpu-dirty-limit-period parameter

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) Introduce "x-vcpu-dirty-limit-period" migration experimental parameter, which is in the range of 1 to 1000ms and used to make dirtyrate calculation period configurable. Currently with the "x-vcpu-dirty-limit-period" varies, the total time of live migration changes, test

[PATCH 11/26] migration-test: Be consistent for ppc

2023-07-24 Thread Juan Quintela
It makes no sense that we don't have the same configuration on both sides. Reviewed-by: Laurent Vivier Message-ID: <20230608224943.3877-2-quint...@redhat.com> Signed-off-by: Juan Quintela --- tests/qtest/migration-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH 25/26] qemu-file: Make qemu_file_get_error_obj() static

2023-07-24 Thread Juan Quintela
It was not used outside of qemu_file.c anyways. Reviewed-by: Peter Xu Message-ID: <20230530183941.7223-21-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 1 - migration/qemu-file.c | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git

[PATCH 24/26] qemu-file: Simplify qemu_file_shutdown()

2023-07-24 Thread Juan Quintela
Reviewed-by: Peter Xu Message-ID: <20230530183941.7223-20-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 9a89e17924..4c577bdff8 100644 ---

[PATCH 22/26] migration: Change qemu_file_transferred to noflush

2023-07-24 Thread Juan Quintela
We do a qemu_fclose() just after that, that also does a qemu_fflush(), so remove one qemu_fflush(). Reviewed-by: Philippe Mathieu-Daudé Message-ID: <20230530183941.7223-3-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/savevm.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH 21/26] qemu-file: Rename qemu_file_transferred_ fast -> noflush

2023-07-24 Thread Juan Quintela
Fast don't say much. Noflush indicates more clearly that it is like qemu_file_transferred but without the flush. Reviewed-by: Philippe Mathieu-Daudé Message-ID: <20230530183941.7223-2-quint...@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 11 +--

[PATCH 17/26] docs/migration: Update postcopy bits

2023-07-24 Thread Juan Quintela
From: Peter Xu We have postcopy recovery but not reflected in the document, do an update for that. Add a very small section on postcopy preempt. Touch up the pagemap section, dropping the unsent map because it's already been dropped in the source code in commit 1e7cf8c323 ("migration/postcopy:

[PATCH 16/26] migration: skipped field is really obsolete.

2023-07-24 Thread Juan Quintela
Has return zero for more than 10 years. Specifically we introduced the field in 1.5.0 commit f1c72795af573b24a7da5eb52375c9aba8a37972 Author: Peter Lieven Date: Tue Mar 26 10:58:37 2013 +0100 migration: do not sent zero pages in bulk stage during bulk stage of ram migration if a

[PATCH 14/26] migration-test: machine_opts is really arch specific

2023-07-24 Thread Juan Quintela
And it needs to be in both source and target, so put it on arch_opts. Reviewed-by: Peter Xu Message-ID: <20230608224943.3877-7-quint...@redhat.com> Signed-off-by: Juan Quintela --- tests/qtest/migration-test.c | 14 +- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git

[PATCH 10/26] migration: Extend query-migrate to provide dirty page limit info

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) Extend query-migrate to provide throttle time and estimated ring full time with dirty-limit capability enabled, through which we can observe if dirty limit take effect during live migration. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Markus Armbruster Reviewed-by: Juan

[PATCH 08/26] migration: Put the detection logic before auto-converge checking

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) This commit is prepared for the implementation of dirty-limit convergence algo. The detection logic of throttling condition can apply to both auto-converge and dirty-limit algo, putting it's position before the checking logic for auto-converge feature. Signed-off-by:

[PATCH 07/26] migration: Refactor auto-converge capability logic

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) Check if block migration is running before throttling guest down in auto-converge way. Note that this modification is kind of like code clean, because block migration does not depend on auto-converge capability, so the order of checks can be adjusted. Signed-off-by: Hyman

[PATCH 05/26] qapi/migration: Introduce vcpu-dirty-limit parameters

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) Introduce "vcpu-dirty-limit" migration parameter used to limit dirty page rate during live migration. "vcpu-dirty-limit" and "x-vcpu-dirty-limit-period" are two dirty-limit-related migration parameters, which can be set before and during live migration by qmp

[PATCH 06/26] migration: Introduce dirty-limit capability

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) Introduce migration dirty-limit capability, which can be turned on before live migration and limit dirty page rate durty live migration. Introduce migrate_dirty_limit function to help check if dirty-limit capability enabled during live migration. Meanwhile, refactor

[PATCH 02/26] migration/multifd: Protect accesses to migration_threads

2023-07-24 Thread Juan Quintela
From: Fabiano Rosas This doubly linked list is common for all the multifd and migration threads so we need to avoid concurrent access. Add a mutex to protect the data from concurrent access. This fixes a crash when removing two MigrationThread objects from the list at the same time during

[PATCH 03/26] softmmu/dirtylimit: Add parameter check for hmp "set_vcpu_dirty_limit"

2023-07-24 Thread Juan Quintela
From: Hyman Huang(黄勇) dirty_rate paraemter of hmp command "set_vcpu_dirty_limit" is invalid if less than 0, so add parameter check for it. Note that this patch also delete the unsolicited help message and clean up the code. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Markus Armbruster

[PATCH 01/26] migration/multifd: Rename threadinfo.c functions

2023-07-24 Thread Juan Quintela
From: Fabiano Rosas We're about to add more functions to this file so make it use the same coding style as the rest of the code. Signed-off-by: Fabiano Rosas Reviewed-by: Juan Quintela Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Peter Xu Message-Id:

[PATCH 00/26] Migration PULL 2023-07-24

2023-07-24 Thread Juan Quintela
Hi This is the migration PULL request. It has: - Fabiano rosas trheadinfo cleanups - Hyman Huang dirtylimit changes - Part of my changes - Peter Xu documentation - Tejus updato to migration descriptions - Wei want improvements for postocpy and multifd setup Please apply. Now a not on CI, thas

Re: [libvirt PATCH 5/5] qemu: Implement support for vDPA block devices

2023-07-24 Thread Peter Krempa
As I've promised a long time ago I gave your patches some testing in regards of cooperation with blockjobs and snapshots. Since the new version of the patches was not yet posted on the list I'm replying here including my observations from testing patches from your gitlab branch: On Tue, Jun 06,

Re: [PATCH 0/2] fix use-after-free in network IO

2023-07-24 Thread Michal Prívozník
On 7/4/23 09:10, Oleg Vasilev wrote: > Found by repeatedly created and destroyed channels with guest agent. More > details in the patch. > > Oleg Vasilev (2): > net: add debug logs > remote: fix stream use-after-free > > src/remote/remote_daemon_stream.c | 13 +++-- >

[PATCH v2 1/2] qemu: Generate shorter channel target paths

2023-07-24 Thread Michal Privoznik
A device is basically an UNIX socket into guest. Whatever is sent from the host, appears in the guest and vice versa. But because of that, the length of the path to the socket is important (underscored by fact that we derive the path from domain short name). But there are still cases where we

[PATCH v2 2/2] qemu: Move channelTargetDir into stateDir

2023-07-24 Thread Michal Privoznik
For historical reasons (i.e. unknown reason) we put channel sockets into a path derived from cfg->libDir which is a path that survives host reboots (e.g. /var/lib/libvirt/...). This is not necessary and in fact for session daemon creates a longer prefix: XDG_CONFIG_HOME -> /home/user/.config

[PATCH v2 0/2] qemu: generate shorter channel target paths

2023-07-24 Thread Michal Privoznik
v2 of: https://listman.redhat.com/archives/libvir-list/2023-July/240680.html diff to v1: - Squashed some patches together - Updated comment in qemuDomainChrDefDropDefaultPath() - fixed more XMLs Michal Prívozník (2): qemu: Generate shorter channel target paths qemu: Move channelTargetDir

Re: PING 2 [PATCH 0/2] fix use-after-free in network IO

2023-07-24 Thread Oleg Vasilev
Please look at. On 13/07/2023 12:49, Oleg Vasilev wrote: On 04/07/2023 13:10, Oleg Vasilev wrote: Found by repeatedly created and destroyed channels with guest agent. More details in the patch. Oleg Vasilev (2):    net: add debug logs    remote: fix stream use-after-free  

Re: [PATCH 1/4] qemu: Generate shorter channel target paths

2023-07-24 Thread Michal Prívozník
On 7/21/23 14:27, Daniel P. Berrangé wrote: > On Wed, Jul 12, 2023 at 04:49:53PM +0200, Michal Privoznik wrote: >> A device is basically an UNIX socket into guest. >> Whatever is sent from the host, appears in the guest and vice >> versa. But because of that, the length of the path to the socket