[PATCH] libxl: Fix domain shutdown

2021-02-19 Thread Jim Fehlig
Commit fa30ee04a2 caused a regression in normal domain shutown. Initiating a shutdown from within the domain or via 'virsh shutdown' does cause the guest OS running in the domain to shutdown, but libvirt never reaps the domain so it is always shown in a running state until calling 'virsh destroy'.

Re: [PATCH v2 14/15] qemu: migration: Migrate block dirty bitmaps corresponding to checkpoints

2021-02-19 Thread Jiri Denemark
On Fri, Feb 19, 2021 at 12:58:26 +0100, Peter Krempa wrote: > Preserve block dirty bitmaps after migration with > QEMU_MONITOR_MIGRATE_NON_SHARED_(DISK|INC). > > This patch implements functions which offer the bitmaps to the > destination, check for eligibility on destination and then configure >

Re: [PATCH v2 07/15] qemu: migration_params: Add infrastructure for 'dirty-bitmaps' migration feature

2021-02-19 Thread Jiri Denemark
On Fri, Feb 19, 2021 at 12:58:19 +0100, Peter Krempa wrote: > Add the migration capability flag and the propagation of the > corresponding mapping configuration. The mapping will be produced from > the bitmaps on disk depending on both sides of the migration and the > necessity to perform merges.

Re: [PATCH v2 04/15] qemu: migration: Create qcow2 v3 images for VIR_MIGRATE_NON_SHARED_DISK

2021-02-19 Thread Jiri Denemark
On Fri, Feb 19, 2021 at 12:58:16 +0100, Peter Krempa wrote: > Use the new format when pre-creating the image for the user. Users > wishing to use the legacy format can always provide their own images or > use hared storage. s/hared/shared/ > Signed-off-by: Peter Krempa > --- >

Re: [PATCH v2 01/15] qemucapabilitiesdata: Update test data for qemu-6.0 on x86_64

2021-02-19 Thread Jiri Denemark
On Fri, Feb 19, 2021 at 12:58:13 +0100, Peter Krempa wrote: > Include the 'transform' member of 'block-bitmap-mapping'. This is based > on qemu commit v5.2.0-2208-gc79f01c945 > > Signed-off-by: Peter Krempa > --- > .../caps_6.0.0.x86_64.replies | 741 ++ >

[libvirt PATCH 1/2] domain_validate: use defines for cpu period and quota limits

2021-02-19 Thread Pavel Hrdina
Commints and <98a09ca48ed4fc011abf2aa290e02ce1b8f1bb5f> fixed the code to use defines instead of magic numbers but missed this place. Following commit changed the cpu quota limit to reflect what kernel actually allows so using the defines fixes XML validations as well. Signed-off-by: Pavel

[libvirt PATCH 2/2] docs: use proper cpu quota value in our documentation

2021-02-19 Thread Pavel Hrdina
Commit changed the cpu quota value that reflects what kernel allows but did not update our documentation. Signed-off-by: Pavel Hrdina --- docs/formatdomain.rst | 8 docs/manpages/virsh.rst | 2 +- docs/schemas/domaincommon.rng | 2 +- 3 files changed, 6 insertions(+), 6

[libvirt PATCH 0/2] followup fix for cpu quota limits

2021-02-19 Thread Pavel Hrdina
Pavel Hrdina (2): domain_validate: use defines for cpu period and quota limits docs: use proper cpu quota value in our documentation docs/formatdomain.rst | 8 docs/manpages/virsh.rst | 2 +- docs/schemas/domaincommon.rng | 2 +- src/conf/domain_validate.c| 20

[PATCH 4/4] ui, monitor: remove deprecated VNC ACL option and HMP commands

2021-02-19 Thread Daniel P . Berrangé
The VNC ACL concept has been replaced by the pluggable "authz" framework which does not use monitor commands. Signed-off-by: Daniel P. Berrangé --- docs/system/deprecated.rst | 16 --- docs/system/removed-features.rst | 13 +++ hmp-commands.hx | 76 -

[PATCH 3/4] ui: deprecate "password" option for SPICE server

2021-02-19 Thread Daniel P . Berrangé
With the new "password-secret" option, there is no reason to use the old inecure "password" option with -spice, so it can be deprecated. Signed-off-by: Daniel P. Berrangé --- docs/system/deprecated.rst | 8 qemu-options.hx| 4 ui/spice-core.c| 4 3

[PATCH 2/4] ui: introduce "password-secret" option for SPICE server

2021-02-19 Thread Daniel P . Berrangé
Currently when using SPICE the "password" option provides the password in plain text on the command line. This is insecure as it is visible to all processes on the host. As an alternative, the password can be provided separately via the monitor. This introduces a "password-secret" option which

[PATCH 1/4] ui: introduce "password-secret" option for VNC servers

2021-02-19 Thread Daniel P . Berrangé
Currently when using VNC the "password" flag turns on password based authentication. The actual password has to be provided separately via the monitor. This introduces a "password-secret" option which lets the password be provided up front. $QEMU --object secret,id=vncsec0,file=passwd.txt \

[PATCH 0/4] ui: add support for 'secret' object to provide VNC/SPICE passwords

2021-02-19 Thread Daniel P . Berrangé
This fixes a long standing limitation of the VNC/SPICE code which was unable to securely accept passswords on the CLI, instead requiring use of separate monitor commands after startup. This takes the opportunity to also remove previously deprecated ACL functionality from VNC. Daniel P. Berrangé

Re: [PATCH RESEND 12/20] libxl_driver.c: modernize libxlNodeDeviceReAttach()

2021-02-19 Thread Laine Stump
On 1/18/21 2:53 PM, Daniel Henrique Barboza wrote: Use g_auto* wherever we can and remove the 'cleanup' label. Signed-off-by: Daniel Henrique Barboza This patch is also obsoleted by 23cdab6a3de0f6336505adcb446f77a6e0628e6b --- src/libxl/libxl_driver.c | 37

Re: [PATCH RESEND 11/20] qemu_driver.c: modernize qemuNodeDeviceReAttach()

2021-02-19 Thread Laine Stump
On 1/18/21 2:53 PM, Daniel Henrique Barboza wrote: Add virObjectUnref an autoptr cleanup func for virNodeDevice, then remove all unref and free calls from qemuNodeDeviceReAttach(). Signed-off-by: Daniel Henrique Barboza This patch is obsoleted by your own commit

Re: [PATCH 4/4] qemu*xml2*test: Cache capabilities between tests

2021-02-19 Thread Jonathon Jongsma
On Fri, 19 Feb 2021 16:53:21 +0100 Peter Krempa wrote: > Invoking the XML parser every time is quite expensive. Since we have a > deep copy function for 'virQEMUCapsPtr' object, we can cache the > parsed results lazily. > > This brings significant speedup to qemuxml2argvtest: > > real

Re: [PATCH v2] qemu: Fix libvirt hang due to early TPM device stop

2021-02-19 Thread Michal Privoznik
On 2/19/21 4:57 PM, Stefan Berger wrote: This patch partially reverts commit 5cde9dee where the qemuExtDevicesStop() was moved to a location before the QEMU process is stopped. It may be alright to tear down some devices before QEMU is stopped, but it doesn't work for the external TPM (swtpm)

[PATCH v2] qemu: Fix libvirt hang due to early TPM device stop

2021-02-19 Thread Stefan Berger
This patch partially reverts commit 5cde9dee where the qemuExtDevicesStop() was moved to a location before the QEMU process is stopped. It may be alright to tear down some devices before QEMU is stopped, but it doesn't work for the external TPM (swtpm) which assumes that QEMU sends it a signal to

[PATCH 3/4] testQemuInfoSetArgs: Use curly braces in else section

2021-02-19 Thread Peter Krempa
Signed-off-by: Peter Krempa --- tests/testutilsqemu.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index c2cf3be9ac..a96c9d487a 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -779,8 +779,10 @@

[PATCH 1/4] qemuxml2argvtest: Cache QAPI schema between tests

2021-02-19 Thread Peter Krempa
It's quite wasteful to reparse the QAPI schema for each _CAPS_ test. Add a simple cache filled lazily by encountered schemas. The time saving on my box is quite significant: real0m3.318s user0m3.203s sys 0m0.107s vs real0m2.223s user0m2.134s sys 0m0.084s

[PATCH 4/4] qemu*xml2*test: Cache capabilities between tests

2021-02-19 Thread Peter Krempa
Invoking the XML parser every time is quite expensive. Since we have a deep copy function for 'virQEMUCapsPtr' object, we can cache the parsed results lazily. This brings significant speedup to qemuxml2argvtest: real0m2.234s user0m2.140s sys 0m0.089s vs. real0m1.161s user

[PATCH 2/4] testCompareXMLToArgvValidateSchema: Improve and fix helper for testing everything

2021-02-19 Thread Peter Krempa
The schema validator has a comment which allows checking all xml2argv input files for schema validity by forcing the latest schema onto files which don't have any schema. Fix it so that it works properly with the caching introduced in previous commit. Signed-off-by: Peter Krempa ---

[PATCH 0/4] qemuxml2*test: Significantly improve performance by caching qemu capabilities and qapi schema

2021-02-19 Thread Peter Krempa
This results in 1/3 of the original execution time. Peter Krempa (4): qemuxml2argvtest: Cache QAPI schema between tests testCompareXMLToArgvValidateSchema: Improve and fix helper for testing everything testQemuInfoSetArgs: Use curly braces in else section qemu*xml2*test: Cache

Re: [PATCH] qemu: Fix libvirt hang due to early TPM device stop

2021-02-19 Thread Daniel P . Berrangé
On Fri, Feb 19, 2021 at 09:49:36AM -0500, Stefan Berger wrote: > This patch partially reverts commit 5cde9dee where the qemuExtDevicesStop() > was moved to a location before the QEMU process is stopped. It may be > alright to tear down some devices before QEMU is stopped, but it doesn't work > for

[PATCH] qemu: Fix libvirt hang due to early TPM device stop

2021-02-19 Thread Stefan Berger
This patch partially reverts commit 5cde9dee where the qemuExtDevicesStop() was moved to a location before the QEMU process is stopped. It may be alright to tear down some devices before QEMU is stopped, but it doesn't work for the external TPM (swtpm) which assumes that QEMU sends it a signal to

Re: [PATCH] event-test: Properly terminate strings printed from callbacks

2021-02-19 Thread Ján Tomko
On a Friday in 2021, Kristina Hanicova wrote: Stdio was buffering strings in functions: myDomainEventBlockJobCallback, myDomainEventBlockThresholdCallback, myDomainEventMemoryFailureCallback. It caused flushing the printed strings from callbacks at the end of a run, not gradually. The solution

[PATCH] event-test: Properly terminate strings printed from callbacks

2021-02-19 Thread Kristina Hanicova
Stdio was buffering strings in functions: myDomainEventBlockJobCallback, myDomainEventBlockThresholdCallback, myDomainEventMemoryFailureCallback. It caused flushing the printed strings from callbacks at the end of a run, not gradually. The solution is to add \n at the end of each string.

[PATCH v2 14/15] qemu: migration: Migrate block dirty bitmaps corresponding to checkpoints

2021-02-19 Thread Peter Krempa
Preserve block dirty bitmaps after migration with QEMU_MONITOR_MIGRATE_NON_SHARED_(DISK|INC). This patch implements functions which offer the bitmaps to the destination, check for eligibility on destination and then configure source for the migration. Signed-off-by: Peter Krempa ---

[PATCH v2 08/15] qemu: migration_cookie: Add XML handling for setting up bitmap migration

2021-02-19 Thread Peter Krempa
In cases where we are copying the storage we need to ensure that also bitmaps are copied properly. This patch adds migration cookie XML infrastructure which will allow the migration sides reach consensus on which bitmaps to migrate. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark ---

[PATCH v2 15/15] qemu: capabilities: Enable QEMU_CAPS_INCREMENTAL_BACKUP

2021-02-19 Thread Peter Krempa
For incremental backup we need QEMU_CAPS_BLOCKDEV, QEMU_CAPS_BLOCKDEV_REOPEN, QEMU_CAPS_MIGRATION_PARAM_BLOCK_BITMAP_MAPPING. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark --- src/qemu/qemu_capabilities.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git

[PATCH v2 13/15] qemu: migration: Clean up temporary bitmaps when cancelling a migration

2021-02-19 Thread Peter Krempa
In case when the block migration job required temporary bitmaps for merging the appropriate checkpoints we need to clean them up when cancelling the job. On success we don't need to do that though as the bitmaps are just temporary thus are not written to disk. Signed-off-by: Peter Krempa

[PATCH v2 12/15] tests: qemumigrationcookie: Add testing for block dirty bitmap migration

2021-02-19 Thread Peter Krempa
Test the XML infrastructure for migration cookie element as well as the conversion to migration parameters for QMP schema validation. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark --- tests/meson.build | 2 +- .../nbd-bitmaps-xml2xml-in.xml

[PATCH v2 11/15] tests: qemustatusxml2xml: Add status XML from migration with bitmaps

2021-02-19 Thread Peter Krempa
The XML sample shows the status XML when migrating with bitmaps including the element added in previous commit. It will also be used for the migration cookie test. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark --- .../migration-out-nbd-bitmaps-in.xml | 574

[PATCH v2 10/15] qemu: domain: Store list of temporary bitmaps for migration in status XML

2021-02-19 Thread Peter Krempa
Add status XML infrastructure for storing a list of block dirty bitmaps which are temporarily used when migrating a VM with VIR_MIGRATE_NON_SHARED_DISK for cleanup after a libvirtd restart during migration. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark --- src/qemu/qemu_domain.c | 87

[PATCH v2 09/15] qemu: migration_cookie: Add helpers for transforming the cookie into migration params

2021-02-19 Thread Peter Krempa
'qemuMigrationCookieBlockDirtyBitmapsMatchDisks' maps the bitmaps from the migration cookie to actual disk objects definition pointers. 'qemuMigrationCookieBlockDirtyBitmapsToParams' converts the bitmap definitions from the migration cookie into parameters for the 'block-bitmap-mapping' migration

[PATCH v2 06/15] qemu: blockjob: Use qemuMonitorBitmapRemove for single bitmap removal

2021-02-19 Thread Peter Krempa
There's no need in the cleanup steps to invoke a transaction to delete a single bitmap. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark --- src/qemu/qemu_blockjob.c | 24 +++- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/src/qemu/qemu_blockjob.c

[PATCH v2 04/15] qemu: migration: Create qcow2 v3 images for VIR_MIGRATE_NON_SHARED_DISK

2021-02-19 Thread Peter Krempa
Use the new format when pre-creating the image for the user. Users wishing to use the legacy format can always provide their own images or use hared storage. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git

[PATCH v2 07/15] qemu: migration_params: Add infrastructure for 'dirty-bitmaps' migration feature

2021-02-19 Thread Peter Krempa
Add the migration capability flag and the propagation of the corresponding mapping configuration. The mapping will be produced from the bitmaps on disk depending on both sides of the migration and the necessity to perform merges. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration_params.c

[PATCH v2 05/15] qemu: monitor: Introduce qemuMonitorBitmapRemove

2021-02-19 Thread Peter Krempa
The non-transaction wrapper is useful for code paths which want to delete individual bitmaps or for cleanup after a failed job where we want to attempt to delete every bitmap individually to prevent a failure from cleaning up the rest. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark ---

[PATCH v2 02/15] qemu: capabilities: Introduce QEMU_CAPS_MIGRATION_PARAM_BLOCK_BITMAP_MAPPING

2021-02-19 Thread Peter Krempa
The capability represents qemu's ability to setup mappings for migrating block dirty bitmaps and is based on presence of the 'transform' property of the 'block-bitmap-mapping' property of 'migrate-set-parameters' QMP command. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark ---

[PATCH v2 00/15] qemu: migrate block bitmaps when migrating storage

2021-02-19 Thread Peter Krempa
When we are copying storage we should also preserve the block dirty bitmaps. This series implements their migration. For standard migration with shared storage we let qemu flush the bitmaps to disk and reload them on destination. There is possibility to migrate them using the migration stream too

[PATCH v2 03/15] qemu: Probe whether an image is 'qcow2 v2' from query-named-block-nodes

2021-02-19 Thread Peter Krempa
Such images don't support stuff like dirty bitmaps. Note that the synthetic test for detecting bitmaps is used as an example to prevent adding additional test cases. Signed-off-by: Peter Krempa Reviewed-by: Jiri Denemark --- src/qemu/qemu_monitor.h | 3 +++

[PATCH v2 01/15] qemucapabilitiesdata: Update test data for qemu-6.0 on x86_64

2021-02-19 Thread Peter Krempa
Include the 'transform' member of 'block-bitmap-mapping'. This is based on qemu commit v5.2.0-2208-gc79f01c945 Signed-off-by: Peter Krempa --- .../caps_6.0.0.x86_64.replies | 741 ++ .../caps_6.0.0.x86_64.xml | 22 +- 2 files changed, 441

Re: [libvirt PATCH v2 1/1] qemu_validate: Allow kvm hint-dedicated on non-passthrough VMs

2021-02-19 Thread Ján Tomko
On a Friday in 2021, Tim Wiederhake wrote: A VM defined similar to: ... ... is currently invalid, as hint-dedicated is only allowed if cpu mode is host-passthrough or maximum. This restriction is unnecessary, see https://bugzilla.redhat.com/show_bug.cgi?id=1857671 Signed-off-by: Tim

[libvirt PATCH v2 1/1] qemu_validate: Allow kvm hint-dedicated on non-passthrough VMs

2021-02-19 Thread Tim Wiederhake
A VM defined similar to: ... ... is currently invalid, as hint-dedicated is only allowed if cpu mode is host-passthrough or maximum. This restriction is unnecessary, see https://bugzilla.redhat.com/show_bug.cgi?id=1857671 Signed-off-by: Tim Wiederhake --- src/qemu/qemu_validate.c | 12

[libvirt PATCH v2 0/1] Allow kvm hint-dedicated on non-passthrough VMs

2021-02-19 Thread Tim Wiederhake
V1: https://listman.redhat.com/archives/libvir-list/2021-January/msg00756.html Changes since V1: * rebased * updated commit message Tim Wiederhake (1): qemu_validate: Allow kvm hint-dedicated on non-passthrough VMs src/qemu/qemu_validate.c | 12 +--- 1 file changed, 1 insertion(+),