Re: [pve-devel] Kernel 5.6/5.7 in Proxmox (Or - what is the lead-time for kernels to ship in Proxmox).

Hi, It seems there's there's were performance improvements and bugfixes for AMD EPYC in Linux 5.4: https://www.phoronix.com/scan.php?page=news_item=Linux-5.4-Improve-EPYC-Balance https://www.phoronix.com/scan.php?page=news_item=AMD-EPYC-Linux-5.4-Early

Re: [pve-devel] applied: [PATCH V2 pve-manager] API2: Network: display vnets in any_bridge

zz Alexandre Derumier Ingénieur système et stockage Manager Infrastructure Fixe : +33 3 59 82 20 10 125 Avenue de la république 59110 La Madeleine [ https://twitter.com/OdisoHosting ] [ https://twitter.com/mindbaz ] [ https://www.linkedin.com/company/odiso ] [

Re: [pve-devel] applied: [PATCH V2 pve-manager] API2: Network: display vnets in any_bridge

>>I also had some small testing of the gui done a bit ago, and now every time >>I install the libpve-network-perl package and thus activate the SDN stuff I >>get quite some errors/warnings logged: mmm, pvestatd look at zone/vnets deployement status. Not sure why $type is empty here. But,

[pve-devel] applied: [PATCH V2 pve-manager] API2: Network: display vnets in any_bridge

Hi Alexandre, On 4/8/20 6:14 PM, Alexandre DERUMIER wrote: > Hi Thomas, > > Any news about this patch ? > applied, mainly waited for some more stuff to bump pve-network (as this patch uses the new method over there), but scratch that :) > This is the last missing part for the beta testing. >

[pve-devel] applied: [PATCH qemu-server] Ignore version checks when using QEMU -rc releases

On 4/8/20 5:11 PM, Stefan Reiter wrote: > Upstream marks these as having a micro-version of >=90, unfortunately the > machine versions are bumped earlier so testing them is made unnecessarily > difficult, since the version checking code would abort on migrations etc... > > Signed-off-by: Stefan

Re: [pve-devel] [PATCH V2 pve-manager] API2: Network: display vnets in any_bridge

Hi Thomas, Any news about this patch ? This is the last missing part for the beta testing. I have seen a lot of users on the forum with questions with routed setup, with server at hertzer or other hosting providers. I really think that bgp-evpn will help a lot (I'll be easy to defined new

[pve-devel] [PATCH qemu-server] Ignore version checks when using QEMU -rc releases

Upstream marks these as having a micro-version of >=90, unfortunately the machine versions are bumped earlier so testing them is made unnecessarily difficult, since the version checking code would abort on migrations etc... Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 17

Re: [pve-devel] [RFC v2 manager 1/5] api: backup: add endpoint to list included guests and volumes

On 4/7/20 5:55 PM, Thomas Lamprecht wrote: > On 4/6/20 4:24 PM, Aaron Lauterer wrote: >> [...] >> +description => 'Root node of the tree object. Children represent guests, grandchildren represent volumes of that guest.', >> +properties => { >> +not_all_permissions => { >> +

Re: [pve-devel] [PATCH manager 4/4] gui: settings: Add recursive search default

comment inline On 4/2/20 1:34 PM, Dominic Jäger wrote: Add a radiobox to the settings to control if search should be done recursively as default or not. Set to no recursion as default. Signed-off-by: Dominic Jäger --- This did not exist in RFC www/manager6/storage/ContentView.js | 2 +-

Re: [pve-devel] [PATCH manager 3/4] gui: content view: Add checkbox for recursive search

comments inline On 4/2/20 1:34 PM, Dominic Jäger wrote: Default is no recursion. This commit depends on "Recursive search for iso and vztmpl" in pve-storage. Signed-off-by: Dominic Jäger --- Changes since RFC: * [0] became obsolte * This did not exist in RFC [0]

Re: [pve-devel] [PATCH storage 2/4] Recursive search for iso and vztmpl

comments inline On 4/2/20 1:34 PM, Dominic Jäger wrote: Each content type has a predefined location in a storage where they can be found. iso and vztmpl files can now be found and and also uploaded in subdirectories of those locations, too. Add a parameter "recursive" (default off) to the API

Re: [pve-devel] [PATCH storage 1/4] base plugin: Increase test coverage

sry for taking so long with the review, i am looking over now, after that i will test comments inline On 4/2/20 1:34 PM, Dominic Jäger wrote: Signed-off-by: Dominic Jäger --- This did not exist separately in RFC test/Makefile| 5 +- test/run_plugin_tests.pl | 184

[pve-devel] [PATCH storage] Add comment for volume_has_feature

Signed-off-by: Fabian Ebner --- Took the opportunity to document the whole function instead of just the new valid_target_formats option. PVE/Storage.pm | 15 +++ 1 file changed, 15 insertions(+) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index 60b8310..48179b5 100755 ---

[pve-devel] [PATCH storage v3 7/7] Fix: #2124 storage: add zstd support

Signed-off-by: Alwin Antreich --- PVE/Storage.pm | 4 +++- PVE/Storage/Plugin.pm | 2 +- test/test_archive_info.pm | 9 ++--- test/test_list_volumes.pm | 4 test/test_parse_volname.pm | 3 +++ test/test_path_to_volume_id.pm | 2 ++ 6 files changed,

[pve-devel] [PATCH qemu-server v3 1/2] restore: replace archive

format/compression regex to reduce the code duplication, as archive_info and decompressor_info provides the same information as well. Signed-off-by: Alwin Antreich --- PVE/QemuServer.pm | 36 ++-- 1 file changed, 6 insertions(+), 30 deletions(-) diff --git

[pve-devel] [PATCH storage v3 6/7] backup: more compact regex for backup filter

The more compact form of the regex should allow easier addition of new file extensions. Signed-off-by: Alwin Antreich --- PVE/Storage.pm| 4 ++-- PVE/Storage/Plugin.pm | 6 -- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index

[pve-devel] [PATCH storage v3 3/7] test: list_volumes

Test to reduce the potential for accidental breakage on regex changes. Signed-off-by: Alwin Antreich --- test/run_parser_tests.pl | 6 +- test/test_list_volumes.pm | 309 ++ 2 files changed, 314 insertions(+), 1 deletion(-) create mode 100644

[pve-devel] [PATCH v3 0/13] Fix: #2124 zstd

Zstandard (zstd) [0] is a data compression algorithm, in addition to gzip, lzo for our backup/restore. It can utilize multiple core CPUs. But by default it has one compression and one writer thread. Here some quick tests I made on my workstation. The files where placed on a ram disk. And with dd

[pve-devel] [PATCH guest-common v3] Fix: #2124 add zstd support

Signed-off-by: Alwin Antreich --- PVE/VZDump/Common.pm | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/PVE/VZDump/Common.pm b/PVE/VZDump/Common.pm index 4789a50..9a7c4f6 100644 --- a/PVE/VZDump/Common.pm +++ b/PVE/VZDump/Common.pm @@ -88,7 +88,7 @@ my $confdesc = {

[pve-devel] [PATCH storage v3 5/7] test: path_to_volume_id

Test to reduce the potential for accidental breakage on regex changes. Signed-off-by: Alwin Antreich --- test/run_parser_tests.pl | 2 +- test/test_path_to_volume_id.pm | 102 + 2 files changed, 103 insertions(+), 1 deletion(-) create mode 100644

[pve-devel] [PATCH storage v3 1/7] storage: test: split archive format/compressor

detection into separate functions so they are reusable and easier modifiable. Signed-off-by: Alwin Antreich --- test/Makefile | 5 ++- PVE/Storage.pm| 79 --- test/run_parser_tests.pl | 12 ++ test/test_archive_info.pm | 54

[pve-devel] [PATCH manager v3] Fix #2124: Add support for zstd

This patch adds zstd to the compression selection for backup on the GUI and add .zst to the backup file filter. Including zstd as package install dependency. Signed-off-by: Alwin Antreich --- PVE/VZDump.pm| 11 +-- debian/control |

[pve-devel] [PATCH storage v3 4/7] Fix: backup: ctime taken from stat

not the file name. The vzdump file was passed with the full path to the regex. That regex captures the time from the file name, to calculate the epoch. As the regex didn't match, the ctime from stat was taken instead. This resulted in the ctime shown when the file was changed, not when the backup

[pve-devel] [PATCH qemu-server v3 2/2] Fix #2124: Add support for zstd

Signed-off-by: Alwin Antreich --- PVE/QemuServer.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index e5bf41b..c72ddf4 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -7039,7 +7039,7 @@ sub complete_backup_archives { my

[pve-devel] [PATCH storage v3 2/7] test: parse_volname

Test to reduce the potential for accidental breakage on regex changes. Signed-off-by: Alwin Antreich --- test/run_parser_tests.pl | 2 +- test/test_parse_volname.pm | 95 ++ 2 files changed, 96 insertions(+), 1 deletion(-) create mode 100644

[pve-devel] [PATCH container v3] Fix: #2124 add zstd

Signed-off-by: Alwin Antreich --- src/PVE/LXC/Create.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm index 9faec63..91904b6 100644 --- a/src/PVE/LXC/Create.pm +++ b/src/PVE/LXC/Create.pm @@ -123,6 +123,7 @@ sub restore_tar_archive {

[pve-devel] [PATCH v5 qemu-server 18/19] cleanup_remotedisks: also include those migrated with storage_migrate

Call cleanup_remotedisks in phase1_cleanup as well, because that's where we end if sync_disks fails and some disks might already have been transfered successfully. Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 30 ++ 1 file changed, 22 insertions(+), 8

[pve-devel] [PATCH v5 storage 19/19] storage_migrate: add volname_for_storage helper

to guess a valid volname for a targetstorage of a different type. This makes it possible to migrate raw volumes between 'dir' and 'lvm' storages. It is only used when the storage type for the source storage X and target storage Y differ and should work as long as Y uses the standard naming scheme

[pve-devel] [PATCH v5 storage 10/19] Introduce allow_rename parameter for pvesm import and storage_migrate

and also return the ID of the allocated volume. This option allows plugins to choose a new name if there is a collision. In storage_migrate, the API version of the receiving side is checked. In Storage.pm's volume_import, when a plugin returns 'undef', it can be assumed that the import with the

[pve-devel] [PATCH v5 qemu-server 16/19] sync_disks: be more verbose if storage_migrate fails

If storage_migrate dies, the error message might not include the volume ID or the target storage ID, but those might be good to know. Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/PVE/QemuMigrate.pm

[pve-devel] [PATCH v5 qemu-server 15/19] sync_disks: use allow_rename to avoid collisions on the target storage

This makes it possible to migrate a VM with volumes store1:vm-123-disk-0 store2:vm-123-disk-0 to some targetstorage. Also prevents migration failure when there is an orphaned disk with the same volid on the target. To avoid confusion, the name should not change for 'vmstate'-volumes.

[pve-devel] [PATCH v5 storage 09/19] Add apiinfo helper to pvesm

Signed-off-by: Fabian Ebner --- PVE/CLI/pvesm.pm | 30 ++ 1 file changed, 30 insertions(+) diff --git a/PVE/CLI/pvesm.pm b/PVE/CLI/pvesm.pm index 510faba..0d1d816 100755 --- a/PVE/CLI/pvesm.pm +++ b/PVE/CLI/pvesm.pm @@ -47,6 +47,30 @@ sub setup_environment {

[pve-devel] [PATCH v5 guest-common 05/19] Use new storage_migrate interface

Signed-off-by: Fabian Ebner --- PVE/Replication.pm | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/PVE/Replication.pm b/PVE/Replication.pm index ae1ade4..5b1e917 100644 --- a/PVE/Replication.pm +++ b/PVE/Replication.pm @@ -186,8 +186,16 @@ sub replicate_volume {

[pve-devel] [PATCH v5 storage 04/19] Collect optional parameters for storage_migrate into $opts

Sanitizing $with_snapshots is done on extraction to save a line. Signed-off-by: Fabian Ebner --- PVE/API2/Storage/Content.pm | 2 +- PVE/Storage.pm | 12 +--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/PVE/API2/Storage/Content.pm

[pve-devel] [PATCH v5 qemu-server 13/19] Allow specifying targetstorage for offline migration

Signed-off-by: Fabian Ebner --- PVE/API2/Qemu.pm | 3 --- 1 file changed, 3 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 87fbd72..ec4c18c 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -3455,9 +3455,6 @@ __PACKAGE__->register_method({ $param->{online}

[pve-devel] [PATCH v5 qemu-server 12/19] Take note of changes to the volume IDs when migrating and update the config

Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 22 +++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 0bcbd04..14f3fcb 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -290,7 +290,9 @@ sub

[pve-devel] [PATCH v5 qemu-server 17/19] sync_disks: log output of storage_migrate

Signed-off-by: Fabian Ebner --- Not sure about this one. On the one hand it adds even more to the migration logs, which are already rather long. On the other hand it might contain useful information. PVE/QemuMigrate.pm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git

[pve-devel] [PATCH v5 qemu-server 14/19] Update volume IDs in one go

Use 'update_volume_ids' for the live-migrated disks as well. Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 26 ++ 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 14f3fcb..eb78537 100644 ---

[pve-devel] [PATCH v5 qemu-server 11/19] Allow parsing vmstate entries

Signed-off-by: Fabian Ebner --- PVE/QemuConfig.pm | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/PVE/QemuConfig.pm b/PVE/QemuConfig.pm index 78f5c60..7874081 100644 --- a/PVE/QemuConfig.pm +++ b/PVE/QemuConfig.pm @@ -5,6 +5,7 @@ use warnings; use

[pve-devel] [PATCH-SERIES v5] Make storage migration more flexible

Previous discussion here: [0]. This series aims to allow offline migration with '--targetstorage' and improve handling unsued/orphaned disks when migrating. It also makes it possible to migrate volumes between storages with a 'path' and storages without if the target storage uses the standard

[pve-devel] [PATCH v5 qemu-server 01/19] Switch to using foreach_volume instead of foreach_drive

It was necessary to move foreach_volid back to QemuServer.pm In VZDump/QemuServer.pm and QemuMigrate.pm the dependency on QemuConfig.pm was already there, just the explicit "use" was missing. Signed-off-by: Fabian Ebner --- PVE/API2/Qemu.pm| 6 ++-- PVE/QemuConfig.pm |

[pve-devel] [PATCH v5 container 03/19] Use parse_volume instead of parse_ct-variants

Signed-off-by: Fabian Ebner --- src/PVE/API2/LXC.pm | 15 +++ src/PVE/CLI/pct.pm| 3 +-- src/PVE/LXC.pm| 7 +++ src/PVE/LXC/Config.pm | 34 ++ 4 files changed, 17 insertions(+), 42 deletions(-) diff --git a/src/PVE/API2/LXC.pm

[pve-devel] [PATCH v5 container 02/19] Use foreach_volume instead of foreach_mountpoint-variants

Signed-off-by: Fabian Ebner --- src/PVE/API2/LXC.pm| 4 ++-- src/PVE/API2/LXC/Config.pm | 2 +- src/PVE/API2/LXC/Status.pm | 2 +- src/PVE/CLI/pct.pm | 4 ++-- src/PVE/LXC.pm | 14 +++--- src/PVE/LXC/Config.pm | 35 +--

[pve-devel] [PATCH v5 guest-common 08/19] Add update_volume_ids

This function is intened to be used after doing a migration where some of the volume IDs changed. Signed-off-by: Fabian Ebner --- PVE/AbstractConfig.pm | 30 ++ 1 file changed, 30 insertions(+) diff --git a/PVE/AbstractConfig.pm b/PVE/AbstractConfig.pm index

[pve-devel] [PATCH v5 container 07/19] Use new storage_migrate interface

Signed-off-by: Fabian Ebner --- src/PVE/LXC/Migrate.pm | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/PVE/LXC/Migrate.pm b/src/PVE/LXC/Migrate.pm index 638ce1f..d0be6d4 100644 --- a/src/PVE/LXC/Migrate.pm +++ b/src/PVE/LXC/Migrate.pm @@ -278,17 +278,21 @@ sub

[pve-devel] [PATCH v5 qemu-server 06/19] Use new storage_migrate interface

Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 2925b90..0bcbd04 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -541,15 +541,19 @@ sub sync_disks {

Re: [pve-devel] [PATCH v3 manager 1/3] Move wipe_disks to PVE::Diskmanage

Series works, test works, could not find anything obviously wrong Reviewed-by: Dominik Csapak Tested-by: Dominik Csapak On 3/11/20 2:05 PM, Dominic Jäger wrote: Move wipe_disks from PVE::Ceph::Tools to PVE::Diskmanage. Relies on the corresponding patch in pve-storage. Signed-off-by: Dominic

[pve-devel] [PATCH access-control v2] auth ldap/ad: make password a parameter for the api

Instead of simply requiring it to exist in /etc/pve. Takes after the password handling of CIFS in pve-storage. Signed-off-by: Dominik Csapak --- changes from v1: * use 'realm' dir instead of 'ldap' (with fallback and comment to removal) PVE/API2/Domains.pm | 26 +++ PVE/Auth/AD.pm