Re: [pve-devel] [PATCH common 1/1] JSONSchema: extend pve-configid regex by '-'

2020-04-22 Thread Thomas Lamprecht
On 4/9/20 4:10 PM, Dominik Csapak wrote: > we use this format for all 'delete' options but we have some options > that have a '-' in the name (e.g. 'sync-defaults-options') that cannot > be deleted if it is not included > > Signed-off-by: Dominik Csapak > --- > src/PVE/JSONSchema.pm | 2 +- > 1

[pve-devel] applied: [PATCH widget-toolkit 1/2] window/Edit: accept bodyPadding from config

2020-04-22 Thread Thomas Lamprecht
On 4/9/20 4:10 PM, Dominik Csapak wrote: > this is sometimes useful, e.g. when using a tabpanel in an edit window > > Signed-off-by: Dominik Csapak > --- > window/Edit.js | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/window/Edit.js b/window/Edit.js > index

Re: [pve-devel] [PATCH storage v4 02/12] storage: replace build-in stat with File::stat

2020-04-22 Thread Dietmar Maurer
> On April 22, 2020 6:00 PM Alwin Antreich wrote: > > > On Wed, Apr 22, 2020 at 05:35:05PM +0200, Dietmar Maurer wrote: > > AFAIK this can have ugly side effects ... > Okay, I was not aware of any know side effects. > > I took the File::stat, since we use it already in pve-cluster, >

[pve-devel] [PATCH qemu-server 0/1] Harmonize log format with disk transfer, add percentage

2020-04-22 Thread helmo
From: Herman van Rink While migrating a VM we show a percentage on the lines about disk status. but not for memory transfer. And the structure of these lines is different. Current output for disk live migration: drive-scsi1: transferred: 84934656 bytes remaining: 988807168 bytes total:

[pve-devel] [PATCH qemu-server 1/1] Harmonize log format with disk transfer, add percentage

2020-04-22 Thread helmo
From: Herman van Rink Signed-off-by: Herman van Rink --- PVE/QemuMigrate.pm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index d90b709..f163d0a 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -1013,8 +1013,9 @@ sub

Re: [pve-devel] [PATCH storage v4 02/12] storage: replace build-in stat with File::stat

2020-04-22 Thread Alwin Antreich
On Wed, Apr 22, 2020 at 05:35:05PM +0200, Dietmar Maurer wrote: > AFAIK this can have ugly side effects ... Okay, I was not aware of any know side effects. I took the File::stat, since we use it already in pve-cluster, qemu-server, pve-common, ... . And a off-list discussion with Thomas and

Re: [pve-devel] [PATCH storage v4 02/12] storage: replace build-in stat with File::stat

2020-04-22 Thread Dietmar Maurer
AFAIK this can have ugly side effects ... > On April 22, 2020 4:57 PM Alwin Antreich wrote: > > > to minimize variable declarations. And allow to mock this method in > tests instead of the perl build-in stat. > > Signed-off-by: Alwin Antreich > --- > PVE/Diskmanage.pm | 9 + >

[pve-devel] [PATCH container v4] Fix: #2124 add zstd

2020-04-22 Thread Alwin Antreich
Signed-off-by: Alwin Antreich --- src/PVE/LXC/Create.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm index 52b0b48..39902a2 100644 --- a/src/PVE/LXC/Create.pm +++ b/src/PVE/LXC/Create.pm @@ -123,6 +123,7 @@ sub restore_tar_archive {

[pve-devel] [PATCH storage v4 00/12] Fix: #2124 zstd

2020-04-22 Thread Alwin Antreich
Zstandard (zstd) [0] is a data compression algorithm, in addition to gzip, lzo for our backup/restore. It can utilize multiple core CPUs. But by default it has one compression and one writer thread. Here some quick tests I made on my workstation. The files where placed on a ram disk. And with dd

[pve-devel] [PATCH storage v4 05/12] Fix: backup: ctime was from stat not file name

2020-04-22 Thread Alwin Antreich
The vzdump file was passed with the full path to the regex. That regex captures the time from the file name, to calculate the epoch. As the regex didn't match, the ctime from stat was taken instead. This resulted in the ctime shown when the file was changed, not when the backup was made.

[pve-devel] [PATCH storage v4 06/12] test: path_to_volume_id

2020-04-22 Thread Alwin Antreich
Test to reduce the potential for accidental breakage on regex changes. Signed-off-by: Alwin Antreich --- test/path_to_volume_id_test.pm | 242 + test/run_plugin_tests.pl | 2 +- 2 files changed, 243 insertions(+), 1 deletion(-) create mode 100644

[pve-devel] [PATCH storage v4 12/12] test: filesystem_path

2020-04-22 Thread Alwin Antreich
Signed-off-by: Alwin Antreich --- test/filesystem_path_test.pm | 91 test/run_plugin_tests.pl | 1 + 2 files changed, 92 insertions(+) create mode 100644 test/filesystem_path_test.pm diff --git a/test/filesystem_path_test.pm

[pve-devel] [PATCH storage v4 11/12] test: get_subdir

2020-04-22 Thread Alwin Antreich
Co-Authored-by: Dominic Jaeger Signed-off-by: Alwin Antreich --- test/get_subdir_test.pm | 44 test/run_plugin_tests.pl | 1 + 2 files changed, 45 insertions(+) create mode 100644 test/get_subdir_test.pm diff --git a/test/get_subdir_test.pm

[pve-devel] [PATCH storage v4 04/12] test: list_volumes

2020-04-22 Thread Alwin Antreich
Test to reduce the potential for accidental breakage on regex changes. Co-Authored-by: Dominic Jaeger Signed-off-by: Alwin Antreich --- test/list_volumes_test.pm | 519 ++ test/run_plugin_tests.pl | 6 +- 2 files changed, 524 insertions(+), 1 deletion(-)

[pve-devel] [PATCH storage v4 10/12] Fix: #2124 storage: add zstd support

2020-04-22 Thread Alwin Antreich
Signed-off-by: Alwin Antreich --- PVE/Storage.pm | 4 +++- PVE/Storage/Plugin.pm | 2 +- test/archive_info_test.pm | 4 +++- test/list_volumes_test.pm | 18 ++ test/parse_volname_test.pm | 6 +++--- test/path_to_volume_id_test.pm | 16

[pve-devel] [PATCH storage v4 08/12] Fix: add missing snippets subdir

2020-04-22 Thread Alwin Antreich
since it is a valid content type and adapt the path_to_volume_id_test. Also adds an extra check if all vtype_subdirs are returned. Signed-off-by: Alwin Antreich --- PVE/Storage.pm | 4 test/path_to_volume_id_test.pm | 26 +- 2 files changed, 25

[pve-devel] [PATCH qemu-server v4 2/2] Fix #2124: Add support for zstd

2020-04-22 Thread Alwin Antreich
Signed-off-by: Alwin Antreich --- PVE/QemuServer.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 265d4f8..fda1acb 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -7165,7 +7165,7 @@ sub complete_backup_archives { my

[pve-devel] [PATCH storage v4 03/12] test: parse_volname

2020-04-22 Thread Alwin Antreich
Test to reduce the potential for accidental breakage on regex changes. And to make sure that all vtype_subdirs are parsed. Signed-off-by: Alwin Antreich --- PVE/Storage/Plugin.pm | 4 + test/parse_volname_test.pm | 253 + test/run_plugin_tests.pl |

[pve-devel] [PATCH storage v4 01/12] storage: test: split archive format/compressor

2020-04-22 Thread Alwin Antreich
detection into separate functions so they are reusable and easier modifiable. This patch also adds the test for archive_info. Signed-off-by: Alwin Antreich --- test/Makefile | 5 +- PVE/Storage.pm| 79 +--- test/archive_info_test.pm | 125

[pve-devel] [PATCH storage v4 07/12] Fix: path_to_volume_id returned wrong content

2020-04-22 Thread Alwin Antreich
type for backup files. Patch includes changes of the test as well. Signed-off-by: Alwin Antreich --- PVE/Storage.pm | 2 +- test/path_to_volume_id_test.pm | 8 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index

[pve-devel] [PATCH storage v4 09/12] backup: compact regex for backup file filter

2020-04-22 Thread Alwin Antreich
the more compact form of the regex should allow easier addition of new file extensions. Signed-off-by: Alwin Antreich --- PVE/Storage.pm| 4 ++-- PVE/Storage/Plugin.pm | 6 -- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index

[pve-devel] [PATCH manager v4] Fix #2124: Add support for zstd

2020-04-22 Thread Alwin Antreich
This patch adds the zstd to the compression selection for backup on the GUI and add .zst to the backup file filter. Including zstd as package install dependency. Signed-off-by: Alwin Antreich --- PVE/VZDump.pm| 11 +-- debian/control

[pve-devel] [PATCH storage v4 02/12] storage: replace build-in stat with File::stat

2020-04-22 Thread Alwin Antreich
to minimize variable declarations. And allow to mock this method in tests instead of the perl build-in stat. Signed-off-by: Alwin Antreich --- PVE/Diskmanage.pm | 9 + PVE/Storage/Plugin.pm | 34 ++ 2 files changed, 15 insertions(+), 28 deletions(-)

[pve-devel] [PATCH qemu-server v4 1/2] restore: replace archive format/compression

2020-04-22 Thread Alwin Antreich
regex to reduce the code duplication, as archive_info and decompressor_info provides the same information as well. Signed-off-by: Alwin Antreich --- PVE/QemuServer.pm | 36 ++-- 1 file changed, 6 insertions(+), 30 deletions(-) diff --git a/PVE/QemuServer.pm

[pve-devel] [PATCH guest-common v4] Fix: #2124 add zstd support

2020-04-22 Thread Alwin Antreich
Signed-off-by: Alwin Antreich --- PVE/VZDump/Common.pm | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/PVE/VZDump/Common.pm b/PVE/VZDump/Common.pm index 4789a50..909e3af 100644 --- a/PVE/VZDump/Common.pm +++ b/PVE/VZDump/Common.pm @@ -88,7 +88,7 @@ my $confdesc = {

[pve-devel] [PATCH qemu-server 1/6] api: check Sys.Audit permissions when setting a custom CPU model

2020-04-22 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- PVE/API2/Qemu.pm | 25 + 1 file changed, 25 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index ec4c18c..fe8c06d 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -21,6 +21,7 @@ use PVE::GuestHelpers; use

[pve-devel] [PATCH manager 5/6] ui: CPUModelSelector: use API call for store

2020-04-22 Thread Stefan Reiter
CPU models are retrieved from the new /nodes/X/cpu call and ordered by vendor to approximate the previous sort order (less change for accustomed users). With this, custom CPU models are now selectable via the GUI. Signed-off-by: Stefan Reiter --- www/manager6/form/CPUModelSelector.js | 221

[pve-devel] [PATCH 0/6] Custom CPU models API/GUI basics

2020-04-22 Thread Stefan Reiter
Permission handling, the beginnings of the API and getting the GUI to play nice with custom models (no editor yet, but it'll behave as expected if a determined user creates a custom model by editing the config). First 3 patches are API stuff, 4 is an independent UI fix/cleanup, rest are new GUI

[pve-devel] [PATCH qemu-server 2/6] api: allow listing custom and default CPU models

2020-04-22 Thread Stefan Reiter
More API calls will follow for this path, for now add the 'index' call to list all custom and default CPU models. Any user can list the default CPU models, as these are public anyway, but custom models are restricted to users with Sys.Audit on /nodes. Signed-off-by: Stefan Reiter ---

[pve-devel] [PATCH manager 6/6] ui: ProcessorEdit: allow modifications with inaccessible CPU model

2020-04-22 Thread Stefan Reiter
An administrator can set a custom CPU model for a VM where the general user does not have permission to use this particular model. Prior to this change the ProcessorEdit component would be broken by this, since the store of the CPU type selector did not contain the configured CPU model. Add it in

[pve-devel] [PATCH manager 4/6] ui: ProcessorEdit: fix total core calculation and use view model

2020-04-22 Thread Stefan Reiter
Clean up the code in ProcessorEdit with a view model and fix a bug while at it - previously, pressing the 'Reset' button on the form would always set the value of the total core count field to 1. Signed-off-by: Stefan Reiter --- The fix is technically only the setValues part, but I started off

[pve-devel] [PATCH manager 3/6] api: register /nodes/X/cpu call for CPU models

2020-04-22 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- Depends on updated qemu-server. PVE/API2/Nodes.pm | 7 +++ 1 file changed, 7 insertions(+) diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index 58497b2b..2ac838ea 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -34,6 +34,7 @@ use

[pve-devel] applied: [PATCH container] CentOS 6: add '--nohangup' as mingetty arg

2020-04-22 Thread Wolfgang Bumiller
applied On Wed, Apr 22, 2020 at 11:24:27AM +0200, Stoiko Ivanov wrote: > When creating an unprivileged container with CentOS 6 (which will be EOL in > Nov 2020 [0]) the console does not work. can't wait > > The problem is mitigated by adding the --nohangup argument to the mingetty >

Re: [pve-devel] [PATCH storage] ZFS: mount subvols in activate_volume

2020-04-22 Thread Wolfgang Bumiller
On Wed, Apr 22, 2020 at 11:28:36AM +0200, Thomas Lamprecht wrote: > On 4/22/20 10:30 AM, Fabian Ebner wrote: > > Turns out that this alone doesn't make starting containers with an > > unmounted zfs subvolume working. But "pct clone" and "pct mount" can still > > benefit from this patch. The

Re: [pve-devel] [PATCH storage] ZFS: mount subvols in activate_volume

2020-04-22 Thread Thomas Lamprecht
On 4/22/20 10:30 AM, Fabian Ebner wrote: > Turns out that this alone doesn't make starting containers with an unmounted > zfs subvolume working. But "pct clone" and "pct mount" can still benefit from > this patch. The problem for starting a container is that when we call >

[pve-devel] [PATCH container] CentOS 6: add '--nohangup' as mingetty arg

2020-04-22 Thread Stoiko Ivanov
When creating an unprivileged container with CentOS 6 (which will be EOL in Nov 2020 [0]) the console does not work. The problem is mitigated by adding the --nohangup argument to the mingetty invocations during bootup (in /etc/init/tty.conf). The idea for the fix is based on the legacy template

Re: [pve-devel] [PATCH manager] HDEdit: warn when IO threads are enabled with incompatible controller

2020-04-22 Thread Dominik Csapak
Looks good one comment inline: On 4/16/20 5:15 PM, Stefan Reiter wrote: The only warning displayed before was in the "VM Start" task log, rather hidden. In the wizard we already auto-selected the correct controller, but not when modifying a disk on an existing VM. Don't break existing

[pve-devel] [PATCH docs] certs: extend ACME section with DNS/plugin info

2020-04-22 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- certificate-management.adoc | 131 ++-- 1 file changed, 125 insertions(+), 6 deletions(-) diff --git a/certificate-management.adoc b/certificate-management.adoc index db76062..dc2834b 100644 --- a/certificate-management.adoc

Re: [pve-devel] [PATCH storage] ZFS: mount subvols in activate_volume

2020-04-22 Thread Fabian Ebner
Turns out that this alone doesn't make starting containers with an unmounted zfs subvolume working. But "pct clone" and "pct mount" can still benefit from this patch. The problem for starting a container is that when we call PVE::Storage::activate_volumes, it happens inside the

[pve-devel] [PATCH widget-toolkit] improve multiselect behaviour for checkboxmodel

2020-04-22 Thread Dominik Csapak
when using the mode 'MULTI' for the checkbox selection model, clicking a row would deselect all others, only clicking exactly on the checkbox would select multiple rows this override makes the whole cell of the checkbox behave like the checkbox. This minimizes misclicks from the user.