Re: [pve-devel] [PATCH pve-docs] Close #1623: replace apt-get to apt

2020-07-07 Thread Fabian Grünbichler
On July 6, 2020 1:38 pm, Moayad Almalat wrote: > Signed-off-by: Moayad Almalat > --- > api-viewer/apidata.js| 8 > local-zfs.adoc | 4 ++-- > pve-firewall.adoc| 2 +- > pve-installation.adoc| 4 ++-- > pve-package-repos.adoc | 2 +- >

Re: [pve-devel] [PATCH xtermjs] termproxy: rewrite in rust

2020-07-07 Thread Fabian Grünbichler
On July 7, 2020 8:36 am, Thomas Lamprecht wrote: > On 07.07.20 08:24, Fabian Grünbichler wrote: >> On July 7, 2020 6:42 am, Dietmar Maurer wrote: >>>> so we have a 'termproxy' crate+binary and a binary package with name >>>> 'pve-xtermjs' >>> &

Re: [pve-devel] [PATCH storage 3/6] Introduce prune-backups property for directory-based storages

2020-07-07 Thread Fabian Grünbichler
On July 7, 2020 8:43 am, Thomas Lamprecht wrote: > On 07.07.20 08:36, Fabian Grünbichler wrote: >> On June 4, 2020 11:08 am, Fabian Ebner wrote: >>> @@ -43,6 +43,54 @@ cfs_register_file ('storage.cfg', >>>sub { __PACKAGE__->parse_config(@_); }, >&g

Re: [pve-devel] [PATCH storage 6/6] Add API and pvesm calls for prune_backups

2020-07-07 Thread Fabian Grünbichler
s/VM/guest in most descriptions - this is not in qemu-server ;) On June 4, 2020 11:08 am, Fabian Ebner wrote: > Signed-off-by: Fabian Ebner > --- > > Not sure if this is the best place for the new API endpoints. > > I decided to opt for two distinct calls rather than just using a > --dry-run

Re: [pve-devel] [PATCH storage 4/6] Add prune_backups to storage API

2020-07-07 Thread Fabian Grünbichler
On June 4, 2020 11:08 am, Fabian Ebner wrote: > Implement it for generic storages supporting backups (i.e. > directory-based storages) and add a wrapper for PBS. > > Signed-off-by: Fabian Ebner > --- > PVE/Storage.pm | 27 - > PVE/Storage/PBSPlugin.pm | 50 >

Re: [pve-devel] [PATCH storage 3/6] Introduce prune-backups property for directory-based storages

2020-07-07 Thread Fabian Grünbichler
small nit in-line On June 4, 2020 11:08 am, Fabian Ebner wrote: > Signed-off-by: Fabian Ebner > --- > PVE/Storage/CIFSPlugin.pm | 1 + > PVE/Storage/CephFSPlugin.pm| 1 + > PVE/Storage/DirPlugin.pm | 5 ++-- > PVE/Storage/GlusterfsPlugin.pm | 5 ++-- >

Re: [pve-devel] [PATCH xtermjs] termproxy: rewrite in rust

2020-07-07 Thread Fabian Grünbichler
On July 7, 2020 6:42 am, Dietmar Maurer wrote: >> so we have a 'termproxy' crate+binary and a binary package with name >> 'pve-xtermjs' > > This is quite confusing ... well, it replaces a 'pve-xtermjs' binary package that ships a 'termproxy' binary (/CLIHandler). the alternative is to bump the

Re: [pve-devel] [PATCH v3 qemu-server 3/3] fix #2671: include CPU format in man page again

2020-07-06 Thread Fabian Grünbichler
this looks good, waiting for pve-common bump before applying this with a versioned dependency. On June 25, 2020 1:35 pm, Stefan Reiter wrote: > Use the new register_format(3) call to use a validator (instead of a > parser) for 'pve-(vm-)?cpu-conf'. This way the $cpu_fmt hash can be used for >

[pve-devel] applied: [PATCH v3 common 2/3] JSONSchema: use validator in print_property_string too

2020-07-06 Thread Fabian Grünbichler
On June 25, 2020 1:35 pm, Stefan Reiter wrote: > Suggested-by: Fabian Grünbichler > Signed-off-by: Stefan Reiter > --- > src/PVE/JSONSchema.pm | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm > index f987006..59

[pve-devel] applied: [PATCH v3 common 1/3] JSONSchema: add format validator support and cleanup check_format

2020-07-06 Thread Fabian Grünbichler
On June 25, 2020 1:35 pm, Stefan Reiter wrote: > Adds a third, optional parameter to register_format that allows specifying > a function that will be called after parsing and can validate the parsed > data. A validator should die on failed validation, and can also change the > parsed object by

Re: [pve-devel] [PATCH qemu] PVE-Backup: remove dirty-bitmap in pvebackup_complete_cb for failed jobs

2020-07-02 Thread Fabian Grünbichler
it should also be possible to keep the old bitmap (and associated backup checksum) in this case? this is what bitmap-mode on-success is supposed to do, but maybe errors are not triggering the right code paths? On July 1, 2020 2:17 pm, Dietmar Maurer wrote: > Note: We remove the device from

Re: [pve-devel] [PATCH container] fix #2820: block adding new volume with same id if it's pending delete

2020-07-01 Thread Fabian Grünbichler
On July 1, 2020 2:05 pm, Thomas Lamprecht wrote: > On 01.07.20 09:11, Fabian Grünbichler wrote: >> - we can actually just put the new mpX into the pending queue, and >> remove the entry from the pending deletion queue? (it's hotplugging >> that is the problem, not queui

Re: [pve-devel] [PATCH v2 container] fix #2820: block adding new volume with same id if it's pending delete

2020-07-01 Thread Fabian Grünbichler
On July 1, 2020 11:56 am, Oguz Bektas wrote: > if a user tries to add a mountpoint mpX which is waiting for a pending > delete, hotplugging a new mountpoint with name mpX before the > previous one is detached should not be allowed. > > do a simple check to see if the given mpX is already in the

[pve-devel] [PATCH vncterm 1/2] libvncserver: update sources to 0.9.13

2020-07-01 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- LibVNCServer-0.9.11.tar.gz | Bin 1413739 -> 0 bytes LibVNCServer-0.9.13.tar.gz | Bin 0 -> 567491 bytes 2 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 LibVNCServer-0.9.11.tar.gz create mode 100644 LibVNCServer-0.9.13.tar.gz

[pve-devel] [PATCH vncterm 2/2] build: rebase on libvncserver 0.9.13

2020-07-01 Thread Fabian Grünbichler
upstream switched to cmake from autotools. it is possible to drop systemd via a build option now, so use that instead of patching. Signed-off-by: Fabian Grünbichler --- Makefile | 7 +++--- vncpatches/tls-auth-pluging.patch | 41 +++ debian

[pve-devel] applied: [PATCH firewall] ebtables: keep policy of custom chains

2020-07-01 Thread Fabian Grünbichler
with bug # added to commit subject. sorry for the delay, and thanks for the fix! On June 2, 2020 10:06 am, Stoiko Ivanov wrote: > currently all ebtalbes chains are created with a hardcoded policy of ACCEPT. > This patch changes the functionality to store the configured policy of a > chain while

[pve-devel] applied: [PATCH pve-network 2/2] build: fix erroneous install with empty DESTDIR

2020-07-01 Thread Fabian Grünbichler
want to do ;) Signed-off-by: Fabian Grünbichler --- noticed this while building as non-root.. PVE/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/PVE/Makefile b/PVE/Makefile index 1fb961d..26e01a4 100644 --- a/PVE/Makefile +++ b/PVE/Makefile @@ -1,3 +1,5 @@ +all: + .PHONY: install

[pve-devel] applied: [PATCH pve-network 1/2] remove more variable declarations with post-ifs

2020-07-01 Thread Fabian Grünbichler
usage of $mtu is always guarded by an if itself anyway, so all of these are redundant post-ifs and can be removed. Signed-off-by: Fabian Grünbichler --- PVE/Network/SDN/Zones/QinQPlugin.pm | 4 ++-- PVE/Network/SDN/Zones/VlanPlugin.pm | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions

[pve-devel] applied: [PATCH pve-network] use PVE::Tools::split_list for ip lists

2020-07-01 Thread Fabian Grünbichler
with the following applied on-top, since we don't want declarations combined with post-if: my $foo = 'bla' if $bar; is undefined behaviour[1]. -8<- diff --git a/PVE/Network/SDN/Controllers/EvpnPlugin.pm b/PVE/Network/SDN/Controllers/EvpnPlugin.pm index 9321af1..d82de2a 100644 ---

Re: [pve-devel] [PATCH container] fix #2820: block adding new volume with same id if it's pending delete

2020-07-01 Thread Fabian Grünbichler
On June 30, 2020 3:56 pm, Oguz Bektas wrote: > do a simple check to see if our $opt is already in the delete section. > > Signed-off-by: Oguz Bektas > --- > src/PVE/LXC/Config.pm | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm > index

Re: [pve-devel] [PATCH container] Move volume activation to vm_start

2020-06-26 Thread Fabian Grünbichler
On June 17, 2020 2:18 pm, Stoiko Ivanov wrote: > currently all volumes for a container are activated in the pre-start hook, > which runs in a separate mount namespace (lxc.monitor.unshare is set to 1 > in our container config). > This leads to problems with ZFS, if a pool is imported by this call

[pve-devel] applied: [PATCH node_config 1/1] node_config: Allow leading underscore in ACME aliases

2020-06-25 Thread Fabian Grünbichler
applied with rename follow-up, thanks! On June 22, 2020 12:10 pm, Fabian Möller wrote: > --- > PVE/NodeConfig.pm | 12 +++- > 1 file changed, 11 insertions(+), 1 deletion(-) > > diff --git a/PVE/NodeConfig.pm b/PVE/NodeConfig.pm > index ad49e288..017f6b30 100644 > ---

Re: [pve-devel] [RFC qemu-server] close #2741: check for VM.Config.Cloudinit permission

2020-06-24 Thread Fabian Grünbichler
On June 3, 2020 3:58 pm, Mira Limbeck wrote: > This allows setting ciuser, cipassword and all other cloudinit settings that > are not part of the network without VM.Config.Network permissions. > > Signed-off-by: Mira Limbeck > --- > PVE/API2/Qemu.pm | 5 - > 1 file changed, 4 insertions(+),

Re: [pve-devel] [PATCH v2 common 1/2] JSONSchema: add format validator support and cleanup check_format

2020-06-24 Thread Fabian Grünbichler
On June 24, 2020 10:54 am, Stefan Reiter wrote: > On 6/23/20 3:39 PM, Fabian Grünbichler wrote: >> LGTM, what do you think about the following follow-up: >> >> --8<- >> >> diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm >> inde

Re: [pve-devel] Package for vma seperately

2020-06-24 Thread Fabian Grünbichler
On June 5, 2020 1:33 pm, Mark Schouten wrote: > > Hi! > > I'm building Linux Images in Gitlab to distribute over customer clusters > easily. We have an (daDup.eu) s3 bucket where we place the vma-files and > mount that bucket via s3fs (works pretty well!). But, to create the VMA, I > need to

Re: [pve-devel] [PATCH pve-network] allow [ ,;] for ip lists

2020-06-24 Thread Fabian Grünbichler
why not use PVE::Tools::split_list ? it's our standard helper for these kind of things, and also correctly trims whitespace and has support for \0-separated lists ;) On June 12, 2020 6:14 pm, Alexandre Derumier wrote: > Signed-off-by: Alexandre Derumier > --- >

Re: [pve-devel] [PATCH qemu-server] api: add option to get pending config returned as object instead of an array

2020-06-24 Thread Fabian Grünbichler
a bit of a rationale would be nice ;) isn't this just a simple map transformation that can be done client-side? my $hash = { map { my $key = delete $_->{key}; return ($key => $_); } @$array }; or in whatever language you need. filtering/sorting/limiting server-side makes sense for some calls

[pve-devel] applied: [PATCH manager] fix #2810: reset state of mounts array in initComponent

2020-06-24 Thread Fabian Grünbichler
On June 24, 2020 9:32 am, Dominik Csapak wrote: > so that each new instance has an empty mounts list > > Signed-off-by: Dominik Csapak > --- > @fabian @oguz, i remembered that i know this issue and had a fix already^^ > www/manager6/lxc/FeaturesEdit.js | 8 +++- > 1 file changed, 7

Re: [pve-devel] [PATCH v2 common 1/2] JSONSchema: add format validator support and cleanup check_format

2020-06-23 Thread Fabian Grünbichler
LGTM, what do you think about the following follow-up: --8<- diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm index b2ba9f7..d28143d 100644 --- a/src/PVE/JSONSchema.pm +++ b/src/PVE/JSONSchema.pm @@ -1878,9 +1878,12 @@ sub generate_typetext { sub print_property_string {

[pve-devel] applied: [PATCH storage] Fix 2763: Revert "storage_migrate: check if target storage supports content type"

2020-06-23 Thread Fabian Grünbichler
On May 25, 2020 9:41 am, Fabian Ebner wrote: > This reverts commit 95015dbbf24b710011965805e689c03923fb830c. > > parse_volname always gives 'images' and not 'rootdir'. In most > cases the volume name alone does not contain the needed information, > e.g. vm-123-disk-0 can be both a VM volume or a

Re: [pve-devel] [PATCH v2 manager] fix #2810: reset state properly when editing mount features of containers

2020-06-23 Thread Fabian Grünbichler
On June 22, 2020 3:55 pm, Oguz Bektas wrote: > initializing 'mounts' array in the panel scope causes edits on subsequent > containers to get the values (mount=nfs) from the previous container. fix > this by > initializing the 'mounts' array in 'onGetValues' and 'setValues' > separately. > >

Re: [pve-devel] [PATCH node_config 1/1] node_config: Allow leading underscore in ACME aliases

2020-06-23 Thread Fabian Grünbichler
LGTM, except for one minor nit. could you please send a CLA as described in our Developer Documentation to off...@proxmox.com ? https://pve.proxmox.com/wiki/Developer_Documentation#Software_License_and_Copyright Thanks in advance! On June 22, 2020 12:10 pm, Fabian Möller wrote: > --- >

Re: [pve-devel] [PATCH manager] fix #2810: don't add options multiple times to features property

2020-06-22 Thread Fabian Grünbichler
does not fix the issue (which is that the state is not properly resetted between editing one container and then another one). On June 22, 2020 1:54 pm, Oguz Bektas wrote: > instead of unconditionally pushing to the 'mounts' array we need to check > if we already have the option in there. without

[pve-devel] applied: [PATCH kernel] fix #2814: config: disable lockdown

2020-06-22 Thread Fabian Grünbichler
since it prevents boot with our current way of building ZFS modules in case a system is booted with secureboot enabled. Signed-off-by: Fabian Grünbichler --- Notes: requires an ABI bump debian/rules | 3 +++ 1 file changed, 3 insertions(+) diff --git a/debian/rules b/debian/rules index

[pve-devel] applied: [PATCH qemu-server] vncproxy: allow to request a generated VNC password

2020-06-22 Thread Fabian Grünbichler
. Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index dcb364d..3965c26 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1597,8 +1597,12 @@ my $gen_rand_chars = sub

[pve-devel] [PATCH manager] fix #2771: relax cert API endpoints permissions

2020-06-17 Thread Fabian Grünbichler
allow users with Sys.Modify to modify custom or ACME certificates. those users can already hose the system in plenty of ways, no reason to restrict this in particular to being root@pam only. Signed-off-by: Fabian Grünbichler --- PVE/API2/ACME.pm | 9 + PVE/API2/Certificates.pm

[pve-devel] [PATCH manager] fix #2784: always compare ACME domains in lower case

2020-06-17 Thread Fabian Grünbichler
otherwise the ACME endpoint might return the ordered domain in lower case and we fail to find our plugin config. Signed-off-by: Fabian Grünbichler --- PVE/API2/ACME.pm | 4 +++- PVE/NodeConfig.pm | 9 - 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/PVE/API2/ACME.pm b

[pve-devel] applied: [PATCH qemu-server] cfg2cmd test: hardcode/mock bridge MTU

2020-06-17 Thread Fabian Grünbichler
otherwise the netdev test reads the MTU value from the test host's vmbr0 bridge, or fails if no such bridge exists. Signed-off-by: Fabian Grünbichler --- might make sense to extend this to actually test the functionality/MTU handling, but just unbreaking the test for now.. test

[pve-devel] [PATCH qemu-server 2/2] properly parse 'vga' for termproxy

2020-06-17 Thread Fabian Grünbichler
'vga' is a property string, we can't just assume it starts with the default key's value here either. Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 95b1922..53dc594 100644

[pve-devel] [PATCH qemu-server 1/2] fix #2787: properly parse 'vga' for vncproxy

2020-06-17 Thread Fabian Grünbichler
'vga' is a property string, we can't just assume it starts with the default key's value. Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 974ee3b..95b1922 100644

Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo

2020-06-09 Thread Fabian Grünbichler
On June 8, 2020 5:16 pm, Michael Rasmussen wrote: > On Mon, 8 Jun 2020 15:00:11 +0200 > Thomas Lamprecht wrote: > >> >> You can effectively provide a full custom plugin, so it has not more >> limitations than any existing one. What extra functionality regarding >> interface ABI would the

[pve-devel] [PATCH qemu-server 1/2] create_disks: fix uninitialized warning

2020-06-02 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index fd51bf3..5e6fd42 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1095,7 +1095,7 @@ my $update_vm_api = sub

[pve-devel] [PATCH qemu-server 2/2] fix #2774: add early check for non-managed volumes

2020-06-02 Thread Fabian Grünbichler
this is just a cosmetic/usability improvement. Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 5e6fd42..974ee3b 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1095,6 +1095,9 @@ my

[pve-devel] [PATCH manager] api: improve node index with missing/broken cert

2020-06-02 Thread Fabian Grünbichler
since this API endpoint is used for the node selector in the GUI, which causes quite widespread breakage. Signed-off-by: Fabian Grünbichler --- PVE/API2/Nodes.pm | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index 58497b2b

Re: [pve-devel] [PATCH kernel] add pve-kernel-X.Y-libc-dev package

2020-05-28 Thread Fabian Grünbichler
ould not break anything. Besides that, but without a full rebuild-test: Reviewed-By: Fabian Grünbichler > > note: This was working really quick, almost suspicious... Tested by building > QEMU (which inspired my doing this now in the first place due to the sizeof > bug > we hav

[pve-devel] [PATCH cluster 1/2] fix #2727: pass correct format for linkX

2020-05-12 Thread Fabian Grünbichler
to unbreak joining via SSH with an explicit link address. Signed-off-by: Fabian Grünbichler --- data/PVE/CLI/pvecm.pm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/data/PVE/CLI/pvecm.pm b/data/PVE/CLI/pvecm.pm index b381f4f..fe099d4 100755 --- a/data/PVE/CLI/pvecm.pm

[pve-devel] [PATCH cluster 2/2] pvecm: pass correct nodename to finish_join

2020-05-12 Thread Fabian Grünbichler
only cosmetic, but printing the wrong nodename might cause confusion. Signed-off-by: Fabian Grünbichler --- data/PVE/CLI/pvecm.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/PVE/CLI/pvecm.pm b/data/PVE/CLI/pvecm.pm index fe099d4..107c4cb 100755 --- a/data/PVE/CLI

[pve-devel] [PATCH manager] ACME: fix fallback to implicit standalone plugin

2020-05-07 Thread Fabian Grünbichler
we need to parse the config even if it does not exist - it will return the 'standalone' entry that's needed to be backwards compatible with existing setups. Signed-off-by: Fabian Grünbichler --- Note: there is an issue when attempting to WRITE the config if /etc/pve/priv/acme does not exist

[pve-devel] [PATCH qemu-server] qmrestore: fix restore from STDIN

2020-05-07 Thread Fabian Grünbichler
the special case was dropped when moving this to pve-storage. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 8e3fadf..6461da3 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm

[pve-devel] applied: [PATCH common] network: fix adding vlan tags to bridge

2020-05-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- makes starting VMs fail, see https://forum.proxmox.com/threads/failed-to-start-vm-failed-to-remove-default-vlan-tags-of-tap104i0-command-sbin-bridge-bridge-vlan-del-dev-tap104i0-vid-1-4094-failed-exit-code.69375/ src/PVE/Network.pm | 4 ++-- 1 file changed

[pve-devel] [PATCH qemu-server] vzdump: fix template backup to stdout

2020-05-06 Thread Fabian Grünbichler
/threads/vzdump-to-stdout.69364 Signed-off-by: Fabian Grünbichler --- PVE/VZDump/QemuServer.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm index f122539..3a990cf 100644 --- a/PVE/VZDump/QemuServer.pm +++ b/PVE/VZDump

[pve-devel] [PATCH container] vzdump: use new 'pbs' option

2020-05-06 Thread Fabian Grünbichler
instead of storage config to determine whether we are in 'PBS mode' Signed-off-by: Fabian Grünbichler --- Notes: requires a break on pve-manager << version setting this option, since the dependency is the other way round src/PVE/VZDump/LXC.pm | 4 ++-- 1 file changed, 2 inse

[pve-devel] [PATCH qemu-server] vzdump: use new 'pbs' option

2020-05-06 Thread Fabian Grünbichler
instead of storage config to determine whether we are in 'PBS mode' Signed-off-by: Fabian Grünbichler --- Notes: requires a break on pve-manager << version setting this option, since the dependency is the other way round. PVE/VZDump/QemuServer.pm | 2 +- 1 file changed, 1 ins

[pve-devel] [PATCH manager] vzdump: set 'pbs' option when backing up to PBS target

2020-05-06 Thread Fabian Grünbichler
-by: Fabian Grünbichler --- Notes: this commit alone does not break anything, but since the plugins in qemu-server and pve-container can't have a versioned depends on pve-manager, we need to break the old versions of pve-manager in those two packages to ensure we get a version setting

[pve-devel] [PATCH RESEND qemu-server] migrate: cleanup forwarding code

2020-05-05 Thread Fabian Grünbichler
starts a new connection over that single socket. I took the liberty of renaming the variables/keys since I found 'tunnel_addr' and 'sock_addr' rather confusing. Reviewed-By: Mira Limbeck Tested-By: Mira Limbeck Signed-off-by: Fabian Grünbichler --- Resend for nbdstop context change PVE

[pve-devel] applied-series: [PATCH v2 container 01/10] LXC: drop unused imported locking functions

2020-05-05 Thread Fabian Grünbichler
with breaks/versioned-depends ;) On May 5, 2020 10:27 am, Fabian Ebner wrote: > From: Fabian Grünbichler > > Signed-off-by: Fabian Grünbichler > Tested-by: Fabian Ebner > --- > > Changes from v1: > * Add patch for container create_vm issue > * Add patch

[pve-devel] applied-series: [PATCH storage v5 00/17] Fix: #2124 zstd

2020-05-04 Thread Fabian Grünbichler
with an additional commit adding '--rsyncable'. thanks! On April 28, 2020 3:58 pm, Alwin Antreich wrote: > Zstandard (zstd) [0] is a data compression algorithm, in addition to > gzip, lzo for our backup/restore. It can utilize multiple core CPUs. But > by default it has one compression and one

Re: [pve-devel] [PATCH manager 2/3] Allow setting targetstorage for offline migration

2020-05-04 Thread Fabian Grünbichler
On May 4, 2020 11:20 am, Tim Marx wrote: > >> Fabian Ebner hat am 4. Mai 2020 09:26 geschrieben: >> >> >> On 5/2/20 11:40 AM, Alexandre DERUMIER wrote: >> >>> The problem is that offline migration with target storage might not >> >>> always work depending on supported export/import formats.

[pve-devel] [PATCH manager] NodeConfig: ensure locked context has current view

2020-04-30 Thread Fabian Grünbichler
similar to the recent changes for pve-guest-common - we start each API call with a cfs_update, but while we were waiting for the flock another R-M-W cycle might have happened, so we need to refresh after obtaining the lock. Signed-off-by: Fabian Grünbichler --- Notes: there's only a single

[pve-devel] [PATCH qemu-server] migrate: don't accidentally take NBD code paths

2020-04-30 Thread Fabian Grünbichler
by avoiding auto-vivification of $self->{online_local_volumes} via iteration. most code paths don't care whether it's undef or a reference to an empty list, but this caused the (already) fixed bug of calling nbd_stop without having started an NBD server in the first place. Signed-off-by: Fab

Re: [pve-devel] [PATCH container] create_vm: fix order of config creation/reading/locking

2020-04-30 Thread Fabian Grünbichler
On April 29, 2020 11:58 am, Fabian Ebner wrote: > The update_pct_config call leads to a write_config call and so the > configuration file was created before it was intended to be created. > > When the CFS is updated in between the write_config call and the > PVE::Cluster::check_vmid_unused call

Re: [pve-devel] applied: [PATCH qemu-server 2/2] qm nbdstop: cope graceful with errors

2020-04-30 Thread Fabian Grünbichler
On April 29, 2020 4:24 pm, Thomas Lamprecht wrote: > as the nbd server could have been stopped by something else. > Further, it makes no sense to die and mark the migration thus as > failed, just because of a NBD server stop issue. > > At this point the migration hand off to the target was done

Re: [pve-devel] applied: [PATCH qemu-server 1/2] migrate: only stop NBD if we got a NBD url from the target

2020-04-30 Thread Fabian Grünbichler
On April 29, 2020 4:24 pm, Thomas Lamprecht wrote: > Signed-off-by: Thomas Lamprecht > --- > > This was rather quickly assembled to fix an obvious issue, some in depth look > at this would be nice, @Fabi or @Fabian :) LGTM! > > PVE/QemuMigrate.pm | 5 - > 1 file changed, 4 insertions(+),

[pve-devel] [PATCH firewall 3/7] api: lock configs

2020-04-29 Thread Fabian Grünbichler
wherever we have a r-m-w cycle. Signed-off-by: Fabian Grünbichler --- Notes: best viewed with -w src/PVE/API2/Firewall/Aliases.pm | 80 +--- src/PVE/API2/Firewall/Cluster.pm | 36 src/PVE/API2/Firewall/Groups.pm | 52 ++- src/PVE/API2/Firewall/Host.pm

[pve-devel] [PATCH firewall 5/7] api/ipsets: parse_cidr before checking for duplicates

2020-04-29 Thread Fabian Grünbichler
for example, the config parser drops a trailing /32 for IPv4, so we should do the same here. otherwise we can have one entry for $IP and one for $IP/32 with different properties until the next R-M-W cycle drops one of them again. Signed-off-by: Fabian Grünbichler --- src/PVE/API2/Firewall

[pve-devel] [PATCH firewall 2/7] api: add locking helpers

2020-04-29 Thread Fabian Grünbichler
for ipset, rules and alias API generation modules. Signed-off-by: Fabian Grünbichler --- Notes: separated from using them for easier reviewing src/PVE/API2/Firewall/Aliases.pm | 24 src/PVE/API2/Firewall/IPSet.pm | 48 src/PVE/API2

[pve-devel] [PATCH firewall 6/7] configs: warn about duplicate ipset entries

2020-04-29 Thread Fabian Grünbichler
instead of silently dropping them when writing the config out. Signed-off-by: Fabian Grünbichler --- src/PVE/Firewall.pm | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 7b26ac5..4d86032 100644 --- a/src/PVE

[pve-devel] [PATCH firewall 7/7] rules: verify referenced security group exists

2020-04-29 Thread Fabian Grünbichler
while this was already handled properly (as empty rules), adding this as error makes it much more visible (in the GUI as well). Signed-off-by: Fabian Grünbichler --- src/PVE/Firewall.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 4d86032

[pve-devel] [RFC cluster 2/2] cfs_lock: re-raise exceptions

2020-04-29 Thread Fabian Grünbichler
so that API paths that raise an exception while holding a CFS lock properly propagate that exception to the client, instead of the stringified version with added noise about locks added to the front. Signed-off-by: Fabian Grünbichler --- Notes: there seems to be nothing that matches

[pve-devel] [PATCH firewall 4/7] clone_vmfw_conf: lock new config

2020-04-29 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: best viewed with -w src/PVE/Firewall.pm | 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index eda39eb..7b26ac5 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE

[pve-devel] [PATCH firewall 1/7] configs: add locking helpers

2020-04-29 Thread Fabian Grünbichler
to allow some level of safe concurrent config modification, instead of the current free for all. Signed-off-by: Fabian Grünbichler --- Notes: require pve-cluster that provides cfs_lock_firewall, or switching to cfs_lock_domain as mentioned in pve-cluster#1 lock_hostfw_conf

[pve-devel] [PATCH firewall/cluster 0/9] add locking to firewall config changes

2020-04-29 Thread Fabian Grünbichler
the second cluster patch is optional, but improves usability of non-worker API calls that do cfs_lock_foo(..., sub { raise_foo }); the last three firewall patches are unrelated bug fixes that I found while testing. pve-cluster: Fabian Grünbichler (2): cfs_lock: add firewall lock helper

[pve-devel] [PATCH cluster 1/2] cfs_lock: add firewall lock helper

2020-04-29 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- alternatively we could re-use 'cfs_lock_domain', which is currently only used by HA and was intended as general-purpose cfs_lock wrapper.. I'd shorten the firewall- prefix to fw- in that case though. domain-fw-host-$foo might be more confusing to end users

Re: [pve-devel] [PATCH guest-common 1/3] snapshot_rollback: flock earlier

2020-04-27 Thread Fabian Grünbichler
On April 27, 2020 1:08 pm, Fabian Ebner wrote: > One not-patch-related observation inline. > > On 27.04.20 10:24, Fabian Grünbichler wrote: >> to protect checks against concurrent modifications >> >> Signed-off-by: Fabian Grünbichler >> --- >> >> No

[pve-devel] [PATCH guest-common 3/3] lock_config: rename lock_config_mode -> lock_config_shared

2020-04-27 Thread Fabian Grünbichler
only has a single user (qemu-server's clone API call) currently. Signed-off-by: Fabian Grünbichler --- Notes: requires breaks on qemu-server that uses lock_config_mode possibly better to postpone since it does not fix an actual issue but just improves the readability/des

[pve-devel] [PATCH container 2/2] api/destroy: repeat early checks after locking

2020-04-27 Thread Fabian Grünbichler
and check_lock before forking as well Signed-off-by: Fabian Grünbichler --- Notes: bested viewed with -w --patience src/PVE/API2/LXC.pm | 31 --- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm index

[pve-devel] [PATCH qemu-server 1/3] QemuServer: drop unused imported locking functions

2020-04-27 Thread Fabian Grünbichler
lock_file is used by PVE::QemuServer::Memory, but it does properly 'use PVE::Tools ...' itself so we can drop them in the main module. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE

[pve-devel] [PATCH qemu-server 2/3] api/destroy: repeat early checks after lock

2020-04-27 Thread Fabian Grünbichler
to protect against concurrent changes Signed-off-by: Fabian Grünbichler --- best viewed with --patience -w PVE/API2/Qemu.pm | 40 +++- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index ec4c18c..f6a98f0

[pve-devel] [PATCH guest-common 2/3] snapshot_delete: check for concurrent modifications at each step

2020-04-27 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/AbstractConfig.pm | 14 ++ 1 file changed, 14 insertions(+) diff --git a/PVE/AbstractConfig.pm b/PVE/AbstractConfig.pm index 70311df..8ec27a6 100644 --- a/PVE/AbstractConfig.pm +++ b/PVE/AbstractConfig.pm @@ -823,6 +823,15 @@ sub

[pve-devel] [PATCH container 1/2] LXC: drop unused imported locking functions

2020-04-27 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- src/PVE/LXC.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm index fbe736e..651ec08 100644 --- a/src/PVE/LXC.pm +++ b/src/PVE/LXC.pm @@ -20,8 +20,8 @@ use PVE::SafeSyslog; use PVE::INotify; use PVE

[pve-devel] [PATCH qemu-server 3/3] clone: use new config_lock_shared

2020-04-27 Thread Fabian Grünbichler
and move the lock call and decision logic closer together Signed-off-by: Fabian Grünbichler --- Notes: needs libpve-guest-common-perl with new lock_config_shared possibly better to postpone since it does not fix an actual issue but just improves the readability PVE/API2/Qemu.pm

[pve-devel] [PATCH guest-common 1/3] snapshot_rollback: flock earlier

2020-04-27 Thread Fabian Grünbichler
to protect checks against concurrent modifications Signed-off-by: Fabian Grünbichler --- Notes: bested viewed with --patience -w PVE/AbstractConfig.pm | 45 +-- 1 file changed, 22 insertions(+), 23 deletions(-) diff --git a/PVE/AbstractConfig.pm b

[pve-devel] [PATCH qemu-server] api/resume: make nocheck root-only

2020-04-27 Thread Fabian Grünbichler
this is only used for migration via 'qm mtunnel', regular users should never need to resume a VM that does not logically belong to the node it is running on Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2

Re: [pve-devel] [PATCH manager] certs: early renew long-lived certificates

2020-04-24 Thread Fabian Grünbichler
On April 23, 2020 9:42 pm, Thomas Lamprecht wrote: > On 4/23/20 1:59 PM, Fabian Grünbichler wrote: >> On April 23, 2020 1:07 pm, Dominik Csapak wrote: >>> LGTM >>> >>> maybe we should shorten the lifespan to 1 year already? >>> accord

Re: [pve-devel] [PATCH manager] certs: early renew long-lived certificates

2020-04-23 Thread Fabian Grünbichler
not affected (for now). I don't have any objections to shortening both the issuance and the check here to 1 year though. > On 4/23/20 12:20 PM, Fabian Grünbichler wrote: >> if our self-signed certificate expires in more than 825 days, but was >> created after July 2019 it won't be acc

[pve-devel] [PATCH manager] certs: early renew long-lived certificates

2020-04-23 Thread Fabian Grünbichler
signature algorithms, or anything else in that direction. Signed-off-by: Fabian Grünbichler --- I'd also be fine with reducing both even more, e.g. to 1 year ;) bin/pveupdate | 15 --- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/bin/pveupdate b/bin/pveupdate index

Re: [pve-devel] [PATCH common 1/1] JSONSchema: extend pve-configid regex by '-'

2020-04-23 Thread Fabian Grünbichler
ill like to see as follow up is re-using this verifier for the guest config parsers when parsing the snapshot section header (by chance, they already contain the '-', but the next time we add a character that might not be the case and we might miss it which would cause some fun bugs ;)). Reviewed-By: Fabian Grü

[pve-devel] [PATCH docs] certs: extend ACME section with DNS/plugin info

2020-04-22 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- certificate-management.adoc | 131 ++-- 1 file changed, 125 insertions(+), 6 deletions(-) diff --git a/certificate-management.adoc b/certificate-management.adoc index db76062..dc2834b 100644 --- a/certificate-management.adoc

Re: [pve-devel] [PATCH qemu-server] fix #2697: map netdev_add options to correct json types

2020-04-21 Thread Fabian Grünbichler
there is some issue with hot-plugging queues (with and without your patch): - virtio NIC, no queues set, start VM OR - virtio NIC, queues X set, start VM - change NIC to have a queues value of Y 400 Parameter verification failed. net0: hotplug problem - error on hot-unplugging device 'net0' -

Re: [pve-devel] [PATCH common 1/2] JSONSchema: add format validator support and cleanup check_format

2020-04-21 Thread Fabian Grünbichler
sorry, took a bit longer than I wanted, but here it is. thanks for looking into this! :) at first glance it seemed like it's more straight-forward than I expected - but as always with JSONSchema.pm, some rabbit-hole chasing later I am not quite so sure anymore. I hope my walls of text below

[pve-devel] applied: [PATCH container] Fix move_volume by using result from parse_volume

2020-04-21 Thread Fabian Grünbichler
thanks! On April 20, 2020 1:12 pm, Fabian Ebner wrote: > This was changed by accident by my refactoring in > commit e4034859fd0e3491fd1aefb4f9ef44ee585aa404 > > Signed-off-by: Fabian Ebner > --- > > AFAICS the other call sites affected by the refactoring > still use the results, but it might

[pve-devel] applied: [PATCH qemu-server] Fix live migration with replicated unused volumes

2020-04-20 Thread Fabian Grünbichler
with fixup to improve readability: my $number_of_online_replicated_volumes = 0; -if ($self->{online_local_volumes}) { - foreach my $volid (keys %{$self->{replicated_volumes}}) { - next if !(grep { $volid eq $_ } @{$self->{online_local_volumes}}); +foreach my $volid

Re: [pve-devel] [PATCH v3 storage 2/3] Incorporate wipe_disks from PVE::Ceph::Tools

2020-04-16 Thread Fabian Grünbichler
On April 16, 2020 8:57 am, Dominik Csapak wrote: > >> >> >> Thanks for clearing that up, I mostly poked at it as Dominic came to me >> asking what to do as you told him to use [0-9] and Fabian told him to go >> back again, not knowing about your suggestion. >> >> The question is, does the

[pve-devel] applied: [PATCH storage 2/2] fix nvme wearout parsing

2020-04-15 Thread Fabian Grünbichler
with switch to \s* instead of .*?, as discussed. On April 14, 2020 4:17 pm, Dominik Csapak wrote: > the '.*' was greedy, also consuming all but one digits of the real percentage > > Signed-off-by: Dominik Csapak > --- > PVE/Diskmanage.pm | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-)

[pve-devel] applied: [PATCH storage 1/2] disk_tests: improve nvme smart test

2020-04-15 Thread Fabian Grünbichler
On April 14, 2020 4:17 pm, Dominik Csapak wrote: > by using an actual percentage for 'Percentage Used' instead of 0% > > Signed-off-by: Dominik Csapak > --- > this test now fails, but is repaired by the next patch of the series > > test/disk_tests/nvme_smart/disklist_expected.json | 2 +-

Re: [pve-devel] [PATCH v5 qemu-server 14/19] Update volume IDs in one go

2020-04-15 Thread Fabian Grünbichler
On April 15, 2020 10:04 am, Fabian Ebner wrote: > On 14.04.20 12:41, Fabian Ebner wrote: >> On 09.04.20 09:53, Fabian Grünbichler wrote: >>> small nit inline >>> >>> On April 8, 2020 11:25 am, Fabian Ebner wrote: >>>> Use 'update_volume_ids' for t

Re: [pve-devel] [PATCH v3 storage 2/3] Incorporate wipe_disks from PVE::Ceph::Tools

2020-04-15 Thread Fabian Grünbichler
some style/design nits below On March 11, 2020 2:05 pm, Dominic Jäger wrote: > Move wipe_disks from PVE::Ceph::Tools to PVE::Diskmanage and improve it by > - Handling invalid parameters > - Adding options for wiping > - Making names clearer > - Adding tests > > Relies on the corresponding

[pve-devel] applied: [PATCH manager 1/3] ceph: remove unused variable assignment

2020-04-15 Thread Fabian Grünbichler
On March 11, 2020 4:22 pm, Alwin Antreich wrote: > Signed-off-by: Alwin Antreich > --- > PVE/Ceph/Services.pm | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/PVE/Ceph/Services.pm b/PVE/Ceph/Services.pm > index c17008cf..7015cafe 100644 > --- a/PVE/Ceph/Services.pm > +++

[pve-devel] applied: [PATCH manager 2/3] Fix: ceph: mon_address not considered by new MON

2020-04-15 Thread Fabian Grünbichler
On March 11, 2020 4:22 pm, Alwin Antreich wrote: > The public_addr option for creating a new MON is only valid for manual > startup (since Ceph Jewel) and is just ignored by ceph-mon during setup. > As the MON is started after the creation through systemd without an IP > specified. It is trying to

Re: [pve-devel] [PATCH manager 3/3] Fix #2422: allow multiple Ceph public networks

2020-04-15 Thread Fabian Grünbichler
On March 11, 2020 4:22 pm, Alwin Antreich wrote: > Multiple public networks can be defined in the ceph.conf. The networks > need to be routed to each other. > > On first service start the Ceph MON will register itself with one of the > IPs configured locally, matching one of the public networks

  1   2   3   4   5   6   7   8   9   10   >