Re: [pve-devel] [PATCH manager] pve5to6: Add warning for some Gluster versions

2019-09-02 Thread Fabian Grünbichler
On August 28, 2019 1:25 pm, Dominic Jäger wrote: > After Fabian had sent his patch [0] we actually thought that this patch > for pve5to6 would be obsolete. Upgrades as well as fresh installs worked > flawlessly in my tests. Consequently, I removed the hint from the > upgrade documentation [1] an

Re: [pve-devel] applied: [RFC container] mountpoints: create parent dirs with correct owner

2019-09-02 Thread Fabian Grünbichler
On August 26, 2019 1:30 pm, Thomas Lamprecht wrote: > On 24.07.19 13:37, Fabian Grünbichler wrote: >> otherwise unprivileged containers might end up with directories that >> they cannot modify since they are owned by the user root in the host >> namespace, instead of root

Re: [pve-devel] [PATCH qemu-server] fix #1934: add qemu fw_cfg variables via 'tags'

2019-09-02 Thread Fabian Grünbichler
On August 23, 2019 2:03 pm, Dominik Csapak wrote: > this add the 'tags' property to vms, which has the format: why 'tags'? seems rather generic for what it does ;) > > key=value(;key=value)* > > each value will be set as > > -fw_cfg 'name=opt/com.proxmox/$key,string=$value' > (qemu recommends

Re: [pve-devel] Gitlab-ci

2019-09-02 Thread Fabian Grünbichler
On August 27, 2019 11:24 am, Alexandre DERUMIER wrote: > Hi, > > I think 1 benefit of gitlab,github,... is the tracking of merge request. IMHO this is a big downside of Github and Gitlab - cleanly checking out an older iteration of a PR/MR is very cumbersome (and wasn't even really possible for

Re: [pve-devel] [PATCH qemu-server] fix #1934: add qemu fw_cfg variables via 'tags'

2019-09-02 Thread Fabian Grünbichler
On September 2, 2019 2:27 pm, Thomas Lamprecht wrote: > On 9/2/19 2:14 PM, Fabian Grünbichler wrote: >> On August 23, 2019 2:03 pm, Dominik Csapak wrote: >>> this add the 'tags' property to vms, which has the format: >> >> why 'tags'? seems rathe

Re: [pve-devel] [PATCH qemu-server] fix #1934: add qemu fw_cfg variables via 'tags'

2019-09-04 Thread Fabian Grünbichler
On September 4, 2019 8:49 am, Dominik Csapak wrote: > On 9/2/19 2:14 PM, Fabian Grünbichler wrote: >> On August 23, 2019 2:03 pm, Dominik Csapak wrote: >>> this add the 'tags' property to vms, which has the format: >> >> why 'tags'? seems r

Re: [pve-devel] [RFC qemu-server] rewrite description for vm_config

2019-09-05 Thread Fabian Grünbichler
On September 4, 2019 3:59 pm, Oguz Bektas wrote: > the description doesn't match the default behaviour, which is to replace > the current values with pending ones in the returned config, unless the > 'current' option is passed. > > Signed-off-by: Oguz Bektas > --- > > i tried to come up with a r

Re: [pve-devel] [PATCH v3 guest-common 1/2] fix #1291: implement remove_vmid_from_backup_jobs

2019-09-05 Thread Fabian Grünbichler
On September 4, 2019 4:41 pm, Thomas Lamprecht wrote: > On 01.07.19 15:43, Christian Ebner wrote: >> remove_vmid_from_backup_jobs updates the vzdump.cron backup jobs, >> excluding the given vmid. >> >> Signed-off-by: Christian Ebner >> --- >> PVE/VZDump/Plugin.pm | 46 +++

[pve-devel] applied: [PATCH common] add postinst hook to fix /etc/aliases whitespace error

2019-09-05 Thread Fabian Grünbichler
thanks to all three involved people ;) On September 3, 2019 1:06 pm, Thomas Lamprecht wrote: > This was wrongly shipped by our ISO since quite a bit (AFAICT, at > least 4.x), so fix it up in a versioned postinst snippet. > > Do so by usind sed with the following pattern: > # sed -E -i -e 's/^www

Re: [pve-devel] [PATCH ha-manager 0/4] Add inital HW based fencing

2019-09-06 Thread Fabian Grünbichler
On March 27, 2019 5:42 pm, Thomas Lamprecht wrote: > Actually this'd be v8, but it's to old and to unreviewed for anybody do > remember anyway, so best seen as new series. > > Thomas Lamprecht (4): > allow LRM lock stealing for fenced nodes > allow use of external fencing devices > send also

Re: [pve-devel] [PATCH ha-manager 4/4] add some infos about HW fencing to README

2019-09-06 Thread Fabian Grünbichler
for reference docs IMHO. one nit inline, otherwise Acked-by: Fabian Grünbichler > --- > README | 110 + > 1 file changed, 110 insertions(+) > > diff --git a/README b/README > index 1c5177f..dbf8d6a 100644 > --- a/READ

Re: [pve-devel] [PATCH ha-manager 3/4] send also email on hardware fence failure

2019-09-06 Thread Fabian Grünbichler
On March 27, 2019 5:42 pm, Thomas Lamprecht wrote: > We introduced sending an email when Fencing fails/succeeds in > general some time ago, send now also an email if a HW fence > fails missing S-O-B > --- > src/PVE/HA/NodeStatus.pm | 7 --- > 1 file changed, 4 insertions(+), 3 deletions(-) >

Re: [pve-devel] [PATCH ha-manager 2/4] allow use of external fencing devices

2019-09-06 Thread Fabian Grünbichler
On March 27, 2019 5:42 pm, Thomas Lamprecht wrote: > A node now can be fenced with the use of external hardware fence > devices. > Those device can be configured at /etc/pve/ha/fence.cfg > also the fencing option in the datacenter configuration file must > be set to either 'hardware' or 'both', els

Re: [pve-devel] [PATCH ha-manager 1/4] allow LRM lock stealing for fenced nodes

2019-09-06 Thread Fabian Grünbichler
r want to start a service on two nodes! small nits inline, otherwise Acked-by: Fabian Grünbichler > --- > src/PVE/HA/Env.pm | 4 ++-- > src/PVE/HA/Env/PVE2.pm | 4 ++-- > src/PVE/HA/Sim/Env.pm | 16 ++-- > 3 files changed, 14 insertions(+), 10 deletions(-) &

Re: [pve-devel] [PATCH qemu 1/7] Trigger pve-api-updates on update

2019-09-06 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > A QEMU update can change supported CPU flags, thus we need to restart > API services (especially pvestatd) to refresh their cached values. given the recent discussion about this exact change, we could also use PVE::QemuServer::kvm_user_version(

Re: [pve-devel] [PATCH manager 2/7] Broadcast supported CPU flags

2019-09-06 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > pvestatd will read supported CPU flags once on startup (since these > never change during runtime, and QEMU updates trigger a service > restart), then broadcasts them as a key-value pair to the cluster. > > Signed-off-by: Stefan Reiter > --- >

Re: [pve-devel] [PATCH qemu-server 3/7] Add QEMU CPU flag querying helpers

2019-09-06 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > * query_understood_cpu_flags returns all flags that QEMU/KVM knows about > * query_supported_cpu_flags returns all flags that QEMU/KVM can use on > this particular host. > > To get supported flags, a temporary VM is started with QEMU, so we ca

Re: [pve-devel] [PATCH qemu-server 4/7] Add CustomCPUConfig for storing/parsing custom CPU models

2019-09-09 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > Inherits from SectionConfig to provide base parsing infrastructure. > > Use with helper functions: > * config_from_file gives bless'd config > * get_model_by_name returns a "formatted" hash for a single CPU model > * config_to_file writes change

Re: [pve-devel] [PATCH qemu-server 6/7] Handle CPU flags defined in custom CPU type

2019-09-09 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > Special care is taken not to overwrite any special flags, or ones > manually set on the VM by the user. We warn if a flag is overruled. hmm. I am unsure whether I like that behaviour or not. it's a bit strange that VM specific flags get applied

Re: [pve-devel] [PATCH qemu-server 5/7] Support custom CPU types in get_cpu_options

2019-09-09 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > Supports custom basemodels (model shown to QEMU, i.e. must be a default nit: s/shown/known/ ? high-level: if we allow basing custom models on other custom models, wouldn't we need to compute an effective model first, and then use that? e.g.,

Re: [pve-devel] [PATCH qemu-server 7/7] Allow custom CPU types in API

2019-09-09 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > Custom CPU types can be specified via the API, but to prevent arbitrary > ones we have to manually check if the given model exists (as default or > custom). > > Signed-off-by: Stefan Reiter > --- > PVE/QemuServer.pm | 31 ++

Re: [pve-devel] [PATCH 0/7] Add basics for custom CPU models

2019-09-09 Thread Fabian Grünbichler
On September 2, 2019 4:27 pm, Stefan Reiter wrote: > Based on the RFC and following on- and off-list discussion about custom CPU > models [0]. > > In essence, this revised patch allows a user to specify custom CPU models in > /etc/pve/cpu-models.conf (section-config style [1]), where VMs using tha

Re: [pve-devel] [PATCH qemu-server 3/7] Add QEMU CPU flag querying helpers

2019-09-09 Thread Fabian Grünbichler
On September 9, 2019 2:56 pm, Stefan Reiter wrote: > On 9/6/19 1:42 PM, Fabian Grünbichler wrote: >>> +# this is neither the default for 'query_supported_cpu_flags' below, nor >>> for >>> +# /proc/cpuinfo. >>> +# >>> +# To compare (o

Re: [pve-devel] [PATCH container] fix issue where ttys aren't correctly set after restore

2019-09-10 Thread Fabian Grünbichler
NAK, see inline On September 10, 2019 3:27 pm, Oguz Bektas wrote: > restore from unpriv to priv causes a problem with the log-in, since the > /etc/securetty file isn't modified after a restore to reflect the change > (/dev/lxc/tty1 and so on). > > template_fixup is normally called in post_create_

Re: [pve-devel] [PATCH guest-common 1/2] move pending changes related functions into AbstractConfig

2019-09-11 Thread Fabian Grünbichler
meta: please send v2 together with v2 of pve-container (and future) patches. On September 4, 2019 6:00 pm, Oguz Bektas wrote: > some functions from Qemu w.r.t. pending changes can be moved to > AbstractConfig, in order to make them available for both QemuConfig and > LXC::Config. > > Signed-off-b

Re: [pve-devel] [PATCH container 2/9] adapt config GET call for taking pending changes

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > the default behaviour is the same as in Qemu, so without the 'current' > flag set, current values will be replaced with their respective pending > counterparts. > > Signed-off-by: Oguz Bektas > --- > src/PVE/API2/LXC/Config.pm | 23 +

Re: [pve-devel] [PATCH container 1/9] add pending section to lxc config parser/writer

2019-09-11 Thread Fabian Grünbichler
NAK, this breaks existing configs with a snapshot called pending (which is an allowed snapshot name, and not such an uncommon word that we can confidently say that noone would be affected). we could do _PENDING_ or __PENDING__ (similar to what we do with __base__ and __replicate__ and __migrati

Re: [pve-devel] [PATCH container 3/9] adapt config PUT to use 'revert' and 'force' parameters

2019-09-11 Thread Fabian Grünbichler
the commit message is misleading - this just adds two parameters, but no code that handles them? the code/change that adds handling only comes in patch #8. either merge these two, or put this one after the other please. On September 5, 2019 4:11 pm, Oguz Bektas wrote: > Signed-off-by: Oguz Bekt

Re: [pve-devel] [PATCH container 5/9] add 'pending' API method to LXC

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > analog to Qemu, it returns an array of elements, which shows the > current value, pending value, and delete requests. and again, this is completely identical to the Qemu API path (modulo comments, one superfluous next statement) except for the ci

Re: [pve-devel] [PATCH container 6/9] add 'pct pending' command

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > analog to 'qm pending', it shows a list of keys and values defined in > configuration. copied verbatim from qm.pm, maybe we could move this code to GuestHelpers.pm (format_pending)? > > cur: current change > new: pending change > del: pending d

Re: [pve-devel] [PATCH container 7/9] add vmconfig_hotplug_pending and vmconfig_apply_pending

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > vmconfig_hotplug_pending is responsible for checking if a key in the > pending section is hotpluggable, if yes; perform a generic config value > replace or perform specific actions if a special case. > > vmconfig_apply_pending is only supposed to

Re: [pve-devel] [PATCH container 9/9] apply pending changes when container is started

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > Signed-off-by: Oguz Bektas > --- > src/PVE/LXC.pm | 7 +++ > 1 file changed, 7 insertions(+) > > diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm > index 475d9be..9dd5bc9 100644 > --- a/src/PVE/LXC.pm > +++ b/src/PVE/LXC.pm > @@ -1939,6 +1939,13

Re: [pve-devel] [PATCH container 0/9] lxc pending changes

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > this series makes it possible to add/revert/delete pending changes in > the backend. > > it depends on my previous patch for pve-guest-common, for enabling > inheritance of pending changes related methods into PVE::LXC::Config thanks for these pa

Re: [pve-devel] [PATCH container 8/9] rework update_pct_config to write and apply pending changes

2019-09-11 Thread Fabian Grünbichler
On September 5, 2019 4:11 pm, Oguz Bektas wrote: > use vmconfig_hotplug_pending (when ct up) and vmconfig_apply_pending > (when ct down) to apply or write pending changes. > > Signed-off-by: Oguz Bektas > --- > src/PVE/API2/LXC/Config.pm | 52 +- > src/PVE/LXC/Config.pm | 328 +

Re: [pve-devel] [PATCH container 1/9] add pending section to lxc config parser/writer

2019-09-12 Thread Fabian Grünbichler
On September 11, 2019 2:02 pm, Thomas Lamprecht wrote: > On 11.09.19 09:39, Fabian Grünbichler wrote: >> NAK, this breaks existing configs with a snapshot called pending (which >> is an allowed snapshot name, and not such an uncommon word that we can >> confidently sa

[pve-devel] [PATCH ceph] build: use dgit for download target

2019-09-12 Thread Fabian Grünbichler
since it has all the necessary wrapping to download and verify dscs from arbitraty repositories, without being affected by the system's APT state and configuration. Signed-off-by: Fabian Grünbichler --- Makefile | 13 +++-- upstream-key.asc | 29 +++

Re: [pve-devel] [PATCH container] don't duplicate lxc.idmap entries during restore

2019-09-25 Thread Fabian Grünbichler
On September 24, 2019 4:04 pm, Oguz Bektas wrote: > merging $conf->{lxc} causes lxc.idmap entries to be duplicated in the > restored configuration. instead, we can overwrite the contents from the > extracted configuration file. this way we don't duplicate these entries. > (having duplicate idmap en

Re: [pve-devel] [PATCH v2 container] don't duplicate lxc.idmap entries during restore

2019-09-25 Thread Fabian Grünbichler
er via recover_config, OR via restore_configuration. non-root behaviour stays the same. Signed-off-by: Fabian Grünbichler --- src/PVE/API2/LXC.pm | 4 ++-- src/PVE/LXC/Create.pm | 6 +- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm index 0728

Re: [pve-devel] libknet1 update don't restart corosync

2019-09-27 Thread Fabian Grünbichler
On September 27, 2019 8:53 am, Alexandre DERUMIER wrote: > Hi, > > I have noticed that when you upgrade libknet1 (and fix the crash of corosync), > > corosync is not auto restarted. yes, just like any other library except libc6 in Debian. there is tooling to handle this in general ('needrestart

Re: [pve-devel] [PATCH ha-manager 09/13] Add crm command 'stop'

2019-09-27 Thread Fabian Grünbichler
one comment inline On September 26, 2019 1:38 pm, Fabian Ebner wrote: > Not every command parameter is 'target' anymore, so > it was necessary to modify the parsing of $sd->{cmd}. > > Just changing the state to request_stop is not enough, > we need to actually update the service configuration as

Re: [pve-devel] [PATCH kernel] backport new FPU register copy helpers

2019-09-27 Thread Fabian Grünbichler
small nit inline, otherwise Acked-by: Fabian Grünbichler On September 27, 2019 12:49 pm, Thomas Lamprecht wrote: > This allows us to fix the ZFS SIMD patch for 5.0 kernel way easier, > as we can use the same code path as used for 5.2 and newer kernels > there. > > The helper i

Re: [pve-devel] [PATCH zfsonlinux] [SIMD]: FPU register save/restore is also required on 5.0 kernels

2019-09-27 Thread Fabian Grünbichler
Acked-by: Fabian Grünbichler On September 27, 2019 12:49 pm, Thomas Lamprecht wrote: > Do a very minimal fix, effectively just making the code path for 5.0 > to 5.1 and 5.2 to 5.3 the same one, which can be done by backporting > the new copy_kernel helpers into the kernel, as the exis

[pve-devel] applied: [PATCH RESEND container] restore lxc.* entries once

2019-09-30 Thread Fabian Grünbichler
either via recover_config, OR via restore_configuration. non-root behaviour stays the same. Tested-by: Oguz Bektas Signed-off-by: Fabian Grünbichler --- Note: added Tested-by src/PVE/API2/LXC.pm | 4 ++-- src/PVE/LXC/Create.pm | 16 ++-- 2 files changed, 8 insertions(+), 12

Re: [pve-devel] [PATCH pve-zsync] Allow detecting a syncing instance of a job

2019-10-01 Thread Fabian Grünbichler
On September 30, 2019 12:55 pm, Fabian Ebner wrote: > Before, the check whether a syncing instance of the same job is already > present > was inside the locked section. This caused cron to continuously spawn new > instances of pve-zsync on syncs (or rather groups of syncs) taking longer > than 15

Re: [pve-devel] [PATCH storage] Use bigger timeouts for zfs operations

2019-10-01 Thread Fabian Grünbichler
On October 1, 2019 12:17 pm, Fabian Ebner wrote: > Seems like 'zfs destroy' can take longer than 5 seconds, see [0]. > I changed the timeout to 15 seconds and also changed the default > timeout to 10 instead of 5 seconds, to be on the safe side > for other commands like 'zfs create'. > > [0]: > h

Re: [pve-devel] [PATCH v2 guest-common 01/18] refactor pending changes related config code into AbstractConfig

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > also use a better naming scheme for methods: > > split_flagged_list -> parse_pending_delete > join_flagged_list -> print_pending_delete > vmconfig_delete_pending_option -> add_to_pending_delete > vmconfig_undelete_pending_option -> remove_from_pe

Re: [pve-devel] [PATCH v2 guest-common 02/18] refactor method used by config GET calls into AbstractConfig

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > since this method will be both used by qemu and lxc config GET calls, it > makes sense to move it into AbstractConfig. only difference is that qemu > also hides the cipassword when it's set. this can be handled by having > qemu overwrite the metho

Re: [pve-devel] [PATCH v2 container 08/18] add lxc/pending API path

2019-10-02 Thread Fabian Grünbichler
you completely ignored my comments for v1 of this patch? the whole code is identical to qemu-server's, except for cipassword handling. pending changes are also encoded identically for both pve-container and qemu-server, so it makes sense to move this to AbstractConfig or GuestHelpers.pm with an

Re: [pve-devel] [PATCH v2 container 10/18] adapt CT config parser for pending changes

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > config parser can now read/write [pve:pending] section. this was named > such, instead of [PENDING], after on- and offline discussion regarding > namespacing the pending section and snapshots. > > this also adds an optional non-capturing regex gr

Re: [pve-devel] [PATCH v2 container 10/18] adapt CT config parser for pending changes

2019-10-02 Thread Fabian Grünbichler
On October 2, 2019 12:22 pm, Thomas Lamprecht wrote: > On 9/30/19 2:44 PM, Oguz Bektas wrote: >> config parser can now read/write [pve:pending] section. this was named >> such, instead of [PENDING], after on- and offline discussion regarding >> namespacing the pending section and snapshots. >> >>

Re: [pve-devel] [PATCH v2 container 16/18] adapt config PUT method for the new update_pct_config

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > we don't need to extract 'delete' here, instead we pass it all as $param > and extract 'delete', 'revert' and most other things in > update_pct_config I already asked that in v1 - wouldn't it make sense to keep the parameter checks in the API ca

Re: [pve-devel] [PATCH v2 container 17/18] rework update_pct_config to write into pending section

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > use vmconfig_hotplug_pending or vmconfig_apply_pending to apply the > pending changes, depending on whether the CT is on- or offline. > > Signed-off-by: Oguz Bektas > --- > src/PVE/LXC/Config.pm | 334 ++

Re: [pve-devel] [PATCH v2 container 18/18] add vmconfig_hotplug_pending and vmconfig_apply_pending

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > vmconfig_hotplug_pending is responsible for checking if a key/value pair in > the > pending section can be hotplugged, if yes; perform a generic replace, > or perform specific actions for hotplugging the special cases. > > vmconfig_apply_pending

Re: [pve-devel] [PATCH v2 00/18] lxc pending changes

2019-10-02 Thread Fabian Grünbichler
On September 30, 2019 2:44 pm, Oguz Bektas wrote: > this series makes it possible to add/delete/revert pending changes in > the backend for containers. > > this v2 took longer than expected, mainly because there were small bugs > popping up everywhere, everytime i tried to change anything :) > >

[pve-devel] [PATCH access-control] parse_user_cfg: correctly parse group names in ACLs

2019-10-03 Thread Fabian Grünbichler
group reference in an ACL can only contain one '@' (as first character). Signed-off-by: Fabian Grünbichler --- Notes: alternatively, we could also disallow usernames starting with '@', but those are currently working as long as they just have ACLs via groups, and not

Re: [pve-devel] [PATCH v2 container 16/18] adapt config PUT method for the new update_pct_config

2019-10-04 Thread Fabian Grünbichler
On October 3, 2019 4:32 pm, Oguz Bektas wrote: > On Wed, Oct 02, 2019 at 01:52:58PM +0200, Fabian Grünbichler wrote: >> On September 30, 2019 2:44 pm, Oguz Bektas wrote: >> > we don't need to extract 'delete' here, instead we pass it all as $param >> > and

Re: [pve-devel] [PATCH qemu-server] fix #2395: check for iscsi on efidisk creation

2019-10-06 Thread Fabian Grünbichler
On October 4, 2019 10:55 am, Dominik Csapak wrote: > otherwise qemu-img uses its default intiator id which may not have access isn't this also missing from PVE/QemuServer/Cloudinit.pm PVE/QemuServer/ImportDisk.pm and isn't the 'cache=none' addition for ZFS missing from all three? maybe it would

Re: [pve-devel] [PATCH qemu-server] Fix #2171: VM statefile was not activated

2019-10-07 Thread Fabian Grünbichler
On October 7, 2019 2:41 pm, Alwin Antreich wrote: > Machine states that were created on snapshots with memory could not be > restored on rollback. The state volume was not activated so KVM couldn't > load the state. > > This patch moves the path generation into vm_start and de-/activates the > sta

Re: [pve-devel] [PATCH qemu-server] Fix #2171: VM statefile was not activated

2019-10-08 Thread Fabian Grünbichler
On October 8, 2019 11:25 am, Alwin Antreich wrote: > On Tue, Oct 08, 2019 at 08:36:57AM +0200, Fabian Grünbichler wrote: >> On October 7, 2019 2:41 pm, Alwin Antreich wrote: >> > Machine states that were created on snapshots with memory could not be >> > restored on rol

Re: [pve-devel] [PATCH storage] Use bigger timeouts for zfs operations

2019-10-10 Thread Fabian Grünbichler
On October 10, 2019 8:55 am, Fabian Ebner wrote: > On 10/1/19 12:28 PM, Fabian Grünbichler wrote: >> On October 1, 2019 12:17 pm, Fabian Ebner wrote: >>> Seems like 'zfs destroy' can take longer than 5 seconds, see [0]. >>> I changed the timeout to 15

Re: [pve-devel] [PATCH qemu-server] Fix #2412: Missing VMs in pools

2019-10-11 Thread Fabian Grünbichler
On October 11, 2019 11:55 am, Dominic Jäger wrote: > Between destroying a VM (unlink config file) and removing it from user.cfg > creating a new VM with the ID that is still in use in user.cfg was possible. > VMs could go missing as a consequence. > > Adding a lock solves this. This lock does not

Re: [pve-devel] [PATCH v4 qemu 02/12] Write understood CPU flags into static file

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > located at /usr/share/kvm/cpu-flags-understood-$arch > > This file can be read by qemu-server's "query_understood_cpu_flags" > function, avoiding a more expensive call to QEMU. > > For now, only x86_64 is implemented, since aarch64 doesn't print

Re: [pve-devel] [PATCH v4 manager 01/12] Broadcast supported CPU flags

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > pvestatd will check if the KVM version has changed using > kvm_user_version (which automatically clears its cache if QEMU/KVM > updates), and if it has, query supported CPU flags and broadcast them as > key-value pairs to the cluster. > > If detec

Re: [pve-devel] [PATCH v4 qemu-server 05/12] Add CPUConfig file and migrate some helpers

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > The package will be used for custom CPU models as a SectionConfig, hence > the name. For now we simply move some CPU related helper functions and > declarations over from QemuServer to reduce clutter there. > > qemu_machine_feature_enabled is move

Re: [pve-devel] [PATCH v4 qemu-server 07/12] Add overrides and convenience functions to CPUConfig

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > Add two overrides to avoid writing redundant information to the config > file. > > get_model_by_name is used to return a cpu config with default values > filled out. > > Signed-off-by: Stefan Reiter > --- > > v3 -> v4: > * add is_custom_model >

Re: [pve-devel] [PATCH v4 qemu-server 06/12] Adapt CPUConfig to handle custom models

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > Turn CPUConfig into a SectionConfig with parsing/writing support for > custom CPU models. IO is handled using cfs. > > Namespacing will be provided using "custom-" prefix for custom model > names (in VM config only, cpu-models.conf will contain un

Re: [pve-devel] [PATCH v4 qemu-server 08/12] Verify VM-specific CPU configs seperately

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > $cpu_fmt is being reused for custom CPUs as well as VM-specific CPU > settings. The "pve-vm-cpu-conf" format is introduced to verify a config > specifically for use as VM-specific settings. > > Signed-off-by: Stefan Reiter > --- > > v3 -> v4: >

Re: [pve-devel] [PATCH v4 qemu-server 09/12] Add helpers to better structure CPU option handling

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > To avoid hardcoding even more CPU-flag related things for custom CPU > models, introduce a dynamic approach to resolving flags. > > resolve_cpu_flags takes a list of hashes (as documented in the > comment) and resolves them to a valid "-cpu" argum

Re: [pve-devel] [PATCH v4 qemu-server 10/12] Rework get_cpu_options and allow custom CPU models

2019-10-14 Thread Fabian Grünbichler
On October 7, 2019 2:47 pm, Stefan Reiter wrote: > If a cputype is custom (check via prefix), try to load options from the > custom CPU model config, and set values accordingly. > > While at it, extract currently hardcoded values into seperate sub and add > reasonings. > > Since the new flag reso

Re: [pve-devel] [PATCH kernel-meta] fix #2403: exclude initrd entries from /proc/cmdline

2019-10-15 Thread Fabian Grünbichler
On October 15, 2019 3:54 pm, Thomas Lamprecht wrote: > On 10/15/19 3:22 PM, Oguz Bektas wrote: >> if we fallback to /proc/cmdline, it can include the booted initrd. >> >> to avoid loader entries with (sometimes even multiple) initrd lines, >> we have to parse them out. >> >> Signed-off-by: Oguz B

Re: [pve-devel] applied: [PATCH v3 kernel-meta] fix #2403: exclude initrd entries from /proc/cmdline

2019-10-17 Thread Fabian Grünbichler
On October 17, 2019 7:45 am, Thomas Lamprecht wrote: > On 10/16/19 1:17 PM, Oguz Bektas wrote: >> if we fallback to /proc/cmdline, it can include the booted initrd. >> >> to avoid loader entries with initrd 'options' lines, we have to parse >> them out. >> >> Signed-off-by: Oguz Bektas >> --- >>

[pve-devel] [PATCH 02/23] add missing 'use PVE::Auth::Plugin'

2019-10-17 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/API2/AccessControl.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/PVE/API2/AccessControl.pm b/PVE/API2/AccessControl.pm index dfbdfc6..9d2da8d 100644 --- a/PVE/API2/AccessControl.pm +++ b/PVE/API2/AccessControl.pm @@ -19,6 +19,7 @@ use PVE::API2

[pve-devel] [RFC 1/23] API schema: add 'notoken' property

2019-10-17 Thread Fabian Grünbichler
to mark API methods which should not be available to clients authenticated using an API token Signed-off-by: Fabian Grünbichler --- Notes: if applied, any users of this need corresponding versioned depends. src/PVE/JSONSchema.pm | 5 + 1 file changed, 5 insertions(+) diff --git a

[pve-devel] [PATCH 04/23] user.cfg: sort ACL members

2019-10-17 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm index 3e52c5f..aff9137 100644 --- a/PVE/AccessControl.pm +++ b/PVE/AccessControl.pm @@ -1135,11 +1135,11 @@ sub

[pve-devel] [PATCH 03/23] user.cfg: sort entries alphabetically in each section

2019-10-17 Thread Fabian Grünbichler
it's not required for dependencies (since those are only ever between sections, and not within), but makes for easier diffing. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/PVE/AccessControl.pm

[pve-devel] [RFC 07/23] auth: pull username REs into variables

2019-10-17 Thread Fabian Grünbichler
for reusage in API token ID format/verification Signed-off-by: Fabian Grünbichler --- PVE/Auth/Plugin.pm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/PVE/Auth/Plugin.pm b/PVE/Auth/Plugin.pm index 5c11991..6d59b72 100755 --- a/PVE/Auth/Plugin.pm +++ b/PVE/Auth

[pve-devel] [RFC 09/23] API token: add REs, helpers, parsing + writing

2019-10-17 Thread Fabian Grünbichler
of the corresponding user and those of the API token itself. Signed-off-by: Fabian Grünbichler --- Notes: I am a bit unsure how to differentiate in a clean way between: A full userid/tokenid (username@realm OR username@real!token) B user (username@realm) C tokenid (username@realm!t

[pve-devel] [PATCH 16/23] proxy_request: drop duplicate, unused parameter

2019-10-17 Thread Fabian Grünbichler
which was mistakenly added back when this was still in pve-manager. Signed-off-by: Fabian Grünbichler --- PVE/APIServer/AnyEvent.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/APIServer/AnyEvent.pm b/PVE/APIServer/AnyEvent.pm index c1dade8..9aba27d 100644 --- a/PVE

[pve-devel] [RFC 10/23] API token: add API helpers

2019-10-17 Thread Fabian Grünbichler
the _exist/_enabled are modelled after the corresponding user methods. the 'tokenid' option goes into PVE::AccessControl, since we need it in multiple API modules. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 32 1 file changed, 32

[pve-devel] [RFC 08/23] refactor acl transformation code

2019-10-17 Thread Fabian Grünbichler
pull it into helper sub, since we need this one more time for token ACL members. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 61 +++- 1 file changed, 26 insertions(+), 35 deletions(-) diff --git a/PVE/AccessControl.pm b/PVE

[pve-devel] [RFC 12/23] API: add API token API endpoints

2019-10-17 Thread Fabian Grünbichler
and integration for user API endpoints. Signed-off-by: Fabian Grünbichler --- Notes: pveum integration will come in a future version, but pveum token add/modify/delete [OPTIONS] or pveum user token add/modify/delete [OPTIONS] seem like likely

[pve-devel] [RFC 06/23] rpcenv: drop unused roles()

2019-10-17 Thread Fabian Grünbichler
it doesn't really serve a purpose, and it's not called anywhere in the codebase. Signed-off-by: Fabian Grünbichler --- Notes: alternatively, we can give this the same semantics w.r.t. tokens as PVE::AccessControl::roles, but with pool roles mixed in via $compile_acl_path

[pve-devel] [RFC 11/23] DO NOT APPLY: API token stubs for token value handling

2019-10-17 Thread Fabian Grünbichler
d-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 47 1 file changed, 47 insertions(+) diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm index 432ccc0..b5dfe4b 100644 --- a/PVE/AccessControl.pm +++ b/PVE/AccessControl.pm @@ -397,6 +397,39 @

[pve-devel] [PATCH 17/23] allow ticket in auth header as fallback

2019-10-17 Thread Fabian Grünbichler
based on idea & RFC by Tim Marx, incorporating feedback by Thomas Lamprecht. this will be extended to support API tokens in the Authorization header as well, so make it generic. Signed-off-by: Fabian Grünbichler --- Notes: semi-independent, could also leave extract_auth_cookie as a

[pve-devel] [RFC/PATCH 0/23] API Tokens

2019-10-17 Thread Fabian Grünbichler
ve-http-server) - checking API endpoints for 'notoken'-ification I tried to order independent clean-ups etc. up front with-in each repo, but some of them require versioned breaks/depends so it might make sense to wait for the full series for those. pve-common: Fabian Grünbichler (1):

[pve-devel] [RFC 15/23] api: mark some paths notoken

2019-10-17 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: requires versioned dependency on libpve-common-perl PVE/API2/AccessControl.pm | 4 1 file changed, 4 insertions(+) diff --git a/PVE/API2/AccessControl.pm b/PVE/API2/AccessControl.pm index 9d2da8d..2a52105 100644 --- a/PVE/API2

[pve-devel] [RFC 13/23] API: include API tokens in ACL API endpoints

2019-10-17 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/API2/ACL.pm | 30 ++ 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/PVE/API2/ACL.pm b/PVE/API2/ACL.pm index 3e42ac0..c340267 100644 --- a/PVE/API2/ACL.pm +++ b/PVE/API2/ACL.pm @@ -46,7 +46,7 @@ __PACKAGE__

[pve-devel] [RFC 05/23] access-control: remove check_permissions/permission

2019-10-17 Thread Fabian Grünbichler
they have been handled by PVE::RPCEnvironment for quite some time already, and the versions there are the complete ones that should be actually used. Signed-off-by: Fabian Grünbichler --- Notes: this requires the corresponding patch for and a versioned breaks on pve-manager, where a

[pve-devel] [RFC 21/23] rest_handler: implement 'notoken' API endpoints

2019-10-17 Thread Fabian Grünbichler
that are not available with API tokens for security reasons, such as access control related endpoints. Signed-off-by: Fabian Grünbichler --- Notes: pairs with patch in pve-common that adds this to the schema-schema. any modules setting that flag need a corresponding versioned depends

[pve-devel] [PATCH 19/23] subscription: use rpcenv for permission check

2019-10-17 Thread Fabian Grünbichler
this is the last caller of PVE::AccessControl::check_permissions(), which is the last caller of PVE::AccessControl::permission(). both can thus be dropped altogether. Signed-off-by: Fabian Grünbichler --- Notes: this is semi-independent, but if applied we need to remember that the

[pve-devel] [RFC 14/23] API token: implement permission checks

2019-10-17 Thread Fabian Grünbichler
non-privsep tokens will always return the roles/permissions of their associated users. privsep tokens will return unfiltered roles, but filtered permissions. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 30 ++ PVE/RPCEnvironment.pm | 27

[pve-devel] [RFC 18/23] api-server: extract, set and handle API token header

2019-10-17 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: versioned breaks/depends with pve-manager and part of PMG? PVE/APIServer/AnyEvent.pm| 22 +++--- PVE/APIServer/Formatter.pm | 9 + PVE/APIServer/Formatter/Bootstrap.pm | 1 + 3 files changed, 25

[pve-devel] [RFC 20/23] auth_handler: handle API tokens

2019-10-17 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: versioned breaks/depends between pve-manager and libpve-http-server-perl! versioned depends on libpve-access-control PVE/HTTPServer.pm | 55 ++- 1 file changed, 30 insertions(+), 25 deletions

[pve-devel] [RFC 23/23] api/tasks: attribute token tasks to user

2019-10-17 Thread Fabian Grünbichler
and store token ID in separate, currently unused member. Signed-off-by: Fabian Grünbichler --- Notes: versioned depends on libpve-access-control alternatively, we could also change the fork_worker signature and encode this inside the task information on disk, but that would be

[pve-devel] [RFC 22/23] pveproxy: use new cookie extraction method

2019-10-17 Thread Fabian Grünbichler
we only care about the regular cookie case for the index. Signed-off-by: Fabian Grünbichler --- Notes: versioned breaks/depends on libpve-http-perl! PVE/Service/pveproxy.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/Service/pveproxy.pm b/PVE/Service

Re: [pve-devel] [RFC/PATCH 0/23] API Tokens

2019-10-17 Thread Fabian Grünbichler
and I just realized that I dropped the per-repo subject-prefix from all patches instead of just the cover-letter.. #1 is pve-common #2-15 are pve-access-control #16-18 are pve-http-server #19-#23 are pve-manager if you want a resend, just shout.. On October 17, 2019 3:13 pm, Fabian Grünbichler

Re: [pve-devel] [RFC 1/23] API schema: add 'notoken' property

2019-10-17 Thread Fabian Grünbichler
On October 17, 2019 4:46 pm, Thomas Lamprecht wrote: > On 10/17/19 3:13 PM, Fabian Grünbichler wrote: >> to mark API methods which should not be available to clients authenticated >> using an API token > > please wrap lines, for commit messages it's common to use <&l

Re: [pve-devel] [PATCH v3 guest-common 04/19] helpers: add method to represent config as a table

2019-10-17 Thread Fabian Grünbichler
On October 17, 2019 7:48 pm, Thomas Lamprecht wrote: > On 10/14/19 10:28 AM, Oguz Bektas wrote: >> in vm_pending API, this method is used to represent the configuration as >> a table with current, pending and delete columns. >> >> by adding it as a guesthelper, we can also use it for container pen

Re: [pve-devel] [PATCH qemu-server v3] Fix #2412: Missing VMs in pools

2019-10-17 Thread Fabian Grünbichler
es from the same node. Additionally, > unlinking must happen at the very end of the deletion process to avoid that > other nodes use the ID in the meanwhile. > > Co-developed-by: Fabian Grünbichler > Signed-off-by: Dominic Jäger > --- > v1->v2: Move unlink to the end o

Re: [pve-devel] [PATCH v4 0/7] add purge option for VM/CT destroy

2019-10-17 Thread Fabian Grünbichler
On October 17, 2019 6:40 pm, Thomas Lamprecht wrote: > On 10/15/19 1:00 PM, Christian Ebner wrote: >> When destroying a VM/CT, we intentionally did not remove all related configs >> such >> as backup or replication jobs. >> The intention of this flag is to allow the removal of such configs on >>

  1   2   3   4   5   6   7   8   9   10   >