[pve-devel] [PATCH storage 2/16] Remove unused string

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- PVE/Storage.pm | 1 - 1 file changed, 1 deletion(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index 0bd103e..5fefa06 100755 --- a/PVE/Storage.pm +++ b/PVE/Storage.pm @@ -573,7 +573,6 @@ sub storage_migrate { my $target_volid =

[pve-devel] [PATCH qemu-server 10/16] rename 'volid' to 'drivestr' where it's not only a volume ID

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 10 +- PVE/QemuServer.pm | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 49848e8..d025b09 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -491,7

[pve-devel] [RFC storage 6/16] pvesm import: allow specifying storage+vmid instead of full volumeid

2020-01-29 Thread Fabian Ebner
Extends the API so that 'volume' can also only be a storage identifier. In that case the VMID needs to be specified as well. In 'import_volume' a new name for the allocation is determined. This is useful for migration where the storage on the target is a different type, since the volume ID might

[pve-devel] [RFC storage 7/16] volume_import_formats: if no volume name is specified, return all formats the storage supports

2020-01-29 Thread Fabian Ebner
Like this it is possible to determine if the transfer of a volume is possible wihout already having the name of the volume on the target storage. When doing the import, 'volume_import' can then choose a new name automatically. Signed-off-by: Fabian Ebner --- For example, migration with a disk

[pve-devel] [RFC qemu-server 13/16] Take note of changes to volume ids when migrating and update config

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- Might make sense to combine this patch and patch 16. PVE/QemuMigrate.pm | 22 +++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 81b52d1..702fda0 100644 --- a/PVE/QemuMigrate.pm +++

[pve-devel] [RFC storage 8/16] storage_migrate: use only storeid when no volume name is specified

2020-01-29 Thread Fabian Ebner
so the migration code doesn't need to know how the volid on the targetstorage would look like (e.g. VM has disk mydir:123/vm-123-disk-0.raw and targetstorage is an LVM). Previously it would fail, because the current volname was passed to 'pvesm import' which is an invalid name for an LVM storage.

[pve-devel] [RFC qemu-server 15/16] sync_disks: log output of storage_migrate

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- Not sure about this one. On the one hand it adds even more to the migration logs, which are already rather long. On the other hand it might contain useful information. PVE/QemuMigrate.pm | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git

[pve-devel] [RFC storage 5/16] storage_migrate: return volume ID of migrated volume

2020-01-29 Thread Fabian Ebner
Since 'pvesm import' uses a new volume ID if the requested one is already present, callers should have a way to get the new volume ID. Signed-off-by: Fabian Ebner --- PVE/CLI/pvesm.pm | 2 +- PVE/Storage.pm | 41 + 2 files changed, 34 insertions(+), 9

[pve-devel] [PATCH qemu-server 11/16] Extract volume ID before calling 'parse_volume_id'

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index d025b09..81b52d1 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -686,8 +686,10 @@ sub phase2 { foreach

[pve-devel] [PATCH storage 4/16] storage_migrate: also log with an insecure connection if there is a log function

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- PVE/Storage.pm | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index 5fefa06..2b292f6 100755 --- a/PVE/Storage.pm +++ b/PVE/Storage.pm @@ -626,11 +626,21 @@ sub storage_migrate {

[pve-devel] [RFC/PATCH] make storage migration more flexible

2020-01-29 Thread Fabian Ebner
This series aims to allow offline migration with '--targetstorage' and improve handling unsued/orphaned disks. An abstract method for recursively updating volume IDs throughout a config file is provided. Like this it should also work when there are snapshots. The API for 'pvesm import' is

[pve-devel] [RFC guest-common 1/16] Implement update_volume_ids and add required helpers: foreach_volume and print_volume

2020-01-29 Thread Fabian Ebner
This function is intened to be used after doing a migration where some of the volume IDs changed. Signed-off-by: Fabian Ebner --- PVE/AbstractConfig.pm | 61 +++ 1 file changed, 61 insertions(+) diff --git a/PVE/AbstractConfig.pm b/PVE/AbstractConfig.pm

[pve-devel] [RFC qemu-server 14/16] Allow specifying targetstorage for offline migration

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- PVE/API2/Qemu.pm | 3 --- 1 file changed, 3 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 89e2477..f21fb69 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -3379,9 +3379,6 @@ __PACKAGE__->register_method({ $param->{online}

[pve-devel] [RFC storage 9/16] storage_migrate: Make error message more verbose

2020-01-29 Thread Fabian Ebner
The volid contains the format and that's relevant information for why migration is not possible. For example, a raw volume can be migrated between an LVM storage and a filesystem based storage, but a qcow2 volume cannot. Signed-off-by: Fabian Ebner --- PVE/Storage.pm | 5 - 1 file changed,

[pve-devel] [RFC qemu-server 12/16] Implement abstract foreach_volume and print_volume

2020-01-29 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- PVE/QemuConfig.pm | 12 1 file changed, 12 insertions(+) diff --git a/PVE/QemuConfig.pm b/PVE/QemuConfig.pm index 1ba728a..a983e52 100644 --- a/PVE/QemuConfig.pm +++ b/PVE/QemuConfig.pm @@ -130,6 +130,18 @@ sub get_replicatable_volumes { return

[pve-devel] [RFC qemu-server 16/16] Update volume IDs in one go

2020-01-29 Thread Fabian Ebner
Use 'update_volume_ids' for the live-migrated disks as well. Signed-off-by: Fabian Ebner --- PVE/QemuMigrate.pm | 23 +-- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index af1cf01..6a0f034 100644 ---

[pve-devel] [RFC storage 3/16] volume_import: Use a new name when the the name to import with already exists

2020-01-29 Thread Fabian Ebner
The ID of the new volume is returned and pvesm import prints it. This is useful for migration, since the storage on the target might already contain unused/orphaned disks. Signed-off-by: Fabian Ebner --- Breaks the current migration in QEMU/LXC if there is a collision, since the code doesn't

[pve-devel] applied: [PATCH storage 2/3] cephfs: mount fuse through systemd with correct order dependencies

2020-01-29 Thread Thomas Lamprecht
This fixes a potential races where fuse get's unmouted to late in the shutdown process, i.e., at a time where network was down and it could not talk to any MDS or monitor anymore. We could fix it the same way we did once with the kernel based mount, i.e., adding _netdev, but doing so would

[pve-devel] applied: [PATCH storage 1/3] fix random hangs on reboot with active CephFS mount ordering cycle

2020-01-29 Thread Thomas Lamprecht
commit 54e0b0034bd6654c566cb4ae7d4a5953c48cd1ca introduced the "_netdev" option, for PVE 5.3. The systemd generator then correctly resolved that in the following resulting order-dependencies: > Wants=network-online.target > Before=umount.target remote-fs.target > After=remote-fs-pre.target

[pve-devel] applied: [PATCH storage 3/3] cephfs mount: reload systemd if existing unit gets regenerated

2020-01-29 Thread Thomas Lamprecht
One the first write bringing the unit file in existence we can just start it, after that we need to tell systemd that we want to actively reload it. While this is slightly shaky due to the fact that we do not check all paths where such a unit could reside, it is something we can do because

[pve-devel] applied: [PATCH v3 access-control 09/20] API: add group and token info to user index

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > otherwise we need 1+N API calls to retrieve the full user+token picture > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > new in v2 > > PVE/API2/User.pm | 21 +++-- > 1 file changed, 19 insertions(+), 2 deletions(-) >

[pve-devel] applied: [PATCH v3 manager 1/9] auth_handler: handle API tokens

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > by verifying them via pve-access-control, as alternative to regular > tickets. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > versioned breaks/depends between pve-manager and libpve-http-server-perl! > versioned depends on

[pve-devel] applied: [PATCH v3 manager 2/9] rest_handler: implement 'allowtoken' property

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > to filter out API paths that are not available with API tokens for > security reasons, such as access control related endpoints. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > pairs with patch in pve-common that adds this to the

[pve-devel] applied: [PATCH v3 manager 3/9] pveproxy: use new cookie extraction method

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > we only care about the regular cookie case for the index. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > versioned breaks/depends on libpve-http-perl! > > PVE/Service/pveproxy.pm | 2 +- > 1 file changed, 1 insertion(+), 1

Re: [pve-devel] [PATCH v3 access-control 08/20] API: add API token API endpoints

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > ... > @@ -53,6 +98,7 @@ my $extract_user_data = sub { > return $res if !$full; > > $res->{groups} = $data->{groups} ? [ keys %{$data->{groups}} ] : []; > +$res->{tokens} = $data->{tokens}; > > return $res; > }; > @@ -228,7

[pve-devel] applied: [PATCH v3 manager 4/9] api/tasks: attribute token tasks to user

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > and store token ID in separate, currently unused member. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > versioned depends on libpve-access-control > > alternatively, we could also change the fork_worker signature and encode

[pve-devel] apllied: [PATCH v3 manager 6/9] www: add permissions button to userview

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > www/manager6/Makefile | 1 + > www/manager6/dc/PermissionView.js | 167 ++ > www/manager6/dc/UserView.js | 14 ++- > 3 files changed, 181 insertions(+), 1

[pve-devel] applied: [PATCH v3 access-control 13/20] roles()/permissions(): also return propagate flag

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > this information is already available, but not exposed. we need it for > dumping an effective permission tree of a given user/token. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > new in v2 > > PVE/AccessControl.pm | 28

[pve-devel] applied: [PATCH v3 access-control 10/20] API: include API tokens in ACL API endpoints

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > PVE/API2/ACL.pm | 30 ++ > 1 file changed, 22 insertions(+), 8 deletions(-) > applied, thanks! ___ pve-devel mailing list

[pve-devel] applied: [PATCH v3 access-control 14/20] API: add 'permissions' API endpoint

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > and related helper, to dump permissions + propagate info for > - a specific, given path > - generic top-level + user.cfg-referenced paths, including pools > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > new in v2 > >

[pve-devel] applied: [PATCH v3 access-control 11/20] API token: implement permission checks

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > non-privsep tokens will always return the roles/permissions of their > associated users. privsep tokens will return unfiltered roles, but > filtered permissions. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > v1->v2: > - fix for

[pve-devel] applied: [PATCH v3 access-control 12/20] api: disallow some paths for API tokens

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > > Notes: > requires versioned dependency on libpve-common-perl > > v2->v3: > - rename notoken to allowtoken, negate semantics accordingly > > requires versioned dependency on

[pve-devel] applied: [PATCH v3 access-control 20/20] user.cfg: skip inexisting roles when parsing ACLs

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > we do the same for missing users, groups and tokens, and just like > groups, roles with an empty privilege set are explicitly allowed so > pre-generating placeholders is possible. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > new in

[pve-devel] applied: [PATCH v3 access-control 16/20] tests: unify config file naming

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > test/perm-test1.pl | 2 +- > test/{user.cfg.ex1 => test1.cfg} | 0 > 2 files changed, 1 insertion(+), 1 deletion(-) > rename test/{user.cfg.ex1 => test1.cfg} (100%) > > diff --git

[pve-devel] applied: [PATCH v3 access-control 15/20] test: add token-related tests

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > > Notes: > v2->v3: retitled commit > new in v2 > > test/Makefile | 1 + > test/parser_writer.pl | 173 +- > test/perm-test8.pl| 68

[pve-devel] applied: [PATCH v3 access-control 18/20] pveum: add 'pveum user token add/update/remove/list'

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > mapping 1-to-1 to the respective API paths > > Signed-off-by: Fabian Grünbichler > --- > PVE/CLI/pveum.pm | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/PVE/CLI/pveum.pm b/PVE/CLI/pveum.pm > index 537215f..c642f6d 100755 > ---

[pve-devel] applied: [PATCH v3 access-control 19/20] pveum: add permissions sub-commands

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > for user and token commands, and some pretty-printing for regular text > output, since the returned nested hash/dict is not very readable. > > Signed-off-by: Fabian Grünbichler > --- > PVE/CLI/pveum.pm | 66

[pve-devel] applied: [PATCH v3 pmg-api 1/1] ensure compatibility with libpve-http-server-perl

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > > Notes: > new in v3 > > requires depends on libpve-http-server which changed these interfaces > > src/PMG/HTTPServer.pm | 4 +++- > src/PMG/Service/pmgproxy.pm | 2 +- > 2 files

[pve-devel] applied: [PATCH v3 http-server 2/2] api-server: extract, set and handle API token header

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > > Notes: > versioned breaks/depends with pve-manager and pmg-api > > PVE/APIServer/AnyEvent.pm| 25 ++--- > PVE/APIServer/Formatter.pm | 9 + >

Re: [pve-devel] [PATCH v3 http-server 1/2] allow ticket in auth header as fallback

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > From: Tim Marx > > based on idea & RFC by Tim Marx, incorporating feedback by Thomas > Lamprecht. this will be extended to support API tokens in the > Authorization header as well, so make it generic. > > Signed-off-by: Fabian Grünbichler > --- >

Re: [pve-devel] [PATCH proxmox-widget-toolkit] add vlan interface support

2020-01-29 Thread Alexandre DERUMIER
Any comment about this patch ? - Mail original - De: "aderumier" À: "pve-devel" Cc: "aderumier" Envoyé: Mardi 28 Janvier 2020 11:24:45 Objet: [PATCH proxmox-widget-toolkit] add vlan interface support vlan-raw-device && vlan-id field are only enabled if interface name is different

[pve-devel] applied: [PATCH v3 access-control 08/20] API: add API token API endpoints

2020-01-29 Thread Thomas Lamprecht
On 1/21/20 1:54 PM, Fabian Grünbichler wrote: > and integration for user API endpoints. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > v1->v2: > - adapted API schema somewhat > - actually allow privileged users to view/modify/delete tokens of other > users > - remove

[pve-devel] [RFC manager 4/6] gui/cluster: add CorosyncLinkEdit component to support up to 8 links

2020-01-29 Thread Stefan Reiter
CorosyncLinkEdit is a Panel that contains between one and 8 CorosyncLinkSelectors. These can be added or removed with according buttons. Values submitted to the API are calculated by each ProxmoxNetworkSelector itself. This works because ExtJS searches recursively through all child components for

[pve-devel] [PATCH+RFC 0/6] Support all 8 corosync3 links in GUI

2020-01-29 Thread Stefan Reiter
Patches 1-3 can be applied immediately, no dependency on anything else. The RFC patches only make sense together with my previously submitted cluster link series[0]. I'm anything but a GUI designer, so I'm happy for any feedback on the new CorosyncLinkEdit component - hence sending it now

[pve-devel] [PATCH widget-toolkit 1/6] ComboGrid: fix on-load validation for blank values

2020-01-29 Thread Stefan Reiter
Commit f32aa3df74 fixed marking multi-select fields with where the store did not contain a valid value after loading. However, it introduced a bug for single-select fields where the value (before the store-load) was explicitly set to be empty (when that should be invalid because of allowBlank ===

[pve-devel] [PATCH manager 3/6] gui/cluster: show cluster name from joinInfo in join dialog

2020-01-29 Thread Stefan Reiter
To help a user identify if they put in the joinInfo for the correct cluster. Signed-off-by: Stefan Reiter --- www/manager6/dc/ClusterEdit.js | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/www/manager6/dc/ClusterEdit.js b/www/manager6/dc/ClusterEdit.js

[pve-devel] [RFC manager 6/6] gui/cluster: add structured peerLinks to join info

2020-01-29 Thread Stefan Reiter
Instead of the old 'ring_addr' property (which is kept for compatibility), we also encode the link numbers into the new peerLinks structure. This allows us to display which IP is assigned to which link on the cluster in the join dialog, helping a user identify which link should receive which

[pve-devel] [RFC manager 5/6] gui/cluster: show all links (up to 8) in cluster overview

2020-01-29 Thread Stefan Reiter
Show only links found on the cluster. 'enableColumnHide' is set to false, since it causes confusing behaviour for the user, considering we'd overwrite part of their choices every 5 seconds. Signed-off-by: Stefan Reiter --- www/manager6/Utils.js | 10 +++ www/manager6/dc/Cluster.js | 128

[pve-devel] [PATCH manager 2/6] gui/cluster: validate cluster name length on create

2020-01-29 Thread Stefan Reiter
API limits this to 15, validate in GUI as well for instant user feedback. Signed-off-by: Stefan Reiter --- www/manager6/dc/ClusterEdit.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/dc/ClusterEdit.js b/www/manager6/dc/ClusterEdit.js index a8c021d2..0af858a5 100644 ---