[pve-devel] applied-series: [PATCH cluster/manager/storage/docs 0/9] fix #4886: improve SSH handling

2024-04-19 Thread Thomas Lamprecht
Am 11/01/2024 um 11:51 schrieb Fabian Grünbichler: > this series replaces the old mechanism that used a cluster-wide merged known > hosts file with distributing of each node's host key via pmxcfs, and pinning > the distributed key explicitly for internal SSH connections. > > the main changes in pv

[pve-devel] [PATCH proxmox-firewall v3 16/39] config: firewall: add conntrack helper types

2024-04-19 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/resources/ct_helper.json | 52 + proxmox-ve-config/src/firewall/ct_helper.rs | 115 proxmox-ve-config/src/firewall/mod.rs

[pve-devel] [PATCH proxmox-firewall v3 28/39] firewall: add config loader

2024-04-19 Thread Stefan Hanreich
We load the firewall configuration from the default paths, as well as only the guest configurations that are local to the node itself. In the future we could change this to use pmxcfs directly instead. We also load information from nftables directly about dynamically created chains (mostly chains

[pve-devel] [PATCH proxmox-firewall v3 25/39] nftables: add nft client

2024-04-19 Thread Stefan Hanreich
Add a thin wrapper around nft, which can be used to run commands defined by the rust types. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-nftables/src/client.rs | 85 ++ proxmox-n

[pve-devel] [PATCH proxmox-firewall v3 22/39] nftables: statement: add conversion traits for config types

2024-04-19 Thread Stefan Hanreich
Some types from the firewall configuration map directly onto nftables statements. For those we implement conversion traits so we can conveniently convert between the configuration types and the respective nftables types. As with the expressions, those are guarded behind a feature so the nftables c

[pve-devel] [PATCH proxmox-firewall v3 31/39] firewall: add ruleset generation logic

2024-04-19 Thread Stefan Hanreich
We create the rules from the firewall config by utilizing the ToNftRules and ToNftObjects traits to convert the firewall config structs to nftables objects/chains/rules. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- pro

[pve-devel] [PATCH proxmox-firewall v3 19/39] nftables: expression: add types

2024-04-19 Thread Stefan Hanreich
Adds an enum containing most of the expressions defined in the nftables-json schema [1]. [1] https://manpages.debian.org/bookworm/libnftables1/libnftables-json.5.en.html#EXPRESSIONS Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanr

[pve-devel] [PATCH pve-firewall v3 37/39] add configuration option for new nftables firewall

2024-04-19 Thread Stefan Hanreich
Introduces new nftables configuration option that en/disables the new nftables firewall. pve-firewall reads this option and only generates iptables rules when nftables is set to `0`. Conversely proxmox-firewall only generates nftables rules when the option is set to `1`. Signed-off-by: Stefan Han

[pve-devel] [PATCH proxmox-firewall v3 34/39] firewall: add integration test

2024-04-19 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich --- .gitignore|1 + debian/control|1 + proxmox-firewall/Cargo.toml |4 + proxmox-firewall/tests/input/100.conf | 10 + proxmox-firewall/tests/input/100.fw

Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > Instead of passing the template strings for subject and body when > constructing a notification, we pass only the name of a template. > When rendering the template, the name of the template is used to find > corresponding template files. For PVE, they ar

Re: [pve-devel] [PATCH proxmox 05/19] notify: make the `mail-forwarder` feature depend on proxmox-sys

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > It uses proxmox_sys::nodename - the dep is needed, otherwise the code > does not compile in some feature flag permutations. > > Signed-off-by: Lukas Wagner > Tested-by: Folke Gleumes > --- > proxmox-notify/Cargo.toml | 6 +++--- > 1 file changed, 3 i

Re: [pve-devel] [PATCH proxmox 07/19] notify: api: add get_targets

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > +/// Get a list of all notification targets. > +pub fn get_targets(config: &Config) -> Result, HttpError> { > +let mut targets = Vec::new(); > + > +#[cfg(feature = "gotify")] > +for endpoint in gotify::get_endpoints(config)? { > +targ

Re: [pve-devel] [PATCH proxmox 07/19] notify: api: add get_targets

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > +#[api] > +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq, PartialOrd)] > +#[serde(rename_all = "kebab-case")] > +/// Target information > +pub struct Target { > +/// Name of the endpoint > +name: String, > +/// Origin of the en

Re: [pve-devel] [PATCH proxmox 09/19] notify: derive Deserialize/Serialize for Notification struct

2024-04-19 Thread Fiona Ebner
Nit: I always like a quick sentence for who needs it for such changes. Am 09.04.24 um 15:25 schrieb Lukas Wagner: > Signed-off-by: Lukas Wagner > --- > proxmox-notify/src/lib.rs | 10 +++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/proxmox-notify/src/lib.rs b/proxm

Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system

2024-04-19 Thread Lukas Wagner
On 2024-04-19 10:14, Fiona Ebner wrote: > Am 09.04.24 um 15:25 schrieb Lukas Wagner: >> Instead of passing the template strings for subject and body when >> constructing a notification, we pass only the name of a template. >> When rendering the template, the name of the template is used to find

Re: [pve-devel] [PATCH manager 7/7] report: add recent boot timestamps which may show fencing/crash events

2024-04-19 Thread Mira Limbeck
On 4/18/24 17:45, Alexander Zeidler wrote: > On Thu, 2024-04-18 at 12:43 +0200, Mira Limbeck wrote: >> On 4/18/24 11:16, Alexander Zeidler wrote: >>> Successful boots which crashed somehow and sometime afterwards, will >>> show the same "until" value ("still running" or timestamp) as the next >>> f

Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system

2024-04-19 Thread Fiona Ebner
Am 19.04.24 um 10:45 schrieb Lukas Wagner: >> Who adds the template files? I don't see a patch for proxmox-ve in this >> series. Does this series require some versioned breaks to some package? > > The pve-manager and pve-ha-manager (for fencing notifications) patches add > the templates. > I can'

[pve-devel] [RFC PATCH docs-common 02/13] installation-media: adapt for common usage

2024-04-19 Thread Christoph Heiss
Small adaptions were necessary; mostly a trivial s/{pve}/{product}/g and replacing the ISO URL with the {iso-url} variable. Signed-off-by: Christoph Heiss --- installation-media.adoc | 21 - 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/installation-media.ado

[pve-devel] [RFC PATCH docs-common 01/13] installation-media: move page from pve-docs here

2024-04-19 Thread Christoph Heiss
Small adaptions were necessary; mostly a s/{pve}/{product}/g and replacing the ISO URL with the {iso-url} variable. Signed-off-by: Christoph Heiss --- installation-media.adoc | 132 1 file changed, 132 insertions(+) create mode 100644 installation-media.

[pve-devel] [RFC PATCH docs{, -common} 0/13] introduce common documentation base

2024-04-19 Thread Christoph Heiss
tl;dr: Introduce a separate repository for shared documentation between all three products. This proposes the introduction of a - aptly named - proxmox-docs-common repo, which can be used for all documentation not specific to a single product. Marked RFC to gather feedback from other people. Afte

[pve-devel] [RFC PATCH docs-common 03/13] partials: add installation flow from pve-docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- partials/installation-flow.adoc | 170 1 file changed, 170 insertions(+) create mode 100644 partials/installation-flow.adoc diff --git a/partials/installation-flow.adoc b/partials/installation-flow.adoc new file mode 100644 ind

[pve-devel] [RFC PATCH docs 08/13] scan-adoc-refs: enable building pages from proxmox-docs-common/ subdir

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- Makefile | 6 -- asciidoc/asciidoc-pve.conf | 2 ++ pve-doc-generator.mk.in| 6 ++ scan-adoc-refs | 25 - 4 files changed, 28 insertions(+), 11 deletions(-) diff --git a/Makefile b/Makefile in

[pve-devel] [RFC PATCH docs-common 05/13] partials: add advanced installation hints from pve-docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- partials/advanced-installation.adoc | 125 1 file changed, 125 insertions(+) create mode 100644 partials/advanced-installation.adoc diff --git a/partials/advanced-installation.adoc b/partials/advanced-installation.adoc new file mo

[pve-devel] [RFC PATCH docs 09/13] images: strip `pve-` prefix from screenshots used in common docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- .../{pve-grub-menu.png => grub-menu.png} | Bin .../{pve-grub-menu.ppm => grub-menu.ppm} | Bin ...ve-install-summary.png => install-summary.png} | Bin ...ve-install-summary.ppm => install-summary.ppm} | Bin .../{pve-installation.png => insta

[pve-devel] [RFC PATCH docs 11/13] installation-media: move to common docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- pve-installation-media.adoc | 132 pve-installation.adoc | 2 +- 2 files changed, 1 insertion(+), 133 deletions(-) delete mode 100644 pve-installation-media.adoc diff --git a/pve-installation-media.adoc b/pve-instal

[pve-devel] [RFC PATCH docs-common 04/13] partials: installation-flow: adapt from pve-docs

2024-04-19 Thread Christoph Heiss
A trivial s/{pve}/{product}/g again and a additional s/screenshot\/pve-/screenshot\//g to fix the screenshot paths. Signed-off-by: Christoph Heiss --- partials/installation-flow.adoc | 47 + 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/partials/

[pve-devel] [RFC PATCH docs 10/13] asciidoc: conf: add iso-url variable

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- asciidoc/asciidoc-pve.conf | 1 + 1 file changed, 1 insertion(+) diff --git a/asciidoc/asciidoc-pve.conf b/asciidoc/asciidoc-pve.conf index 47139b8..faa190a 100644 --- a/asciidoc/asciidoc-pve.conf +++ b/asciidoc/asciidoc-pve.conf @@ -6,6 +6,7 @@ pve=Proxmox VE

[pve-devel] [RFC PATCH docs-common 06/13] partials: advanced-installation: adapt from pve-docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- partials/advanced-installation.adoc | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/partials/advanced-installation.adoc b/partials/advanced-installation.adoc index ed2709f..271533b 100644 --- a/partials/advanced-installati

[pve-devel] [RFC PATCH docs 07/13] gitmodules: add proxmox-docs-common

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- .gitmodules | 3 +++ proxmox-docs-common | 1 + 2 files changed, 4 insertions(+) create mode 100644 .gitmodules create mode 16 proxmox-docs-common diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 000..eff8adf --- /dev/null +++

[pve-devel] [RFC PATCH docs 13/13] installation: use new 'advanced-installation' partial from common docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- pve-installation.adoc | 126 +- 1 file changed, 1 insertion(+), 125 deletions(-) diff --git a/pve-installation.adoc b/pve-installation.adoc index 2f4642e..0a0dc76 100644 --- a/pve-installation.adoc +++ b/pve-installation.

[pve-devel] [RFC PATCH docs 12/13] installation: use new 'installation-flow' partial from common docs

2024-04-19 Thread Christoph Heiss
Signed-off-by: Christoph Heiss --- pve-installation.adoc | 172 +- 1 file changed, 1 insertion(+), 171 deletions(-) diff --git a/pve-installation.adoc b/pve-installation.adoc index f492ad0..2f4642e 100644 --- a/pve-installation.adoc +++ b/pve-installation.

Re: [pve-devel] [PATCH cluster 15/19] notify: use named template instead of passing template strings

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > The notification system will now load template files from a defined > location. The template to use is now passed to proxmox_notify, instead > of separate template strings for subject/body. > > Signed-off-by: Lukas Wagner > Tested-by: Folke Gleumes >

Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system

2024-04-19 Thread Lukas Wagner
On 2024-04-19 10:57, Fiona Ebner wrote: > Am 19.04.24 um 10:45 schrieb Lukas Wagner: >>> Who adds the template files? I don't see a patch for proxmox-ve in this >>> series. Does this series require some versioned breaks to some package? >> >> The pve-manager and pve-ha-manager (for fencing noti

[pve-devel] [PATCH pve-container v4 2/5] firewall: add handling for new nft firewall

2024-04-19 Thread Stefan Hanreich
When the nftables firewall is enabled, we do not need to create firewall bridges. Signed-off-by: Stefan Hanreich --- src/PVE/LXC.pm | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm index 400cf4f..44f5ccf 100644 --- a/src/PVE/LXC.pm +++ b/

[pve-devel] [PATCH qemu-server v4 1/5] firewall: add handling for new nft firewall

2024-04-19 Thread Stefan Hanreich
When the nftables firewall is enabled, we do not need to create firewall bridges. Signed-off-by: Stefan Hanreich --- vm-network-scripts/pve-bridge | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/vm-network-scripts/pve-bridge b/vm-network-scripts/pve-bridge index 85997a

[pve-devel] [PATCH container/docs/firewall/manager/qemu-server v4 0/5] proxmox firewall nftables

2024-04-19 Thread Stefan Hanreich
This patch series contains the remaining patches that are necessary for proxmox-firewall to work. It adds documentation as well as changes how firewall-bridges are created when proxmox-firewall is activated. It also patches pve-firewall to not generate rules when proxmox-firewall is active. Depend

[pve-devel] [PATCH pve-firewall v4 3/5] add configuration option for new nftables firewall

2024-04-19 Thread Stefan Hanreich
Introduces new nftables configuration option that en/disables the new nftables firewall. pve-firewall reads this option and only generates iptables rules when nftables is set to `0` or if the proxmox-firewall package is not installed at all. Conversely, proxmox-firewall only generates rules when t

[pve-devel] [PATCH pve-manager v4 4/5] firewall: expose configuration option for new nftables firewall

2024-04-19 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich --- www/manager6/grid/FirewallOptions.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/grid/FirewallOptions.js b/www/manager6/grid/FirewallOptions.js index 0ac9979c4..6aacb47be 100644 --- a/www/manager6/grid/FirewallOptions.js +++ b/www/manager6/

[pve-devel] [PATCH pve-docs v4 5/5] firewall: add documentation for proxmox-firewall

2024-04-19 Thread Stefan Hanreich
Add a section that explains how to use the new nftables-based proxmox-firewall. Signed-off-by: Stefan Hanreich --- pve-firewall.adoc | 181 ++ 1 file changed, 181 insertions(+) diff --git a/pve-firewall.adoc b/pve-firewall.adoc index a5e40f9..9fb4e46

Re: [pve-devel] [PATCH manager 17/19] gitignore: ignore any test artifacts

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > Signed-off-by: Lukas Wagner > --- > .gitignore | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/.gitignore b/.gitignore > index e8d1eb27..48975d55 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -9,3 +9,5 @@ dest/ > /www/mobile/pvemanager

[pve-devel] [PATCH storage/qemu-server/manager v2] implement ova/ovf import for file based storages

2024-04-19 Thread Dominik Csapak
This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware from vmware exported files). I opted to move the OVF.pm to pve-storage, since there is no

[pve-devel] [PATCH storage v2 02/10] plugin: dir: implement import content type

2024-04-19 Thread Dominik Csapak
in DirPlugin and not Plugin (because of cyclic dependency of Plugin -> OVF -> Storage -> Plugin otherwise) only ovf is currently supported (though ova will be shown in import listing), expects the files to not be in a subdir, and adjacent to the ovf file. Signed-off-by: Dominik Csapak --- change

[pve-devel] [PATCH manager v2 1/6] ui: fix special 'import' icon for non-esxi storages

2024-04-19 Thread Dominik Csapak
we only want to show that icon in the tree when the storage is solely used for importing, not when it's just one of several content types. Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- www/manager6/Utils.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/man

[pve-devel] [PATCH storage v2 03/10] plugin: dir: handle ova files for import

2024-04-19 Thread Dominik Csapak
since we want to handle ova files (which are only ovf+images bundled in a tar file) for import, add code that handles that. we introduce a valid volname for files contained in ovas like this: storage:import/archive.ova/disk-1.vmdk by basically treating the last part of the path as the name for

[pve-devel] [PATCH manager v2 2/6] ui: guest import: add ova-needs-extracting warning text

2024-04-19 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- changes from v1: * adapted text for new 'extract-stroage' www/manager6/window/GuestImport.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/window/GuestImport.js b/www/manager6/window/GuestImport.js index 944d275b..2c8bc189 100644 --- a/www/man

[pve-devel] [PATCH storage v2 09/10] plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs

2024-04-19 Thread Dominik Csapak
and reuse the DirPlugin implementation Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- changes from v1: * added glusterfs too * fixed typo in commit subject src/PVE/Storage/BTRFSPlugin.pm | 5 + src/PVE/Storage/CIFSPlugin.pm | 6 +- src/PVE/Storage/CephFSPlugin.pm

[pve-devel] [PATCH qemu-server v2 1/4] api: delete unused OVF.pm

2024-04-19 Thread Dominik Csapak
the api part was never in use by anything Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- no changes in v2 PVE/API2/Qemu/Makefile | 2 +- PVE/API2/Qemu/OVF.pm | 53 -- 2 files changed, 1 insertion(+), 54 deletions(-) delete mode 100644 PVE

[pve-devel] [PATCH manager v2 6/6] ui: guest import: change icon/text for non-esxi import storage

2024-04-19 Thread Dominik Csapak
since 'virtual guests' only make sense for a hypervisor, not e.g. a directory for OVAs also change the icon from 'desktop' to 'cloud-download' in the non-esxi case Signed-off-by: Dominik Csapak --- new in v2 www/manager6/storage/Browser.js | 6 -- 1 file changed, 4 insertions(+), 2 deletion

[pve-devel] [PATCH qemu-server v2 3/4] api: create: implement extracting disks when needed for import-from

2024-04-19 Thread Dominik Csapak
when 'import-from' contains a disk image that needs extraction (currently only from an 'ova' archive), do that in 'create_disks' and overwrite the '$source' volid. Collect the names into a 'delete_sources' list, that we use later to clean it up again (either when we're finished with importing or i

[pve-devel] [PATCH storage v2 01/10] copy OVF.pm from qemu-server

2024-04-19 Thread Dominik Csapak
copies the OVF.pm and relevant ovf tests from qemu-server. We need it here, and it uses PVE::Storage already, and since there is no intermediary package/repository we could put it, it seems fitting in here. Put it in a new GuestImport module Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner

[pve-devel] [PATCH qemu-server v2 2/4] use OVF from Storage

2024-04-19 Thread Dominik Csapak
and delete it here (incl tests; they live in pve-storage now). Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- changes from v1: * use new module location (that makes the list now ordered as well :P ) PVE/CLI/qm.pm | 4 +- PVE/QemuServer/Makefile

[pve-devel] [PATCH storage v2 10/10] add 'import' content type to 'check_volume_access'

2024-04-19 Thread Dominik Csapak
in the same branch as 'vztmpl' and 'iso' Signed-off-by: Dominik Csapak --- new in v2 src/PVE/Storage.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm index c628ebd..7e70df2 100755 --- a/src/PVE/Storage.pm +++ b/src/PVE/Storage.pm @@ -

[pve-devel] [PATCH storage v2 06/10] ovf: implement rudimentary boot order

2024-04-19 Thread Dominik Csapak
simply add all parsed disks to the boot order in the order we encounter them (similar to the esxi plugin). Signed-off-by: Dominik Csapak --- changes from v1: * renamed variable to boot_order * fixed bracket issue (was a rebase problem) * only add bootorder if list is not empty src/PVE/GuestImpo

[pve-devel] [PATCH storage v2 05/10] ovf: implement parsing out firmware type

2024-04-19 Thread Dominik Csapak
it seems there is no part of the ovf standard that handles which type of bios there is (at least i could not find it). Every ovf/ova i tested either has no info about it, or has it in a vmware specific property which we parse here. Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- chang

[pve-devel] [PATCH storage v2 07/10] ovf: implement parsing nics

2024-04-19 Thread Dominik Csapak
by iterating over the relevant parts and trying to parse out the 'ResourceSubType'. The content of that is not standardized, but I only ever found examples that are compatible with vmware, meaning it's either 'e1000', 'e1000e' or 'vmxnet3' (in various capitalizations; thus the `lc()`) As a fallbac

[pve-devel] [PATCH manager v2 4/6] ui: enable upload/download/remove buttons for 'import' type storages

2024-04-19 Thread Dominik Csapak
but only for non esxi ones, since that does not allow uploading/downloading there Signed-off-by: Dominik Csapak --- changes from v1: * show remove button for non-esxi * order extensions alphabetically * add missing '.' for ova extension www/manager6/storage/Browser.js| 9 +++-- www/

[pve-devel] [PATCH manager v2 3/6] ui: enable import content type for relevant storages

2024-04-19 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- changes from v1: * added import to glusterfs www/manager6/Utils.js| 1 + www/manager6/form/ContentTypeSelector.js | 2 +- www/manager6/storage/CephFSEdit.js | 2 +- www/manager6/storage/GlusterFsEdit.js| 2 +- 4 files changed, 4 inse

[pve-devel] [PATCH storage v2 04/10] ovf: implement parsing the ostype

2024-04-19 Thread Dominik Csapak
use the standards info about the ostypes to map to our own (see comment for link to the relevant part of the dmtf schema) every type that is not listed we map to 'other', so no need to have it in a list. Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- changes from v1: * added comment

[pve-devel] [PATCH storage v2 08/10] api: allow ova upload/download

2024-04-19 Thread Dominik Csapak
introducing a separate regex that only contains ova, since upload/downloading ovfs does not make sense (since the disks are then missing). Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- changes from v1: * typo fixes * added OVA mention to description src/PVE/API2/Storage/Status.pm |

[pve-devel] [PATCH qemu-server v2 4/4] api: create: add 'import-extraction-storage' parameter

2024-04-19 Thread Dominik Csapak
this is to override the target extraction storage for the option disk extraction for 'import-from'. This way if the storage does not supports the content type 'images', one can give an alternative one. Signed-off-by: Dominik Csapak --- new in v2 PVE/API2/Qemu.pm | 56 +++

[pve-devel] [PATCH manager v2 5/6] ui: guest import: add storage selector for ova extraction storage

2024-04-19 Thread Dominik Csapak
but only when we detect the 'ova-needs-extraction' warning. This can be used to select the storage where the disks contained in an OVA will be extracted to temporarily. Signed-off-by: Dominik Csapak --- new in v2 www/manager6/window/GuestImport.js | 23 +++ 1 file changed, 23

Re: [pve-devel] [PATCH manager 19/19] notifications: use named templates instead of in-code templates

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm > index 152eb3e5..2ea626f0 100644 > --- a/PVE/VZDump.pm > +++ b/PVE/VZDump.pm The existing $subject_template and $body_template could be removed now like for the others > diff --git a/templates/Makefile b/temp

Re: [pve-devel] [PATCH many 00/19] notifications: move template strings to template files; PBS preparations

2024-04-19 Thread Fiona Ebner
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > The notification system uses handlebar templates to render the subject > and the body of notifications. Previously, the template strings were > defined inline at the call site. This patch series extracts the templates > into template files and installs t

[pve-devel] [PATCH qemu-server v5 2/3] config: QEMU AMD SEV enable

2024-04-19 Thread Markus Frank
This patch is for enabling AMD SEV (Secure Encrypted Virtualization) support in QEMU VM-Config-Examples: amd_sev: type=std,nodbg=1,noks=1 amd_sev: es,nodbg=1,kernel-hashes=1 Node-Config-Example (gets generated automatically): amd_sev: cbitpos=47,reduced-phys-bios=1 kernel-hashes, reduced-phys-bi

[pve-devel] [PATCH qemu-server v5 1/3] add C program to get AMD SEV hardware parameters from CPUID

2024-04-19 Thread Markus Frank
Implement a systemd service that runs a C program that extracts AMD SEV hardware parameters such as reduced-phys-bios and cbitpos from CPUID at boot time, verifies that SEV, SEV-ES & SEV-SNP are enabled, and outputs these details as JSON to /run/amd-sev-params. Signed-off-by: Markus Frank --- Ma

[pve-devel] [PATCH docs v5 3/3] add AMD SEV documentation

2024-04-19 Thread Markus Frank
add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV enable" patch. Signed-off-by: Markus Frank --- v5: * removed NodeConfig part v4: * added text that SEV-ES is experimental qm.adoc | 103 1 file changed, 103 insertions(+

Re: [pve-devel] [PATCH manager v3 5/5] fix #4474: ui: guest stop: offer to overrule active shutdown tasks

2024-04-19 Thread Dominik Csapak
some minor nits inline, aside from those Reviewed-by: Dominik Csapak On 4/12/24 16:15, Friedrich Weber wrote: Implement a new "guest stop" confirmation message box which first checks if there is an active shutdown task for the same guest that is visible to the logged-in user. If there is at l

Re: [pve-devel] [PATCH manager v5 04/16] ui: dc: backup: allow to set custom job id in advanced settings

2024-04-19 Thread Fiona Ebner
Am 15.04.24 um 10:26 schrieb Lukas Wagner: > This might be useful if somebody wants to match on the new > 'backup-job' field in a notification match rule. > > Signed-off-by: Lukas Wagner Needs a rebase, because the advanced settings were moved to a new "Advanced" tab. _

Re: [pve-devel] [PATCH manager v5 04/16] ui: dc: backup: allow to set custom job id in advanced settings

2024-04-19 Thread Fiona Ebner
Am 15.04.24 um 10:26 schrieb Lukas Wagner: > This might be useful if somebody wants to match on the new > 'backup-job' field in a notification match rule. > > Signed-off-by: Lukas Wagner > --- > www/manager6/Utils.js | 4 > www/manager6/dc/Backup.js | 11 +++ > 2 files changed,

Re: [pve-devel] [PATCH manager 19/19] notifications: use named templates instead of in-code templates

2024-04-19 Thread Lukas Wagner
On 2024-04-19 11:59, Fiona Ebner wrote: > Am 09.04.24 um 15:25 schrieb Lukas Wagner: >> diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm >> index 152eb3e5..2ea626f0 100644 >> --- a/PVE/VZDump.pm >> +++ b/PVE/VZDump.pm > > The existing $subject_template and $body_template could be removed now > like

[pve-devel] applied: [PATCH manager v5 01/16] api: notifications: add 'smtp' to target index

2024-04-19 Thread Fiona Ebner
Am 15.04.24 um 10:25 schrieb Lukas Wagner: > Signed-off-by: Lukas Wagner > --- > PVE/API2/Cluster/Notifications.pm | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/PVE/API2/Cluster/Notifications.pm > b/PVE/API2/Cluster/Notifications.pm > index 7047f0b1..68fdda2a 100644

Re: [pve-devel] [RFC PATCH docs-common 01/13] installation-media: move page from pve-docs here

2024-04-19 Thread Aaron Lauterer
On 2024-04-19 11:05, Christoph Heiss wrote: Small adaptions were necessary; mostly a s/{pve}/{product}/g and replacing the ISO URL with the {iso-url} variable. except there are still plenty of `{pve}`s in there? another thing looking at this patch, how do we handle product specifics? havi

[pve-devel] [PATCH docs v6 3/3] add AMD SEV documentation

2024-04-19 Thread Markus Frank
add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV enable" patch. Signed-off-by: Markus Frank --- v5: * removed NodeConfig part v4: * added text that SEV-ES is experimental qm.adoc | 103 1 file changed, 103 insertions(+

[pve-devel] [PATCH qemu-server v6 2/3] config: QEMU AMD SEV enable

2024-04-19 Thread Markus Frank
This patch is for enabling AMD SEV (Secure Encrypted Virtualization) support in QEMU VM-Config-Examples: amd_sev: type=std,nodbg=1,noks=1 amd_sev: es,nodbg=1,kernel-hashes=1 Node-Config-Example (gets generated automatically): amd_sev: cbitpos=47,reduced-phys-bios=1 kernel-hashes, reduced-phys-bi

[pve-devel] [PATCH qemu-server v6 1/3] add C program to get AMD SEV hardware parameters from CPUID

2024-04-19 Thread Markus Frank
Implement a systemd service that runs a C program that extracts AMD SEV hardware parameters such as reduced-phys-bios and cbitpos from CPUID at boot time, looks if SEV, SEV-ES & SEV-SNP are enabled, and outputs these details as JSON to /run/amd-sev-params. Signed-off-by: Markus Frank --- Makefil

Re: [pve-devel] [RFC PATCH docs-common 01/13] installation-media: move page from pve-docs here

2024-04-19 Thread Christoph Heiss
Thanks for the review! On Fri, Apr 19, 2024 at 12:51:07PM +0200, Aaron Lauterer wrote: > > > On 2024-04-19 11:05, Christoph Heiss wrote: > > Small adaptions were necessary; mostly a s/{pve}/{product}/g and > > replacing the ISO URL with the {iso-url} variable. > > except there are still plenty o

Re: [pve-devel] [PATCH many 00/19] notifications: move template strings to template files; PBS preparations

2024-04-19 Thread Fabian Grünbichler
On April 19, 2024 12:09 pm, Fiona Ebner wrote: > Am 09.04.24 um 15:25 schrieb Lukas Wagner: >> Bumps/dependencies: >> - proxmox_notify >> - libproxmox-rs-perl/libpve-rs-perl (needs bumped proxmox_notify) >> - libpve-notify-perl (needs bumped >> libproxmox-rs-perl/libpve-rs-perl)

Re: [pve-devel] [PATCH many 00/19] notifications: move template strings to template files; PBS preparations

2024-04-19 Thread Lukas Wagner
On 2024-04-19 13:22, Fabian Grünbichler wrote: > On April 19, 2024 12:09 pm, Fiona Ebner wrote: >> Am 09.04.24 um 15:25 schrieb Lukas Wagner: >>> Bumps/dependencies: >>> - proxmox_notify >>> - libproxmox-rs-perl/libpve-rs-perl (needs bumped proxmox_notify) >>> - libpve-notify-p

Re: [pve-devel] [PATCH manager v5 02/16] api: jobs: vzdump: pass job 'id' parameter

2024-04-19 Thread Fiona Ebner
Am 15.04.24 um 10:26 schrieb Lukas Wagner: > This allows us to access us the backup job id in the send_notification > function, where we can set it as metadata for the notification. > > Signed-off-by: Lukas Wagner > --- > PVE/API2/VZDump.pm | 8 > PVE/Jobs/VZDump.pm | 4 +++- > PVE/VZDu

Re: [pve-devel] [PATCH manager v5 05/16] api: replication: add 'replication-job' to notification metadata

2024-04-19 Thread Fiona Ebner
Am 15.04.24 um 10:26 schrieb Lukas Wagner: > This allows users to create notification match rules for specific > replication jobs, if they so desire. > > Signed-off-by: Lukas Wagner > --- > PVE/API2/Replication.pm | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/PVE/API2

Re: [pve-devel] [PATCH manager v5 05/16] api: replication: add 'replication-job' to notification metadata

2024-04-19 Thread Lukas Wagner
On 2024-04-19 14:02, Fiona Ebner wrote: > Am 15.04.24 um 10:26 schrieb Lukas Wagner: >> This allows users to create notification match rules for specific >> replication jobs, if they so desire. >> >> Signed-off-by: Lukas Wagner >> --- >> PVE/API2/Replication.pm | 2 +- >> 1 file changed, 1 in

Re: [pve-devel] [PATCH manager v5 04/16] ui: dc: backup: allow to set custom job id in advanced settings

2024-04-19 Thread Lukas Wagner
On 2024-04-19 12:31, Fiona Ebner wrote: > Am 15.04.24 um 10:26 schrieb Lukas Wagner: >> This might be useful if somebody wants to match on the new >> 'backup-job' field in a notification match rule. >> >> Signed-off-by: Lukas Wagner >> --- >> www/manager6/Utils.js | 4 >> www/manage

[pve-devel] [PATCH guest-common v3 1/4] mapping: pci: rework properties check

2024-04-19 Thread Dominik Csapak
rename '$cfg' to '$mapping', 'correct' to 'expected' reword the error messages also check keys from the configured props not only the expected ones previously we only checked the keys from the 'correct_props' hash but that was unintended. We now check the keys from both, but extract the relevant p

[pve-devel] [PATCH qemu-server v3 02/10] pci: mapping: move implementation of find_on_current_node here

2024-04-19 Thread Dominik Csapak
this was the only user, and it's easy enough Signed-off-by: Dominik Csapak --- PVE/QemuServer/PCI.pm | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm index 1673041b..7ff9cad7 100644 --- a/PVE/QemuServer/PCI.pm +++ b/PVE/QemuSer

[pve-devel] [PATCH guest-common/qemu-server/manager/docs v3 0/4] implement experimental vgpu live migration

2024-04-19 Thread Dominik Csapak
and some useful cleanups Resending even there was not much feedback, because i worked in some minor fixes/changes in the meantime. A user tested the previous patch series and only found one issue with the ui, see the comments on bug #5175 https://bugzilla.proxmox.com/show_bug.cgi?id=5175 This

[pve-devel] [PATCH qemu-server v3 01/10] usb: mapping: move implementation of find_on_current_node here

2024-04-19 Thread Dominik Csapak
this was the only user, and it's easy enough Signed-off-by: Dominik Csapak --- PVE/QemuServer/USB.pm | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/PVE/QemuServer/USB.pm b/PVE/QemuServer/USB.pm index 49957444..ecd0361d 100644 --- a/PVE/QemuServer/USB.pm +++ b/PVE/QemuSer

[pve-devel] [PATCH qemu-server v3 07/10] pci: set 'enable-migration' to on for live-migration marked mapped devices

2024-04-19 Thread Dominik Csapak
the default is 'auto', but for those which are marked as capable for live migration, we want to explicitly enable that, so we get an early error on start if the driver does not support that. Signed-off-by: Dominik Csapak --- PVE/QemuServer/PCI.pm | 9 - 1 file changed, 8 insertions(+), 1

[pve-devel] [PATCH qemu-server v3 06/10] migrate: call vm_stop_cleanup after stopping in phase3_cleanup

2024-04-19 Thread Dominik Csapak
we currently only call deactivate_volumes, but we actually want to call the whole vm_stop_cleanup, since that is not invoked by the vm_stop above (we cannot parse the config anymore) and might do other cleanups we also want to do (like mdev cleanup). For this to work properly we have to clone the

[pve-devel] [PATCH docs v3 1/2] qm: resource mapping: add description for `mdev` option

2024-04-19 Thread Dominik Csapak
in a new section about additional options Signed-off-by: Dominik Csapak --- qm.adoc | 13 + 1 file changed, 13 insertions(+) diff --git a/qm.adoc b/qm.adoc index 42c26db..3f4e59a 100644 --- a/qm.adoc +++ b/qm.adoc @@ -1802,6 +1802,19 @@ To create mappings `Mapping.Modify` on `/mapp

[pve-devel] [PATCH qemu-server v3 08/10] check_local_resources: add more info per mapped device and return as hash

2024-04-19 Thread Dominik Csapak
such as the mapping name and if it's marked for live-migration (pci only) Signed-off-by: Dominik Csapak --- PVE/API2/Qemu.pm | 2 +- PVE/QemuMigrate.pm | 7 --- PVE/QemuServer.pm | 17 ++--- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/PVE/API2/Qemu.pm b/

[pve-devel] [PATCH manager v3 2/5] bulk migrate: improve precondition checks

2024-04-19 Thread Dominik Csapak
this now takes into account the 'not_allowed_nodes' hash we get from the api call. With that, we can now limit the 'local_resources' check for online vms only, as for offline guests, the 'unavailable-resources' hash already includes mapped devices that don't exist on the target node. This now also

[pve-devel] [PATCH manager v3 4/5] ui: adapt migration window to precondition api change

2024-04-19 Thread Dominik Csapak
we now return the 'allowed_nodes'/'not_allowed_nodes' also if the vm is running, when it has mapped resources. So do that checks independently so that the user has instant feedback where those resources exist. Signed-off-by: Dominik Csapak --- www/manager6/window/Migrate.js | 26 +---

[pve-devel] [PATCH qemu-server v3 09/10] api: enable live migration for marked mapped pci devices

2024-04-19 Thread Dominik Csapak
They have to be marked as 'live-migration-capable' in the mapping config, and the driver and qemu must support it. For the gui checks, we now return the whole object of the mapped resources, which includes info like the name and if it's marked as live-migration capable. (while deprecating the old

[pve-devel] [PATCH manager v3 5/5] fix #5175: ui: allow configuring and live migration of mapped pci resources

2024-04-19 Thread Dominik Csapak
if the hardware/driver is capable, the admin can now mark a pci device as 'live-migration-capable', which then tries enabling live migration for such devices. mark it as experimental when configuring and in the migrate window Signed-off-by: Dominik Csapak --- www/manager6/window/Migrate.js|

[pve-devel] [PATCH guest-common v3 3/4] mapping: pci: add 'live-migration-capable' flag to mappings

2024-04-19 Thread Dominik Csapak
so that we can decide in qemu-server to allow live-migration. The driver and QEMU must be capable of that, and it's the admin's responsibility to know and configure that Mark the option as experimental in the description. Signed-off-by: Dominik Csapak --- src/PVE/Mapping/PCI.pm | 8 1

[pve-devel] [PATCH guest-common v3 2/4] mapping: pci: check the mdev configuration on the device too

2024-04-19 Thread Dominik Csapak
but that lives int he 'global' part of the mapping config, not in a specific mapping. To check that, add it to the $configured_props from there. this requires all call sites to be adapted otherwise the check will always fail for devices that are capable of mediated devices Signed-off-by: Dominik

[pve-devel] [PATCH guest-common v3 4/4] mapping: remove find_on_current_node

2024-04-19 Thread Dominik Csapak
they only have one user each (where we can inline the implementation). It's easy enough to recreate should we need to. Signed-off-by: Dominik Csapak --- src/PVE/Mapping/PCI.pm | 10 -- src/PVE/Mapping/USB.pm | 9 - 2 files changed, 19 deletions(-) diff --git a/src/PVE/Mapping/P

[pve-devel] [PATCH qemu-server v3 03/10] pci: mapping: check mdev config against hardware

2024-04-19 Thread Dominik Csapak
by giving the mapping config to assert_valid, not only the specific mapping Signed-off-by: Dominik Csapak --- PVE/QemuServer/PCI.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm index 7ff9cad7..6ba43ee8 100644 --- a/PVE/QemuServ

[pve-devel] [PATCH qemu-server v3 04/10] stop cleanup: remove unnecessary tpmstate cleanup

2024-04-19 Thread Dominik Csapak
tpmstate0 is already included in `get_vm_volumes`, and our only storage plugin that has unmap_volume implemented is the RBDPlugin, where we call unmap in `deactivate_volume`. So it's already ummapped by the `deactivate_volumes` calls above. Signed-off-by: Dominik Csapak --- PVE/QemuServer.pm | 8

Re: [pve-devel] [PATCH proxmox 09/19] notify: derive Deserialize/Serialize for Notification struct

2024-04-19 Thread Lukas Wagner
On 2024-04-19 10:45, Fiona Ebner wrote: > Nit: I always like a quick sentence for who needs it for such changes. > > Am 09.04.24 um 15:25 schrieb Lukas Wagner: >> Signed-off-by: Lukas Wagner >> --- >> proxmox-notify/src/lib.rs | 10 +++--- >> 1 file changed, 7 insertions(+), 3 deletions(

  1   2   >