[pve-devel] applied: [PATCH manager v2 0/2] fix #5093 add custom directory and eab to ui

2024-04-17 Thread Thomas Lamprecht
Am 17/04/2024 um 17:55 schrieb Folke Gleumes: > This patch series adds the option to set a custom directory for ACME and > enables the user to use external account binding, which is required by > some providers. > > manager: > > Folke Gleumes (2): > fix #5093: webui: acme: custom directory

[pve-devel] applied: [PATCH manager v3 4/5] ui: fix typo to make pve-cluster-tasks store globally available

2024-04-17 Thread Thomas Lamprecht
Am 12/04/2024 um 16:15 schrieb Friedrich Weber: > This way, it can be used to retrieve the current list of tasks. > > Signed-off-by: Friedrich Weber > --- > > Notes: > changes v2 -> v3: > * no changes > > new in v2: > * moved fix for pve-cluster-tasks store into its own

[pve-devel] applied: [PATCH qemu-server v3 3/5] fix #4474: qemu api: add overrule-shutdown parameter to stop endpoint

2024-04-17 Thread Thomas Lamprecht
Am 12/04/2024 um 16:15 schrieb Friedrich Weber: > The new `overrule-shutdown` parameter is boolean and defaults to 0. If > it is 1, all active `qmshutdown` tasks for the same VM (which are > visible to the user/token) are aborted before attempting to stop the > VM. > > Passing

[pve-devel] applied: [PATCH guest-common v3 1/5] guest helpers: add helper to abort active guest tasks of a certain type

2024-04-17 Thread Thomas Lamprecht
Am 12/04/2024 um 16:15 schrieb Friedrich Weber: > Given a `(type, user, vmid)` tuple, the helper aborts all tasks of the > given `type` for guest `vmid` that `user` is allowed to abort: > > - If `user` has `Sys.Modify` on the node, they can abort any task > - If `user` is an API token, it can

[pve-devel] applied: [PATCH container v3 2/5] fix #4474: lxc api: add overrule-shutdown parameter to stop endpoint

2024-04-17 Thread Thomas Lamprecht
Am 12/04/2024 um 16:15 schrieb Friedrich Weber: > The new `overrule-shutdown` parameter is boolean and defaults to 0. If > it is 1, all active `vzshutdown` tasks for the same CT (which are > visible to the user/token) are aborted before attempting to stop the > CT. > > Passing

[pve-devel] [PATCH manager v2 0/2] fix #5093 add custom directory and eab to ui

2024-04-17 Thread Folke Gleumes
This patch series adds the option to set a custom directory for ACME and enables the user to use external account binding, which is required by some providers. manager: Folke Gleumes (2): fix #5093: webui: acme: custom directory option webui: acme: add eab fields www/manager6/node/ACME.js

[pve-devel] [PATCH manager v2 2/2] webui: acme: add eab fields

2024-04-17 Thread Folke Gleumes
Adds fields for eab credentials. By default eab is optional, but if the directory should report that eab is required, the eab credential fields are marked as mandatory and prevent the form from being submittable until credentials are provided. Signed-off-by: Folke Gleumes ---

[pve-devel] [PATCH manager v2 1/2] fix #5093: webui: acme: custom directory option

2024-04-17 Thread Folke Gleumes
This patch allows the user to set a custom ACME directory by providing a 'Custom' option in the directory dropdown. This in turn reveals an input for the url. When using a custom directory the directory has to be manually queried via button press to prevent from spamming the directory on every

Re: [pve-devel] [PATCH ksm-control-daemon] ksmtuned: use PSS instead of RSZ for caluculating memory usage

2024-04-17 Thread Thomas Lamprecht
Am 11/04/2024 um 12:04 schrieb Roland: > where arcsize is not taken into account > > https://bugzilla.proxmox.com/show_bug.cgi?id=3859 I think this bug should be split, as those are two completely different things implementation wise. The existing one could be kept for RRD, and a new one added

[pve-devel] applied: [PATCH ha-manager] d/postinst: make deb-systemd-invoke non-fatal

2024-04-17 Thread Thomas Lamprecht
Am 11/04/2024 um 12:10 schrieb Fabian Grünbichler: > else this can break an upgrade for unrelated reasons. > > this also mimics debhelper behaviour more (which we only not use here because > of lack of reload support) - restructured the snippet to be more similar with > an explicit `if` as well.

[pve-devel] applied: [PATCH manager] d/postinst: make deb-systemd-invoke non-fatal

2024-04-17 Thread Thomas Lamprecht
Am 11/04/2024 um 12:10 schrieb Fabian Grünbichler: > else this can break an upgrade for unrelated reasons (regular debhelper also > constructs the restart invocations like this, it even redirects output to > /dev/null) > > Signed-off-by: Fabian Grünbichler > --- > debian/postinst | 12

Re: [pve-devel] [PATCH storage] plugin: move definition for 'port' option to base plugin

2024-04-17 Thread Thomas Lamprecht
Am 15/04/2024 um 14:48 schrieb Fiona Ebner: > Commit 7020491 ("esxi: add 'port' config parameter") started using > the 'port' option in a second plugin, but the definition stayed in the > PBS plugin. Avoid the hidden dependency and move the definition to the > base plugin instead. > > It is

[pve-devel] applied: [PATCH docs] storage: pbs: document port option

2024-04-17 Thread Thomas Lamprecht
Am 15/04/2024 um 14:48 schrieb Fiona Ebner: > Signed-off-by: Fiona Ebner > --- > pve-storage-pbs.adoc | 4 > 1 file changed, 4 insertions(+) > > applied, thanks! ___ pve-devel mailing list pve-devel@lists.proxmox.com

[pve-devel] applied: [PATCH kernel 1/1] cherry-pick improved erratum 1386 workaround

2024-04-17 Thread Thomas Lamprecht
Am 15/04/2024 um 14:56 schrieb Folke Gleumes: > The original fix disabled the xsaves feature for zen1/2. The issue has > since been fixed in the cpus microcode and this patch keeps the feature > enabled > if the microcode version is recent enough to contain the fix. > > Signed-off-by: Folke

Re: [pve-devel] [PATCH v2 container] fix #4846: Avoid the outdated noacl mount option on ext4

2024-04-17 Thread Filip Schauer
On 11/04/2024 15:44, Fabian Grünbichler wrote: I am not sure this is correct.. or rather, wouldn't it be simpler to say if $storage && $format eq 'raw' => no noacl ? if we get complains that somebody did something non-standard (i.e., manually formatted a raw volume using a different

[pve-devel] [PATCH v3 container] fix #4846: Avoid the outdated noacl mount option on ext4

2024-04-17 Thread Filip Schauer
Do not use the 'noacl' mount option when mounting a container disk with an ext4 file system. The option was removed from the kernel in commit 2d544ec923db Signed-off-by: Filip Schauer --- Changes since v3: * Simplify ext4 detection * Do not add noacl if $acl is undefined src/PVE/LXC.pm | 16

Re: [pve-devel] [PATCH manager 1/2] fix #5093: webui: acme: custom directory option

2024-04-17 Thread Mira Limbeck
On 1/16/24 15:33, Folke Gleumes wrote: > This patch allows the user to set a custom ACME directory by providing > a 'Custom' option in the directory dropdown. This in turn reveals an > input for the url. When using a custom directory the directory has to > be manually queried via button press

Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import

2024-04-17 Thread Dominik Csapak
On 4/17/24 15:52, Fabian Grünbichler wrote: On April 17, 2024 3:10 pm, Dominik Csapak wrote: On 4/17/24 14:45, Fabian Grünbichler wrote: On April 16, 2024 3:18 pm, Dominik Csapak wrote: +sub cleanup_extracted_image { same for this? +my ($source) = @_; + +if ($source =~

[pve-devel] applied-series: [PATCH-SERIES v4 manager/docs] close #4513: add advanced tab for backup jobs and improve performance fallback/default

2024-04-17 Thread Thomas Lamprecht
Am 16/04/2024 um 14:09 schrieb Fiona Ebner: > Changes in v4 (Thanks to Thomas for feedback!): > * rename tab from 'Performance' to 'Advanced' > * move repeat-missed setting there too > * update docs to clarify that those settings can be found in the > advanced tab > > Changes in

[pve-devel] [PATCH pve-container v2 36/39] firewall: add handling for new nft firewall

2024-04-17 Thread Stefan Hanreich
When the nftables firewall is enabled, we do not need to create firewall bridges. Signed-off-by: Stefan Hanreich --- src/PVE/LXC.pm | 5 + 1 file changed, 5 insertions(+) diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm index e688ea6..85800ea 100644 --- a/src/PVE/LXC.pm +++ b/src/PVE/LXC.pm @@

[pve-devel] [PATCH proxmox-firewall v2 16/39] config: firewall: add conntrack helper types

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/resources/ct_helper.json | 52 + proxmox-ve-config/src/firewall/ct_helper.rs | 115 proxmox-ve-config/src/firewall/mod.rs

[pve-devel] [PATCH proxmox-firewall v2 33/39] firewall: add files for debian packaging

2024-04-17 Thread Stefan Hanreich
Suggested-By: Fabian Grünbichler Signed-off-by: Stefan Hanreich --- .gitignore | 3 ++ Makefile| 70 + debian/changelog| 5 +++ debian/control | 38 ++

[pve-devel] [PATCH proxmox-firewall v2 19/39] nftables: expression: add types

2024-04-17 Thread Stefan Hanreich
Adds an enum containing most of the expressions defined in the nftables-json schema [1]. [1] https://manpages.debian.org/bookworm/libnftables1/libnftables-json.5.en.html#EXPRESSIONS Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan

[pve-devel] [PATCH proxmox-firewall v2 29/39] firewall: add rule generation logic

2024-04-17 Thread Stefan Hanreich
ToNftRules is basically a conversion trait for firewall config structs to convert them into the respective nftables statements. We are passing a list of rules to the method, which then modifies the list of rules such that all relevant rules in the list have statements appended that apply the

[pve-devel] [PATCH proxmox-firewall v2 28/39] firewall: add config loader

2024-04-17 Thread Stefan Hanreich
We load the firewall configuration from the default paths, as well as only the guest configurations that are local to the node itself. In the future we could change this to use pmxcfs directly instead. We also load information from nftables directly about dynamically created chains (mostly chains

[pve-devel] [PATCH pve-docs v2 39/39] firewall: add documentation for proxmox-firewall

2024-04-17 Thread Stefan Hanreich
Add a section that explains how to use the new nftables-based proxmox-firewall. Signed-off-by: Stefan Hanreich --- pve-firewall.adoc | 162 ++ 1 file changed, 162 insertions(+) diff --git a/pve-firewall.adoc b/pve-firewall.adoc index a5e40f9..ac3d9ba

[pve-devel] [PATCH proxmox-firewall v2 24/39] nftables: types: add conversion traits

2024-04-17 Thread Stefan Hanreich
Some parts of the firewall config map directly to nftables objects, so we introduce conversion traits for convenient conversion into the respective nftables objects / types. They are guarded behind a feature, so the nftables crate can be used standalone without depending on the proxmox-ve-config

[pve-devel] [PATCH proxmox-firewall v2 22/39] nftables: statement: add conversion traits for config types

2024-04-17 Thread Stefan Hanreich
Some types from the firewall configuration map directly onto nftables statements. For those we implement conversion traits so we can conveniently convert between the configuration types and the respective nftables types. As with the expressions, those are guarded behind a feature so the nftables

[pve-devel] [PATCH proxmox-firewall v2 34/39] firewall: add integration test

2024-04-17 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich --- .gitignore|1 + debian/control|1 + proxmox-firewall/Cargo.toml |4 + proxmox-firewall/src/lib.rs |4 + proxmox-firewall/tests/input/100.conf

[pve-devel] [PATCH proxmox-firewall v2 23/39] nftables: commands: add types

2024-04-17 Thread Stefan Hanreich
Add rust types for most of the nftables commands as defined by libnftables-json [1]. Different commands require different keys to be set for the same type of object. E.g. deleting an object usually only requires a name + name of the container (table/chain/rule). Creating an object usually

[pve-devel] [PATCH qemu-server v2 35/39] firewall: add handling for new nft firewall

2024-04-17 Thread Stefan Hanreich
When the nftables firewall is enabled, we do not need to create firewall bridges. Signed-off-by: Stefan Hanreich --- vm-network-scripts/pve-bridge | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/vm-network-scripts/pve-bridge b/vm-network-scripts/pve-bridge index

[pve-devel] [PATCH proxmox-firewall v2 09/39] config: firewall: add types for rules

2024-04-17 Thread Stefan Hanreich
Additionally we implement FromStr for all rule types and parts, which can be used for parsing firewall config rules. Initial rule parsing works by parsing the different options into a HashMap and only then de-serializing a struct from the parsed options. This intermediate step makes rule parsing

[pve-devel] [PATCH proxmox-firewall v2 07/39] config: guest: add helpers for parsing guest network config

2024-04-17 Thread Stefan Hanreich
Currently this is parsing the config files via the filesystem. In the future we could also get this information from pmxcfs directly via IPC which should be more performant, particularly for a large number of VMs. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang

[pve-devel] [PATCH proxmox-firewall v2 15/39] config: firewall: add firewall macros

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/resources/macros.json | 914 proxmox-ve-config/src/firewall/fw_macros.rs | 69 ++ proxmox-ve-config/src/firewall/mod.rs |

[pve-devel] [PATCH proxmox-firewall v2 20/39] nftables: expression: implement conversion traits for firewall config

2024-04-17 Thread Stefan Hanreich
Some types from the firewall configuration map directly onto nftables expressions. For those we implement conversion traits so we can conveniently convert between the configuration types and the respective nftables types. Those are guarded behind a feature so the nftables crate can be used

[pve-devel] [PATCH proxmox-firewall v2 26/39] firewall: add firewall crate

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- Cargo.toml | 1 + proxmox-firewall/Cargo.toml | 17 + proxmox-firewall/src/main.rs | 5 + 3 files changed, 23 insertions(+) create

[pve-devel] [PATCH proxmox-firewall v2 11/39] config: firewall: add generic parser for firewall configs

2024-04-17 Thread Stefan Hanreich
Since the basic format of cluster, host and guest firewall configurations is the same, we create a generic parser that can handle the common config format. The main difference is in the available options, which can be passed via a generic parameter. Reviewed-by: Lukas Wagner Reviewed-by: Max

[pve-devel] [PATCH proxmox-firewall v2 13/39] config: firewall: add host specific config + option types

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/src/firewall/host.rs | 372 + proxmox-ve-config/src/firewall/mod.rs | 1 + 2 files changed, 373 insertions(+) create mode

[pve-devel] [PATCH proxmox-firewall v2 14/39] config: firewall: add guest-specific config + option types

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/src/firewall/guest.rs | 237 proxmox-ve-config/src/firewall/mod.rs | 1 + 2 files changed, 238 insertions(+) create mode

[pve-devel] [PATCH proxmox-firewall v2 08/39] config: firewall: add types for ipsets

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/src/firewall/types/ipset.rs | 349 ++ proxmox-ve-config/src/firewall/types/mod.rs | 2 + 2 files changed, 351 insertions(+) create

[pve-devel] [PATCH proxmox-firewall v2 12/39] config: firewall: add cluster-specific config + option types

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/src/firewall/cluster.rs | 374 ++ proxmox-ve-config/src/firewall/mod.rs | 1 + 2 files changed, 375 insertions(+) create mode

[pve-devel] [PATCH proxmox-firewall v2 17/39] nftables: add crate for libnftables bindings

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- Cargo.toml | 1 + proxmox-nftables/Cargo.toml | 16 proxmox-nftables/src/lib.rs | 0 3 files changed, 17 insertions(+) create mode

[pve-devel] [PATCH proxmox-firewall v2 27/39] firewall: add base ruleset

2024-04-17 Thread Stefan Hanreich
This is the skeleton for the firewall that contains all the base chains required for the firewall. The file applies atomically, which means that it flushes all objects and recreates them - except for the cluster/host/guest chain. This means that it can be run at any point in time, since it only

[pve-devel] [PATCH proxmox-firewall v2 31/39] firewall: add ruleset generation logic

2024-04-17 Thread Stefan Hanreich
We create the rules from the firewall config by utilizing the ToNftRules and ToNftObjects traits to convert the firewall config structs to nftables objects/chains/rules. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich ---

[pve-devel] [PATCH pve-firewall v2 37/39] add configuration option for new nftables firewall

2024-04-17 Thread Stefan Hanreich
Introduces new nftables configuration option that en/disables the new nftables firewall. pve-firewall reads this option and only generates iptables rules when nftables is set to `0`. Conversely proxmox-firewall only generates nftables rules when the option is set to `1`. Signed-off-by: Stefan

[pve-devel] [PATCH pve-manager v2 38/39] firewall: expose configuration option for new nftables firewall

2024-04-17 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich --- www/manager6/grid/FirewallOptions.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/grid/FirewallOptions.js b/www/manager6/grid/FirewallOptions.js index 0ac9979c4..6aacb47be 100644 --- a/www/manager6/grid/FirewallOptions.js +++

[pve-devel] [PATCH proxmox-firewall v2 25/39] nftables: add libnftables bindings

2024-04-17 Thread Stefan Hanreich
Add a thin wrapper around libnftables, which can be used to run commands defined by the rust types. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-nftables/src/context.rs | 243

[pve-devel] [PATCH proxmox-firewall v2 30/39] firewall: add object generation logic

2024-04-17 Thread Stefan Hanreich
ToNftObjects is basically a conversion trait that converts firewall config structs into nftables objects. It returns a list of commands that create the respective nftables objects. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan

[pve-devel] [PATCH proxmox-firewall v2 21/39] nftables: statement: add types

2024-04-17 Thread Stefan Hanreich
Adds an enum containing most of the statements defined in the nftables-json schema [1]. [1] https://manpages.debian.org/bookworm/libnftables1/libnftables-json.5.en.html#STATEMENTS Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan

[pve-devel] [PATCH proxmox-firewall v2 32/39] firewall: add proxmox-firewall binary

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-firewall/src/main.rs | 34 ++ 1 file changed, 34 insertions(+) diff --git a/proxmox-firewall/src/main.rs

[pve-devel] [PATCH proxmox-firewall v2 02/39] config: firewall: add types for ip addresses

2024-04-17 Thread Stefan Hanreich
Includes types for all kinds of IP values that can occur in the firewall config. Additionally, FromStr implementations are available for parsing from the config files. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich ---

[pve-devel] [PATCH proxmox-firewall v2 05/39] config: firewall: add types for aliases

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/src/firewall/parse.rs | 52 ++ proxmox-ve-config/src/firewall/types/alias.rs | 160 ++

[pve-devel] [PATCH proxmox-firewall v2 06/39] config: host: add helpers for host network configuration

2024-04-17 Thread Stefan Hanreich
Currently the helpers for obtaining the host network configuration panic on error, which could be avoided by the use of OnceLock::get_or_init, but this method is currently only available in nightly versions. Generally, if there is a problem with obtaining the network config for the node I would

[pve-devel] [PATCH proxmox-firewall v2 18/39] nftables: add helpers

2024-04-17 Thread Stefan Hanreich
Several objects, statements and expressions in nftables-json require null values, for instance: { "flush": { "ruleset": null }} For this purpose we define our own Null type, which we can then easily use for defining types that accept Null as value. Several keys accept as value either a

[pve-devel] [PATCH container/docs/firewall/manager/proxmox-firewall/qemu-server v2 00/39] proxmox firewall nftables implementation

2024-04-17 Thread Stefan Hanreich
## Introduction This RFC provides a drop-in replacement for the current pve-firewall package that is based on Rust and nftables. It consists of three crates: * proxmox-ve-config for parsing firewall and guest configuration files, as well as some helpers to access host configuration

[pve-devel] [PATCH proxmox-firewall v2 03/39] config: firewall: add types for ports

2024-04-17 Thread Stefan Hanreich
Adds types for all kinds of port-related values in the firewall config as well as FromStr implementations for parsing them from the config. Also adds a helper for parsing the named ports from `/etc/services`. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller

[pve-devel] [PATCH proxmox-firewall v2 10/39] config: firewall: add types for security groups

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/src/firewall/types/group.rs | 36 +++ proxmox-ve-config/src/firewall/types/mod.rs | 2 ++ 2 files changed, 38 insertions(+) create mode

[pve-devel] [PATCH proxmox-firewall v2 04/39] config: firewall: add types for log level and rate limit

2024-04-17 Thread Stefan Hanreich
Adds types for log and (log-)rate-limiting firewall config options as well as FromStr implementations for parsing them from the config. Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- proxmox-ve-config/Cargo.toml

[pve-devel] [PATCH proxmox-firewall v2 01/39] config: add proxmox-ve-config crate

2024-04-17 Thread Stefan Hanreich
Reviewed-by: Lukas Wagner Reviewed-by: Max Carrara Co-authored-by: Wolfgang Bumiller Signed-off-by: Stefan Hanreich --- .cargo/config| 5 + .gitignore | 6 ++ Cargo.toml | 4 proxmox-ve-config/Cargo.toml | 19

Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import

2024-04-17 Thread Fabian Grünbichler
On April 17, 2024 3:10 pm, Dominik Csapak wrote: > On 4/17/24 14:45, Fabian Grünbichler wrote: >> On April 16, 2024 3:18 pm, Dominik Csapak wrote: >>> +sub cleanup_extracted_image { >> >> same for this? >> >>> +my ($source) = @_; >>> + >>> +if ($source =~

[pve-devel] applied: [PATCH pve-flutter-frontend] node overview: power menu: reorder/reword confirm buttons

2024-04-17 Thread Thomas Lamprecht
Am 17/04/2024 um 10:53 schrieb Dominik Csapak: > move the confirm action to the right as mentioned in the material spec[0] > also rewords the buttons to 'cancel' and 'shutdown/reboot' > for that to work properly slightly rename the confirm message > > 0: >

Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import

2024-04-17 Thread Fabian Grünbichler
On April 17, 2024 3:07 pm, Dominik Csapak wrote: > On 4/17/24 12:52, Fiona Ebner wrote: >> Am 16.04.24 um 15:18 schrieb Dominik Csapak: >>> since we want to handle ova files (which are only ovf+vmdks bundled in a >>> tar file) for import, add code that handles that. >>> >>> we introduce a valid

Re: [pve-devel] [PATCH storage/qemu-server/pve-manager] implement ova/ovf import for directory type storages

2024-04-17 Thread Dominik Csapak
On 4/17/24 15:11, Fabian Grünbichler wrote: On April 16, 2024 3:18 pm, Dominik Csapak wrote: This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and

Re: [pve-devel] [PATCH storage 7/9] ovf: implement parsing nics

2024-04-17 Thread Dominik Csapak
On 4/17/24 14:09, Fiona Ebner wrote: Am 16.04.24 um 15:19 schrieb Dominik Csapak: by iterating over the relevant parts and trying to parse out the 'ResourceSubType'. The content of that is not standardized, but I only ever found examples that are compatible with vmware, meaning it's either

Re: [pve-devel] [PATCH storage 6/9] ovf: implement rudimentary boot order

2024-04-17 Thread Dominik Csapak
On 4/17/24 13:54, Fiona Ebner wrote: Am 16.04.24 um 15:18 schrieb Dominik Csapak: simply add all parsed disks to the boot order in the order we encounter them (similar to the esxi plugin). Signed-off-by: Dominik Csapak --- src/PVE/Storage/OVF.pm| 6 ++ src/test/run_ovf_tests.pl | 3

Re: [pve-devel] [PATCH storage 4/9] ovf: implement parsing the ostype

2024-04-17 Thread Dominik Csapak
On 4/17/24 13:32, Fiona Ebner wrote: Am 16.04.24 um 15:18 schrieb Dominik Csapak: use the standards info about the ostypes to map to our own (see comment for link to the relevant part of the dmtf schema) every type that is not listed we map to 'other', so no need to have it in a list.

Re: [pve-devel] [PATCH storage 2/9] plugin: dir: implement import content type

2024-04-17 Thread Dominik Csapak
On 4/17/24 12:07, Fiona Ebner wrote: Am 16.04.24 um 15:18 schrieb Dominik Csapak: in DirPlugin and not Plugin (because of cyclic dependency of Plugin -> OVF -> Storage -> Plugin otherwise) only ovf is currently supported (though ova will be shown in import listing), expects the files to not be

Re: [pve-devel] [PATCH storage/qemu-server/pve-manager] implement ova/ovf import for directory type storages

2024-04-17 Thread Fabian Grünbichler
On April 16, 2024 3:18 pm, Dominik Csapak wrote: > This series enables importing ova/ovf from directory based storages, > inclusive upload/download via the webui (ova only). > > It also improves the ovf importer by parsing the ostype, nics, bootorder > (and firmware from vmware exported files). >

Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import

2024-04-17 Thread Dominik Csapak
On 4/17/24 14:45, Fabian Grünbichler wrote: On April 16, 2024 3:18 pm, Dominik Csapak wrote: since we want to handle ova files (which are only ovf+vmdks bundled in a tar file) for import, add code that handles that. we introduce a valid volname for files contained in ovas like this:

Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import

2024-04-17 Thread Dominik Csapak
On 4/17/24 12:52, Fiona Ebner wrote: Am 16.04.24 um 15:18 schrieb Dominik Csapak: since we want to handle ova files (which are only ovf+vmdks bundled in a tar file) for import, add code that handles that. we introduce a valid volname for files contained in ovas like this:

Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import

2024-04-17 Thread Fabian Grünbichler
On April 16, 2024 3:18 pm, Dominik Csapak wrote: > since we want to handle ova files (which are only ovf+vmdks bundled in a > tar file) for import, add code that handles that. > > we introduce a valid volname for files contained in ovas like this: > > storage:import/archive.ova/disk-1.vmdk > >

Re: [pve-devel] [PATCH storage 2/9] plugin: dir: implement import content type

2024-04-17 Thread Fabian Grünbichler
On April 16, 2024 3:18 pm, Dominik Csapak wrote: > in DirPlugin and not Plugin (because of cyclic dependency of > Plugin -> OVF -> Storage -> Plugin otherwise) > > only ovf is currently supported (though ova will be shown in import > listing), expects the files to not be in a subdir, and adjacent

[pve-devel] [PATCH installer v6 30/36] add proxmox-chroot utility

2024-04-17 Thread Aaron Lauterer
it is meant as a helper utility to prepare an installation for chroot and clean up afterwards It tries to determine the used FS from the previous installation, will do what is necessary to mount/import the root FS to /target. It then will set up all bind mounts. Tested-by: Christoph Heiss

[pve-devel] [PATCH installer v6 26/36] auto installer: factor out fetch-answer and autoinst-helper

2024-04-17 Thread Aaron Lauterer
Putting proxmox-fetch-answer into it's own crate, will keep the use of OpenSSL localized to where we need it. Otherwise building other binaries will always depend on OpenSSL as well, even without actually needing it. Having a dedicated crate for the proxmox-autoinst-helper should make it easier

[pve-devel] [PATCH installer v6 31/36] auto-installer: answer: deny unknown fields

2024-04-17 Thread Aaron Lauterer
This way, serde will throw errors if fields are not known. This can help to reduce frustration if one might think to have set an option, but for example a small type has happened. Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer ---

Re: [pve-devel] [PATCH manager 4/4] ui: enable upload/download buttons for 'import' type storages

2024-04-17 Thread Fabian Grünbichler
On April 16, 2024 3:19 pm, Dominik Csapak wrote: > but only for non esxi ones, since that does not allow > uploading/downloading there what about a remove button? :) > > Signed-off-by: Dominik Csapak > --- > www/manager6/storage/Browser.js| 7 ++- >

[pve-devel] [PATCH installer v6 13/36] auto-installer: add tests for answer file parsing

2024-04-17 Thread Aaron Lauterer
By matching the resulting json to be passed to the low level installer against known good ones. The environment info was gathered from one of our AMD Epyc Rome test servers to have a realistic starting point. Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer

Re: [pve-devel] [PATCH installer v5 00/36] add automated/unattended installation

2024-04-17 Thread Aaron Lauterer
a new v6 has been posted that includes the t-b and r-b tags as well as some smaller style fixes in the most recent patches https://lists.proxmox.com/pipermail/pve-devel/2024-April/063139.html On 2024-04-16 17:32, Aaron Lauterer wrote: patches until 31 got a [0,1] Tested-by: Christoph Heiss

[pve-devel] [PATCH installer v6 11/36] auto-installer: add utils

2024-04-17 Thread Aaron Lauterer
contains several utility structs and functions. For example: a simple pattern matcher that matches wildcards at the beginning or end of the filter. It currently uses a dedicated function (parse_answer) to generate the InstallConfig struct instead of a From implementation. This is because for now

[pve-devel] [PATCH installer v6 21/36] auto-installer: fetch: add gathering of system identifiers and restructure code

2024-04-17 Thread Aaron Lauterer
They will be used as payload when POSTing a request for an answer file. The idea is, that with this information, it should be possible to identify the system and generate a matching answer file on the fly. Many of these properties can also be found on the machine or packaging of the machine and

[pve-devel] [PATCH installer v6 22/36] auto-installer: helper: add subcommand to view indentifiers

2024-04-17 Thread Aaron Lauterer
It will collect the information from the current system and show the payload of identifiers that will be send. To avoid confusion, the subcommands for the device info and filter matching have been renamed. Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer

[pve-devel] [PATCH installer v6 28/36] common: add deserializer for FsType

2024-04-17 Thread Aaron Lauterer
Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-installer-common/Cargo.toml | 1 + proxmox-installer-common/src/options.rs | 10 ++--- proxmox-installer-common/src/setup.rs | 30 ++--- 3 files changed, 35

[pve-devel] [PATCH installer v6 14/36] auto-installer: add auto-installer binary

2024-04-17 Thread Aaron Lauterer
It expects the contents of an answer file via stdin. It will then be parsed and the JSON for the low level installer is generated. It then calls the low level installer directly. The output of the installaton progress is kept rather simple for now. If configured in the answer file, commands will

[pve-devel] [PATCH installer v6 33/36] auto-installer: utils: define ISO specified settings

2024-04-17 Thread Aaron Lauterer
These will be expected on the ISO itself and define the behavior of the automated installation. Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/src/utils.rs | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/proxmox-auto-installer/src/utils.rs

[pve-devel] [PATCH installer v6 09/36] auto-installer: add answer file definition

2024-04-17 Thread Aaron Lauterer
Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/Cargo.toml| 1 + proxmox-auto-installer/src/answer.rs | 248 +++ proxmox-auto-installer/src/lib.rs| 1 + 3 files changed, 250 insertions(+)

[pve-devel] [PATCH installer v6 27/36] low-level: write low level config to /tmp

2024-04-17 Thread Aaron Lauterer
This helps to know how the system was set up in steps after the installation. For example in debug mode or when using post commands in the automatic/unattended installation. Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-low-level-installer |

[pve-devel] [PATCH installer v6 25/36] control: update build depends for auto installer

2024-04-17 Thread Aaron Lauterer
Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- debian/control | 10 ++ 1 file changed, 10 insertions(+) diff --git a/debian/control b/debian/control index 3ca208b..1326400 100644 --- a/debian/control +++ b/debian/control @@ -8,10 +8,20 @@

[pve-devel] [PATCH installer v6 18/36] auto-installer: utils: make get_udev_index functions public

2024-04-17 Thread Aaron Lauterer
because we will need to access them directly in the future from a separate binary Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/src/utils.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git

[pve-devel] [PATCH installer v6 07/36] add auto-installer crate

2024-04-17 Thread Aaron Lauterer
Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- Cargo.toml| 1 + Makefile | 1 + proxmox-auto-installer/Cargo.toml | 10 ++ proxmox-auto-installer/src/lib.rs | 0 4 files changed, 12

[pve-devel] [PATCH installer v6 08/36] auto-installer: add dependencies

2024-04-17 Thread Aaron Lauterer
Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/Cargo.toml | 4 1 file changed, 4 insertions(+) diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index 75cfb2c..67218dd 100644 ---

[pve-devel] [PATCH installer v6 10/36] auto-installer: add struct to hold udev info

2024-04-17 Thread Aaron Lauterer
Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/src/lib.rs | 1 + proxmox-auto-installer/src/udevinfo.rs | 9 + 2 files changed, 10 insertions(+) create mode 100644 proxmox-auto-installer/src/udevinfo.rs diff --git

[pve-devel] [PATCH installer v6 01/36] tui: common: move InstallConfig struct to common crate

2024-04-17 Thread Aaron Lauterer
It describes the data structure expected by the low-level-installer. We do this so we can use it in more than the TUI installer, for example the planned auto installer. Make the members public so we can easily implement a custom From method for each dependent crate. Tested-by: Christoph Heiss

[pve-devel] [PATCH installer v6 05/36] common: options: add Deserialize trait

2024-04-17 Thread Aaron Lauterer
For the Enums that will be used to deserialize an answer file. Tested-by: Christoph Heiss Reviewed-by: Christoph Heiss Signed-off-by: Aaron Lauterer --- proxmox-installer-common/src/options.rs | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git

[pve-devel] [PATCH installer v6 24/36] auto-installer: fetch: add http plugin to fetch answer

2024-04-17 Thread Aaron Lauterer
This plugin will send a HTTP POST request with identifying sysinfo to fetch an answer file. The provided sysinfo can be used to identify the system and generate a matching answer file on demand. The URL to send the request to, can be defined in two ways. Via a custom DHCP option or a TXT record

[pve-devel] [PATCH installer v6 19/36] auto-installer: add proxmox-autoinst-helper tool

2024-04-17 Thread Aaron Lauterer
It can parse an answer file to check against syntax errors, test match filters against the current hardware and list properties of the current hardware to match against. Since this tool should be able to run outside of the installer environment, it does not rely on the device information provided

[pve-devel] [PATCH installer v6 15/36] auto-installer: add fetch answer binary

2024-04-17 Thread Aaron Lauterer
it is supposed to be run first and fetch an answer file. The initial implementation searches for a partition/filesystem called 'proxmoxinst' or 'PROXMOXINST' with an 'answer.toml' file in the root directory. Once it has an answer file, it will call the 'proxmox-auto-installer' and pipe in the

[pve-devel] [PATCH installer v6 36/36] autoinst-helper: add prepare-iso subcommand

2024-04-17 Thread Aaron Lauterer
This new subcommand makes it possible to prepare an ISO to use it for an automated installation. It is possible to control the behavior of the resulting automated ISO with optional parameters. If no target file is specified, the new ISO will be named with suffixes to indicate it as automated and

[pve-devel] [PATCH installer v6 34/36] fetch-answer: use ISO specified configurations

2024-04-17 Thread Aaron Lauterer
This patch switches the behavior to use the settings that can be specified in the ISO. This means, that it is possible to control how the answer file should be fetched: * auto - as usually, go through the options until one works (partition, http) * included - the answer file is included in the

[pve-devel] [PATCH installer v6 35/36] fetch-answer: dpcp: improve logging of steps taken

2024-04-17 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-fetch-answer/src/fetch_plugins/http.rs | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/proxmox-fetch-answer/src/fetch_plugins/http.rs b/proxmox-fetch-answer/src/fetch_plugins/http.rs index 4093131..cd3775f 100644 ---

[pve-devel] [PATCH installer v6 23/36] auto-installer: fetch: add http post utility module

2024-04-17 Thread Aaron Lauterer
It sends a http(s) POST request with the sysinfo as payload and expects an answer file in return. In order to handle non FQDN URLs (e.g. IP addresses) and self signed certificates, it can optionally take an SHA256 fingerprint of the certificate. This can of course also be used to pin a

  1   2   >