[pve-devel] [PATCH v3 13/30] auto-installer: add tests for answer file parsing

2024-03-28 Thread Aaron Lauterer
By matching the resulting json to be passed to the low level installer against known good ones. The environment info was gathered from one of our AMD Epyc Rome test servers to have a realistic starting point. Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/tests/parse-answer.rs | 102

[pve-devel] applied-series: [PATCH v2 pve-manager pve-docs 0/4] add optional WoL config options

2024-03-28 Thread Thomas Lamprecht
Am 26/03/2024 um 10:16 schrieb Christian Ebner: > For certain network setups the default values currently used to send > a wake on lan magic packet are not correct, e.g. it will get send via > the interface for which the default gateway is configured. > > This patches add optional configuration

[pve-devel] [PATCH v3 23/30] auto-installer: fetch: add http plugin to fetch answer

2024-03-28 Thread Aaron Lauterer
This plugin will send a HTTP POST request with identifying sysinfo to fetch an answer file. The provided sysinfo can be used to identify the system and generate a matching answer file on demand. The URL to send the request to, can be defined in two ways. Via a custom DHCP option or a TXT record

[pve-devel] [PATCH v3 15/30] auto-installer: add fetch answer binary

2024-03-28 Thread Aaron Lauterer
it is supposed to be run first and fetch an answer file. The initial implementation searches for a partition/filesystem called 'proxmoxinst' or 'PROXMOXINST' with an 'answer.toml' file in the root directory. Once it has an answer file, it will call the 'proxmox-auto-installer' and pipe in the

[pve-devel] [PATCH v3 20/30] auto-installer: fetch: add gathering of system identifiers and restructure code

2024-03-28 Thread Aaron Lauterer
They will be used as payload when POSTing a request for an answer file. The idea is, that with this information, it should be possible to identify the system and generate a matching answer file on the fly. Many of these properties can also be found on the machine or packaging of the machine and

Re: [pve-devel] [PATCH v3 00/30] add automated/unattended installation

2024-03-28 Thread Aaron Lauterer
sorry, forgot to set the subject: pve-installer ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v3 30/30] add proxmox-chroot utility

2024-03-28 Thread Aaron Lauterer
it is meant as a helper utility to prepare an installation for chroot and clean up afterwards It tries to determine the used FS from the previous installation, will do what is necessary to mount/import the root FS to /target. It then will set up all bind mounts. Signed-off-by: Aaron Lauterer

[pve-devel] [PATCH v3 27/30] common: add deserializer for FsType

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-installer-common/Cargo.toml | 1 + proxmox-installer-common/src/options.rs | 10 -- proxmox-installer-common/src/setup.rs | 43 +++-- 3 files changed, 48 insertions(+), 6 deletions(-) diff --git

[pve-devel] [PATCH v3 21/30] auto-installer: helper: add subcommand to view indentifiers

2024-03-28 Thread Aaron Lauterer
It will collect the information from the current system and show the payload of identifiers that will be send. To avoid confusion, the subcommands for the device info and filter matching have been renamed. Signed-off-by: Aaron Lauterer --- .../src/bin/proxmox-autoinst-helper.rs| 54

[pve-devel] [PATCH v3 28/30] common: skip target_hd when deserializing InstallConfig

2024-03-28 Thread Aaron Lauterer
as only the 'path' property is serialized -> deserialization is problematic. The information would be present in the 'run-env-info-json', but for now there is no need for it in any code that deserializes the low-level config. Therefore we are currently skipping it on deserialization If we need it

[pve-devel] [PATCH v3 26/30] low-level: write low level config to /tmp

2024-03-28 Thread Aaron Lauterer
This helps to know how the system was set up in steps after the installation. For example in debug mode or when using post commands in the automatic/unattended installation. Signed-off-by: Aaron Lauterer --- proxmox-low-level-installer | 1 + 1 file changed, 1 insertion(+) diff --git

[pve-devel] [PATCH v3 24/30] control: update build depends for auto installer

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- debian/control | 10 ++ 1 file changed, 10 insertions(+) diff --git a/debian/control b/debian/control index 3ca208b..1326400 100644 --- a/debian/control +++ b/debian/control @@ -8,10 +8,20 @@ Build-Depends: cargo:native, libgtk3-perl,

[pve-devel] [PATCH v3 12/30] auto-installer: add simple logging

2024-03-28 Thread Aaron Lauterer
Log to stdout and the file the binary needs to set up. This is a first variant. By using the log crate macros we can change that in the future without too much effort. Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/Cargo.toml | 2 ++ proxmox-auto-installer/src/lib.rs | 1 +

[pve-devel] [PATCH v3 06/30] low-level: add dump-udev command

2024-03-28 Thread Aaron Lauterer
Fetches UDEV device properties prepended with 'E:' for NICs and disks. The result is stored in its own JSON file. This information is needed to filter for specific devices. Mainly for the auto-installer for now. Signed-off-by: Aaron Lauterer --- Proxmox/Makefile| 1 +

[pve-devel] [PATCH v3 08/30] auto-installer: add dependencies

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/Cargo.toml | 4 1 file changed, 4 insertions(+) diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index 75cfb2c..67218dd 100644 --- a/proxmox-auto-installer/Cargo.toml +++

[pve-devel] [PATCH v3 05/30] common: options: add Deserialize trait

2024-03-28 Thread Aaron Lauterer
For the Enums that will be used to deserialize an answer file. Signed-off-by: Aaron Lauterer --- proxmox-installer-common/src/options.rs | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/proxmox-installer-common/src/options.rs

[pve-devel] [PATCH v3 02/30] common: make InstallZfsOption members public

2024-03-28 Thread Aaron Lauterer
as they will be used directly by the auto installer Signed-off-by: Aaron Lauterer --- proxmox-installer-common/src/setup.rs | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs index

[pve-devel] [PATCH v3 25/30] auto installer: factor out fetch-answer and autoinst-helper

2024-03-28 Thread Aaron Lauterer
Putting proxmox-fetch-answer into it's own crate, will keep the use of OpenSSL localized to where we need it. Otherwise building other binaries will always depend on OpenSSL as well, even without actually needing it. Having a dedicated crate for the proxmox-autoinst-helper should make it easier

[pve-devel] [PATCH v3 19/30] auto-installer: add proxmox-autoinst-helper tool

2024-03-28 Thread Aaron Lauterer
It can parse an answer file to check against syntax errors, test match filters against the current hardware and list properties of the current hardware to match against. Since this tool should be able to run outside of the installer environment, it does not rely on the device information provided

[pve-devel] [PATCH v3 14/30] auto-installer: add auto-installer binary

2024-03-28 Thread Aaron Lauterer
It expects the contents of an answer file via stdin. It will then be parsed and the JSON for the low level installer is generated. It then calls the low level installer directly. The output of the installaton progress is kept rather simple for now. If configured in the answer file, commands will

[pve-devel] [PATCH v3 11/30] auto-installer: add utils

2024-03-28 Thread Aaron Lauterer
contains several utility structs and functions. For example: a simple pattern matcher that matches wildcards at the beginning or end of the filter. It currently uses a dedicated function (parse_answer) to generate the InstallConfig struct instead of a From implementation. This is because for now

[pve-devel] [PATCH v3 22/30] auto-installer: fetch: add http post utility module

2024-03-28 Thread Aaron Lauterer
It sends a http(s) POST request with the sysinfo as payload and expects an answer file in return. In order to handle non FQDN URLs (e.g. IP addresses) and self signed certificates, it can optionally take an SHA256 fingerprint of the certificate. This can of course also be used to pin a

[pve-devel] [PATCH v3 00/30] add automated/unattended installation

2024-03-28 Thread Aaron Lauterer
This patch series adds the possibility to do an automated / unattended installation of Proxmox VE. The overall idea is that we will have a dedicated ISO for the unattended installation. It should be configured in such a way that it will start the installation without any user interaction. Though

[pve-devel] [PATCH v3 09/30] auto-installer: add answer file definition

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/Cargo.toml| 1 + proxmox-auto-installer/src/answer.rs | 256 +++ proxmox-auto-installer/src/lib.rs| 1 + 3 files changed, 258 insertions(+) create mode 100644 proxmox-auto-installer/src/answer.rs diff

[pve-devel] [PATCH v3 17/30] auto-installer: use glob crate for pattern matching

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/Cargo.toml | 1 + proxmox-auto-installer/src/utils.rs | 46 +++-- 2 files changed, 18 insertions(+), 29 deletions(-) diff --git a/proxmox-auto-installer/Cargo.toml b/proxmox-auto-installer/Cargo.toml index

[pve-devel] [PATCH v3 29/30] common: add Display trait to ProxmoxProduct

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-installer-common/src/setup.rs | 10 ++ 1 file changed, 10 insertions(+) diff --git a/proxmox-installer-common/src/setup.rs b/proxmox-installer-common/src/setup.rs index f9f9e82..19113e1 100644 --- a/proxmox-installer-common/src/setup.rs +++

[pve-devel] [PATCH v3 18/30] auto-installer: utils: make get_udev_index functions public

2024-03-28 Thread Aaron Lauterer
because we will need to access them directly in the future from a separate binary Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/src/utils.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proxmox-auto-installer/src/utils.rs

[pve-devel] [PATCH v3 16/30] unconfigured: add proxauto as option to start auto installer

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- unconfigured.sh | 8 1 file changed, 8 insertions(+) diff --git a/unconfigured.sh b/unconfigured.sh index 2b371f0..f02336a 100755 --- a/unconfigured.sh +++ b/unconfigured.sh @@ -5,6 +5,7 @@ trap "err_reboot" ERR # NOTE: we nowadays get exec'd by the

[pve-devel] [PATCH v3 01/30] tui: common: move InstallConfig struct to common crate

2024-03-28 Thread Aaron Lauterer
It describes the data structure expected by the low-level-installer. We do this so we can use it in more than the TUI installer, for example the planned auto installer. Make the members public so we can easily implement a custom From method for each dependent crate. Signed-off-by: Aaron Lauterer

[pve-devel] [PATCH v3 07/30] add auto-installer crate

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- Cargo.toml| 1 + Makefile | 1 + proxmox-auto-installer/Cargo.toml | 10 ++ proxmox-auto-installer/src/lib.rs | 0 4 files changed, 12 insertions(+) create mode 100644 proxmox-auto-installer/Cargo.toml

[pve-devel] [PATCH v3 10/30] auto-installer: add struct to hold udev info

2024-03-28 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer --- proxmox-auto-installer/src/lib.rs | 1 + proxmox-auto-installer/src/udevinfo.rs | 9 + 2 files changed, 10 insertions(+) create mode 100644 proxmox-auto-installer/src/udevinfo.rs diff --git a/proxmox-auto-installer/src/lib.rs

[pve-devel] [PATCH v3 03/30] common: tui: use BTreeMap for predictable ordering

2024-03-28 Thread Aaron Lauterer
necessary for the disk selection and network interfaces maps to have tests with results that can be compared without much additional effort. Signed-off-by: Aaron Lauterer --- proxmox-installer-common/src/setup.rs | 8 proxmox-tui-installer/src/options.rs | 4 ++--

[pve-devel] [PATCH v3 04/30] common: utils: add deserializer for CidrAddress

2024-03-28 Thread Aaron Lauterer
so it can be deserialized from a string Signed-off-by: Aaron Lauterer --- proxmox-installer-common/src/utils.rs | 11 +++ 1 file changed, 11 insertions(+) diff --git a/proxmox-installer-common/src/utils.rs b/proxmox-installer-common/src/utils.rs index 36b1d53..f6521eb 100644 ---

Re: [pve-devel] [PATCH docs] system-requirements: mention that SSDs with PLP should be used

2024-03-28 Thread Aaron Lauterer
Should we stick with v1 or v2? V1 is definitely more strongly worded than v2. v1: +**·SSDs·with·Power-Loss-Protection·(PLP)·are·recommended·for·good·performance. +··Using·consumer·SSDs·is·discouraged. vs v2: +**·Enterprise·grade·SSDs·are·recommended·for·good·performance.·Check·for·power

[pve-devel] applied: [PATCH docs] installation: fix volume group name

2024-03-28 Thread Aaron Lauterer
On 2024-03-28 11:45, Christoph Heiss wrote: This slipped through while taking the wording from PMG. Fixes: 67d2d94 ("installation: lvm-options: improve & align wording with pmg-docs") Signed-off-by: Christoph Heiss --- Probably have been staring at the docs too long to not notice this

Re: [pve-devel] [PATCH zfsonlinux] fix #4835: order zfs-import@ before -cache/-scan

2024-03-28 Thread Stoiko Ivanov
LGTM - Thanks for the idea and patch! gave it a spin by creating a pool through the GUI, rebooting, seeing the spurious warning, installing a patched version, rebooting multiple times (no warning), adding yet another pool and enabling zfs-import-scan.service (did not manage to boot without

[pve-devel] [PATCH docs] installation: fix volume group name

2024-03-28 Thread Christoph Heiss
This slipped through while taking the wording from PMG. Fixes: 67d2d94 ("installation: lvm-options: improve & align wording with pmg-docs") Signed-off-by: Christoph Heiss --- Probably have been staring at the docs too long to not notice this earlier .. pve-installation.adoc | 2 +- 1 file

Re: [pve-devel] [PATCH zfsonlinux] fix #4835: order zfs-import@ before -cache/-scan

2024-03-28 Thread Fabian Grünbichler
missed the subjectprefix, this is for ZFS ;) thanks @Stoiko for pointing it out off-list On March 28, 2024 10:41 am, Fabian Grünbichler wrote: > this should fix failures of the template instances because either of the two > other import services picked up the pool in question first. > >

[pve-devel] [PATCH] fix #4835: order zfs-import@ before -cache/-scan

2024-03-28 Thread Fabian Grünbichler
this should fix failures of the template instances because either of the two other import services picked up the pool in question first. Signed-off-by: Fabian Grünbichler --- ...dd-systemd-unit-for-importing-specific-pools.patch | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-)