[pve-devel] [PATCH manager v2 1/6] ui: fix special 'import' icon for non-esxi storages
we only want to show that icon in the tree when the storage is solely
used for importing, not when it's just one of several content types.
Signed-off-by: Dominik Csapak
Reviewed-by: Fiona Ebner
---
www/manager6/Utils.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 54aa8bac..84d5eef1 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1244,7 +1244,7 @@ Ext.define('PVE.Utils', {
// templates
objType = 'template';
status = type;
- } else if (type === 'storage' && record.content.indexOf('import') !==
-1) {
+ } else if (type === 'storage' && record.content === 'import') {
return 'fa fa-cloud-download';
} else {
// everything else
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH storage v2 02/10] plugin: dir: implement import content type
in DirPlugin and not Plugin (because of cyclic dependency of
Plugin -> OVF -> Storage -> Plugin otherwise)
only ovf is currently supported (though ova will be shown in import
listing), expects the files to not be in a subdir, and adjacent to the
ovf file.
Signed-off-by: Dominik Csapak
---
changes from v1:
* only allow 'safe characters' in the relative filepath
* don't return 'image' type for vmdk/qcow2/raw files, instead use
'import' with a file_format, makes much of fionas review not necessary
* incorporated fionas suggestions
* added failure test
src/PVE/GuestImport/OVF.pm | 3 +++
src/PVE/Storage.pm | 8 +++
src/PVE/Storage/DirPlugin.pm | 36 +-
src/PVE/Storage/Plugin.pm | 13 ++-
src/test/parse_volname_test.pm | 18 +++
src/test/path_to_volume_id_test.pm | 21 +
6 files changed, 97 insertions(+), 2 deletions(-)
diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm
index 055ebf5..0eb5e9c 100644
--- a/src/PVE/GuestImport/OVF.pm
+++ b/src/PVE/GuestImport/OVF.pm
@@ -222,6 +222,8 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType",
$controller_id);
}
($backing_file_path) = $backing_file_path =~ m|^(/.*)|; # untaint
+ ($filepath) = $filepath =~ m|^(${PVE::Storage::SAFE_CHAR_CLASS_RE}+)$|;
# untaint & check no sub/parent dirs
+ die "invalid path\n" if !$filepath;
my $virtual_size = PVE::Storage::file_size_info($backing_file_path);
die "error parsing $backing_file_path, cannot determine file size\n"
@@ -231,6 +233,7 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType",
$controller_id);
disk_address => $pve_disk_address,
backing_file => $backing_file_path,
virtual_size => $virtual_size
+ relative_path => $filepath,
};
push @disks, $pve_disk;
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index f19a115..863dbdb 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -114,6 +114,10 @@ our $VZTMPL_EXT_RE_1 = qr/\.tar\.(gz|xz|zst)/i;
our $BACKUP_EXT_RE_2 =
qr/\.(tgz|(?:tar|vma)(?:\.(${\PVE::Storage::Plugin::COMPRESSOR_RE}))?)/;
+our $IMPORT_EXT_RE_1 = qr/\.(ov[af])/;
+
+our $SAFE_CHAR_CLASS_RE = qr/[a-zA-Z0-9\-\.\+\=\_]/;
+
# FIXME remove with PVE 8.0, add versioned breaks for pve-manager
our $vztmpl_extension_re = $VZTMPL_EXT_RE_1;
@@ -612,6 +616,7 @@ sub path_to_volume_id {
my $backupdir = $plugin->get_subdir($scfg, 'backup');
my $privatedir = $plugin->get_subdir($scfg, 'rootdir');
my $snippetsdir = $plugin->get_subdir($scfg, 'snippets');
+ my $importdir = $plugin->get_subdir($scfg, 'import');
if ($path =~ m!^$imagedir/(\d+)/([^/\s]+)$!) {
my $vmid = $1;
@@ -640,6 +645,9 @@ sub path_to_volume_id {
} elsif ($path =~ m!^$snippetsdir/([^/]+)$!) {
my $name = $1;
return ('snippets', "$sid:snippets/$name");
+ } elsif ($path =~ m!^$importdir/([^/]+${IMPORT_EXT_RE_1})$!) {
+ my $name = $1;
+ return ('import', "$sid:import/$name");
}
}
diff --git a/src/PVE/Storage/DirPlugin.pm b/src/PVE/Storage/DirPlugin.pm
index 2efa8d5..6f9c2b9 100644
--- a/src/PVE/Storage/DirPlugin.pm
+++ b/src/PVE/Storage/DirPlugin.pm
@@ -10,6 +10,7 @@ use IO::File;
use POSIX;
use PVE::Storage::Plugin;
+use PVE::GuestImport::OVF;
use PVE::JSONSchema qw(get_standard_option);
use base qw(PVE::Storage::Plugin);
@@ -22,7 +23,7 @@ sub type {
sub plugindata {
return {
- content => [ { images => 1, rootdir => 1, vztmpl => 1, iso => 1, backup
=> 1, snippets => 1, none => 1 },
+ content => [ { images => 1, rootdir => 1, vztmpl => 1, iso => 1, backup
=> 1, snippets => 1, none => 1, import => 1 },
{ images => 1, rootdir => 1 }],
format => [ { raw => 1, qcow2 => 1, vmdk => 1, subvol => 1 } , 'raw' ],
};
@@ -247,4 +248,37 @@ sub check_config {
return $opts;
}
+sub get_import_metadata {
+my ($class, $scfg, $volname, $storeid) = @_;
+
+my ($vtype, $name, undef, undef, undef, undef, $fmt) =
$class->parse_volname($volname);
+die "invalid content type '$vtype'\n" if $vtype ne 'import';
+die "invalid format\n" if defined($fmt);
+
+# NOTE: all types of warnings must be added to the return schema of the
import-metadata API endpoint
+my $warnings = [];
+
+my $path = $class->path($scfg, $volname, $storeid, undef);
+my $res = PVE::GuestImport::OVF::parse_ovf($path);
+my $disks = {};
+for my $disk ($res->{disks}->@*) {
+ my $id = $disk->{disk_address};
+ my $size = $disk->{virtual_size};
+ my $path = $disk->{relative_path};
+ $disks->{$id} = {
+ volid => "$storeid:import/$path",
+ defined($size) ? (size => $size) : (),
+ };
+}
+
+return {
+ type => 'vm',
+ source => $volname,
+ 'create-args' => $res
[pve-devel] [PATCH storage/qemu-server/manager v2] implement ova/ovf import for file based storages
This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware from vmware exported files). I opted to move the OVF.pm to pve-storage, since there is no real other place where we could put it. I put it in a new module 'GuestImport' We now extract the images into either a given target storage or in the import storage in the 'images' dir so accidentally left over images are discoverable by the ui/cli. changes from v1: * move ovf code to GuestImport * move extract/checking code to GuestImport * don't return 'image' types from import volumes * use allow 'safe' characters for filenames of ova/ovfs and inside * check for non-regular files (e.g. symlinks) after extraction * add new 'import-extraction-storage' for import * rename panel in gui for directory storages * typo fixes * and probably more, see the individual patches for details pve-storage: Dominik Csapak (10): copy OVF.pm from qemu-server plugin: dir: implement import content type plugin: dir: handle ova files for import ovf: implement parsing the ostype ovf: implement parsing out firmware type ovf: implement rudimentary boot order ovf: implement parsing nics api: allow ova upload/download plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs add 'import' content type to 'check_volume_access' src/PVE/API2/Storage/Status.pm| 19 +- src/PVE/GuestImport.pm| 99 + src/PVE/GuestImport/Makefile | 3 + src/PVE/GuestImport/OVF.pm| 383 ++ src/PVE/Makefile | 2 + src/PVE/Storage.pm| 21 +- src/PVE/Storage/BTRFSPlugin.pm| 5 + src/PVE/Storage/CIFSPlugin.pm | 6 +- src/PVE/Storage/CephFSPlugin.pm | 6 +- src/PVE/Storage/DirPlugin.pm | 52 ++- src/PVE/Storage/GlusterfsPlugin.pm| 6 +- src/PVE/Storage/Makefile | 1 + src/PVE/Storage/NFSPlugin.pm | 6 +- src/PVE/Storage/Plugin.pm | 18 +- src/test/Makefile | 5 +- src/test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 0 -> 65536 bytes src/test/ovf_manifests/Win10-Liz.ovf | 142 +++ .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 143 +++ .../ovf_manifests/Win_2008_R2_two-disks.ovf | 145 +++ src/test/ovf_manifests/disk1.vmdk | Bin 0 -> 65536 bytes src/test/ovf_manifests/disk2.vmdk | Bin 0 -> 65536 bytes src/test/parse_volname_test.pm| 33 ++ src/test/path_to_volume_id_test.pm| 21 + src/test/run_ovf_tests.pl | 85 24 files changed, 1189 insertions(+), 12 deletions(-) create mode 100644 src/PVE/GuestImport.pm create mode 100644 src/PVE/GuestImport/Makefile create mode 100644 src/PVE/GuestImport/OVF.pm create mode 100644 src/test/ovf_manifests/Win10-Liz-disk1.vmdk create mode 100755 src/test/ovf_manifests/Win10-Liz.ovf create mode 100755 src/test/ovf_manifests/Win10-Liz_no_default_ns.ovf create mode 100755 src/test/ovf_manifests/Win_2008_R2_two-disks.ovf create mode 100644 src/test/ovf_manifests/disk1.vmdk create mode 100644 src/test/ovf_manifests/disk2.vmdk create mode 100755 src/test/run_ovf_tests.pl qemu-server: Dominik Csapak (4): api: delete unused OVF.pm use OVF from Storage api: create: implement extracting disks when needed for import-from api: create: add 'import-extraction-storage' parameter PVE/API2/Qemu.pm | 92 ++- PVE/API2/Qemu/Makefile| 2 +- PVE/API2/Qemu/OVF.pm | 53 PVE/CLI/qm.pm | 4 +- PVE/QemuServer.pm | 5 +- PVE/QemuServer/Helpers.pm | 10 + PVE/QemuServer/Makefile | 1 - PVE/QemuServer/OVF.pm | 242 -- test/Makefile | 5 +- test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 65536 -> 0 bytes test/ovf_manifests/Win10-Liz.ovf | 142 -- .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 -- test/ovf_manifests/Win_2008_R2_two-disks.ovf | 145 --- test/ovf_manifests/disk1.vmdk | Bin 65536 -> 0 bytes test/ovf_manifests/disk2.vmdk | Bin 65536 -> 0 bytes test/run_ovf_tests.pl | 71 - 16 files changed, 96 insertions(+), 818 deletions(-) delete mode 100644 PVE/API2/Qemu/OVF.pm delete mode 100644 PVE/QemuServer/OVF.pm delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk delete mode 100755 test/ovf_manifests/Win10-Liz.ovf delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf delete mode
Re: [pve-devel] [PATCH manager 17/19] gitignore: ignore any test artifacts
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > Signed-off-by: Lukas Wagner > --- > .gitignore | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/.gitignore b/.gitignore > index e8d1eb27..48975d55 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -9,3 +9,5 @@ dest/ > /www/mobile/pvemanager-mobile.js > /www/touch/touch-[0-9]*/ > /pve-manager-[0-9]*/ > +/test/.mocked* > +/test/*.log.tmp Nit: not exactly the same as in the clean target of the Makefile rm -rf *~ .mocked_* *.tmp but shouldn't matter. ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-docs v4 5/5] firewall: add documentation for proxmox-firewall
Add a section that explains how to use the new nftables-based
proxmox-firewall.
Signed-off-by: Stefan Hanreich
---
pve-firewall.adoc | 181 ++
1 file changed, 181 insertions(+)
diff --git a/pve-firewall.adoc b/pve-firewall.adoc
index a5e40f9..9fb4e46 100644
--- a/pve-firewall.adoc
+++ b/pve-firewall.adoc
@@ -379,6 +379,7 @@ discovery protocol to work.
+[[pve_firewall_services_commands]]
Services and Commands
-
@@ -637,6 +638,186 @@ Ports used by {pve}
* corosync cluster traffic: 5405-5412 UDP
* live migration (VM memory and local-disk data): 6-60050 (TCP)
+
+nftables
+
+
+As an alternative to `pve-firewall` we offer `proxmox-firewall`, which is an
+implementation of the Proxmox VE firewall based on the newer
+https://wiki.nftables.org/wiki-nftables/index.php/What_is_nftables%3F[nftables]
+rather than iptables.
+
+WARNING: `proxmox-firewall` is currently in tech preview. There might be bugs
or
+incompatibilies with the original firewall. It is currently not suited for
+production use.
+
+This implementation uses the same configuration files and configuration format,
+so you can use your old configuration when switching. It provides the exact
same
+functionality with a few exceptions:
+
+* REJECT is currently not possible for guest traffic (traffic will instead be
+ dropped).
+* Using the `NDP`, `Router Advertisement` or `DHCP` options will *always*
create
+ firewall rules, irregardless of your default policy.
+* firewall rules for guests are evaluated even for connections that have
+ conntrack table entries.
+
+
+Installation and Usage
+~~
+
+Install the `proxmox-firewall` package:
+
+
+apt install proxmox-firewall
+
+
+Enable the nftables backend via the Web UI on your hosts (Host > Firewall >
+Options > nftables), or by enabling it in the configuration file for your hosts
+(`/etc/pve/nodes//host.fw`):
+
+
+[OPTIONS]
+
+nftables: 1
+
+
+NOTE: After enabling/disabling `proxmox-firewall`, all running VMs and
+containers need to be restarted for the old/new firewall to work properly.
+
+After setting the `nftables` configuration key, the new `proxmox-firewall`
+service will take over. You can check if the new service is working by
+checking the systemctl status of `proxmox-firewall`:
+
+
+systemctl status proxmox-firewall
+
+
+You can also examine the generated ruleset. You can find more information about
+this in the section xref:pve_firewall_nft_helpful_commands[Helpful Commands].
+You should also check whether `pve-firewall` is no longer generating iptables
+rules, you can find the respective commands in the
+xref:pve_firewall_services_commands[Services and Commands] section.
+
+Switching back to the old firewall can be done by simply setting the
+configuration value back to 0 / No.
+
+Usage
+~
+
+`proxmox-firewall` will create two tables that are managed by the
+`proxmox-firewall` service: `proxmox-firewall` and `proxmox-firewall-guests`.
If
+you want to create custom rules that live outside the Proxmox VE firewall
+configuration you can create your own tables to manage your custom firewall
+rules. `proxmox-firewall` will only touch the tables it generates, so you can
+easily extend and modify the behavior of the `proxmox-firewall` by adding your
+own tables.
+
+Instead of using the `pve-firewall` command, the nftables-based firewall uses
+`proxmox-firewall`. It is a systemd service, so you can start and stop it via
+`systemctl`:
+
+
+systemctl start proxmox-firewall
+systemctl stop proxmox-firewall
+
+
+Stopping the firewall service will remove all generated rules.
+
+To query the status of the firewall, you can query the status of the systemctl
+service:
+
+
+systemctl status proxmox-firewall
+
+
+
+[[pve_firewall_nft_helpful_commands]]
+Helpful Commands
+
+You can check the generated ruleset via the following command:
+
+
+nft list ruleset
+
+
+If you want to debug `proxmox-firewall` you can simply run the daemon in
+foreground with the `RUST_LOG` environment variable set to `trace`. This should
+provide you with detailed debugging output:
+
+
+RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall
+
+
+You can also edit the systemctl service if you want to have detailed output for
+your firewall daemon:
+
+
+systemctl edit proxmox-firewall
+
+
+Then you need to add the override for the `RUST_LOG` environment variable:
+
+
+[Service]
+Environment="RUST_LOG=trace"
+
+
+This will generate a large amount of logs very quickly, so only use this for
+debugging purposes. Other, less verbose, log levels are `info` and `debug`.
+
+Running in foreground writes the log output to STDERR, so you can redirect it
+with the following command (e.g. for submitting logs to the community forum):
+
+
+RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall 2>
firewall_log_$(hostname).txt
+
+
+It can be hel
[pve-devel] [PATCH pve-manager v4 4/5] firewall: expose configuration option for new nftables firewall
Signed-off-by: Stefan Hanreich
---
www/manager6/grid/FirewallOptions.js | 1 +
1 file changed, 1 insertion(+)
diff --git a/www/manager6/grid/FirewallOptions.js
b/www/manager6/grid/FirewallOptions.js
index 0ac9979c4..6aacb47be 100644
--- a/www/manager6/grid/FirewallOptions.js
+++ b/www/manager6/grid/FirewallOptions.js
@@ -83,6 +83,7 @@ Ext.define('PVE.FirewallOptions', {
add_log_row('log_level_out');
add_log_row('tcp_flags_log_level', 120);
add_log_row('smurf_log_level');
+ add_boolean_row('nftables', gettext('nftables (tech preview)'), 0);
} else if (me.fwtype === 'vm') {
me.rows.enable = {
required: true,
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-firewall v4 3/5] add configuration option for new nftables firewall
Introduces new nftables configuration option that en/disables the new
nftables firewall.
pve-firewall reads this option and only generates iptables rules when
nftables is set to `0` or if the proxmox-firewall package is not
installed at all. Conversely, proxmox-firewall only generates rules
when the option is set to `1`.
Signed-off-by: Stefan Hanreich
---
This looks a bit awkward, but I wanted to avoid having to re-parse the
configuration when calling from pve-firewall but also avoid having to
load the config manually when calling from qemu-server / pve-container
src/PVE/Firewall.pm | 41 -
1 file changed, 32 insertions(+), 9 deletions(-)
diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 81a8798..21eb5fc 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -1408,6 +1408,12 @@ our $host_option_properties = {
default => 0,
optional => 1
},
+nftables => {
+ description => "Enable nftables based firewall",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+},
};
our $vm_option_properties = {
@@ -2929,7 +2935,7 @@ sub parse_hostfw_option {
my $loglevels = "emerg|alert|crit|err|warning|notice|info|debug|nolog";
-if ($line =~
m/^(enable|nosmurfs|tcpflags|ndp|log_nf_conntrack|nf_conntrack_allow_invalid|protection_synflood):\s*(0|1)\s*$/i)
{
+if ($line =~
m/^(enable|nosmurfs|tcpflags|ndp|log_nf_conntrack|nf_conntrack_allow_invalid|protection_synflood|nftables):\s*(0|1)\s*$/i)
{
$opt = lc($1);
$value = int($2);
} elsif ($line =~
m/^(log_level_in|log_level_out|tcp_flags_log_level|smurf_log_level):\s*(($loglevels)\s*)?$/i)
{
@@ -4673,12 +4679,30 @@ sub remove_pvefw_chains_ebtables {
ebtables_restore_cmdlist(get_ebtables_cmdlist({}));
}
-sub init {
-my $cluster_conf = load_clusterfw_conf();
-my $cluster_options = $cluster_conf->{options};
-my $enable = $cluster_options->{enable};
+sub is_nftables {
+my ($cluster_conf, $host_conf) = @_;
+
+if (!-x "/usr/libexec/proxmox/proxmox-firewall") {
+ return 0;
+}
+
+$cluster_conf = load_clusterfw_conf() if !defined($cluster_conf);
+$host_conf = load_hostfw_conf($cluster_conf) if !defined($host_conf);
-return if !$enable;
+return $host_conf->{options}->{nftables};
+}
+
+sub is_enabled {
+my ($cluster_conf, $host_conf) = @_;
+
+$cluster_conf = load_clusterfw_conf() if !defined($cluster_conf);
+$host_conf = load_hostfw_conf($cluster_conf) if !defined($host_conf);
+
+return $cluster_conf->{options}->{enable} && !is_nftables($cluster_conf,
$host_conf);
+}
+
+sub init {
+return if !is_enabled();
# load required modules here
}
@@ -4687,14 +4711,13 @@ sub update {
my $code = sub {
my $cluster_conf = load_clusterfw_conf();
- my $cluster_options = $cluster_conf->{options};
+ my $hostfw_conf = load_hostfw_conf($cluster_conf);
- if (!$cluster_options->{enable}) {
+ if (!is_enabled($cluster_conf, $hostfw_conf)) {
PVE::Firewall::remove_pvefw_chains();
return;
}
- my $hostfw_conf = load_hostfw_conf($cluster_conf);
my ($ruleset, $ipset_ruleset, $rulesetv6, $ebtables_ruleset) =
compile($cluster_conf, $hostfw_conf);
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH container/docs/firewall/manager/qemu-server v4 0/5] proxmox firewall nftables
This patch series contains the remaining patches that are necessary for proxmox-firewall to work. It adds documentation as well as changes how firewall-bridges are created when proxmox-firewall is activated. It also patches pve-firewall to not generate rules when proxmox-firewall is active. Dependencies: * qemu-server, pve-container & pve-manager depend on a bump of pve-firewall Changes from v3 -> v4: * additionally check for the existence of proxmox-firewall bin * extracted checks into helper functions * update docs to reflect the changes in behavior (omitted description & changes only relevant for the firewall itself) qemu-server: Stefan Hanreich (1): firewall: add handling for new nft firewall vm-network-scripts/pve-bridge | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) pve-container: Stefan Hanreich (1): firewall: add handling for new nft firewall src/PVE/LXC.pm | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) pve-firewall: Stefan Hanreich (1): add configuration option for new nftables firewall src/PVE/Firewall.pm | 41 - 1 file changed, 32 insertions(+), 9 deletions(-) pve-manager: Stefan Hanreich (1): firewall: expose configuration option for new nftables firewall www/manager6/grid/FirewallOptions.js | 1 + 1 file changed, 1 insertion(+) pve-docs: Stefan Hanreich (1): firewall: add documentation for proxmox-firewall pve-firewall.adoc | 181 ++ 1 file changed, 181 insertions(+) Summary over all repositories: 5 files changed, 224 insertions(+), 13 deletions(-) -- Generated by git-murpp 0.6.0 ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH qemu-server v4 1/5] firewall: add handling for new nft firewall
When the nftables firewall is enabled, we do not need to create
firewall bridges.
Signed-off-by: Stefan Hanreich
---
vm-network-scripts/pve-bridge | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/vm-network-scripts/pve-bridge b/vm-network-scripts/pve-bridge
index 85997a0..fe5a702 100755
--- a/vm-network-scripts/pve-bridge
+++ b/vm-network-scripts/pve-bridge
@@ -6,6 +6,7 @@ use warnings;
use PVE::QemuServer;
use PVE::Tools qw(run_command);
use PVE::Network;
+use PVE::Firewall;
my $have_sdn;
eval {
@@ -44,13 +45,15 @@ die "unable to get network config '$netid'\n"
my $net = PVE::QemuServer::parse_net($netconf);
die "unable to parse network config '$netid'\n" if !$net;
+my $firewall = $net->{firewall} && !PVE::Firewall::is_nftables();
+
if ($have_sdn) {
PVE::Network::SDN::Vnets::add_dhcp_mapping($net->{bridge},
$net->{macaddr}, $vmid, $conf->{name});
PVE::Network::SDN::Zones::tap_create($iface, $net->{bridge});
-PVE::Network::SDN::Zones::tap_plug($iface, $net->{bridge}, $net->{tag},
$net->{firewall}, $net->{trunks}, $net->{rate});
+PVE::Network::SDN::Zones::tap_plug($iface, $net->{bridge}, $net->{tag},
$firewall, $net->{trunks}, $net->{rate});
} else {
PVE::Network::tap_create($iface, $net->{bridge});
-PVE::Network::tap_plug($iface, $net->{bridge}, $net->{tag},
$net->{firewall}, $net->{trunks}, $net->{rate});
+PVE::Network::tap_plug($iface, $net->{bridge}, $net->{tag}, $firewall,
$net->{trunks}, $net->{rate});
}
exit 0;
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-container v4 2/5] firewall: add handling for new nft firewall
When the nftables firewall is enabled, we do not need to create
firewall bridges.
Signed-off-by: Stefan Hanreich
---
src/PVE/LXC.pm | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 400cf4f..44f5ccf 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -18,6 +18,7 @@ use PVE::AccessControl;
use PVE::CGroup;
use PVE::CpuSet;
use PVE::Exception qw(raise_perm_exc);
+use PVE::Firewall;
use PVE::GuestHelpers qw(check_vnet_access safe_string_ne safe_num_ne
safe_boolean_ne);
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
@@ -946,8 +947,10 @@ sub net_tap_plug : prototype($$) {
return;
}
-my ($bridge, $tag, $firewall, $trunks, $rate, $hwaddr) =
- $net->@{'bridge', 'tag', 'firewall', 'trunks', 'rate', 'hwaddr'};
+my ($bridge, $tag, $trunks, $rate, $hwaddr) =
+ $net->@{'bridge', 'tag', 'trunks', 'rate', 'hwaddr'};
+
+my $firewall = $net->{firewall} && !PVE::Firewall::is_nftables();
if ($have_sdn) {
PVE::Network::SDN::Zones::tap_plug($iface, $bridge, $tag, $firewall,
$trunks, $rate);
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system
On 2024-04-19 10:57, Fiona Ebner wrote:
> Am 19.04.24 um 10:45 schrieb Lukas Wagner:
>>> Who adds the template files? I don't see a patch for proxmox-ve in this
>>> series. Does this series require some versioned breaks to some package?
>>
>> The pve-manager and pve-ha-manager (for fencing notifications) patches add
>> the templates.
>> I can't use `/usr/share/pve-manager` and `/usr/share/pve-ha-manager` because
>> proxmox_notify needs to have a single base directory from where to load
>> template files.
>> Maybe we should use some other base dir to avoid confusion with the
>> `proxmox-ve` metapackage?
>>>
> Ah, I see. Yes, maybe a directory named based on libpve-notify-perl
> would be better or proxmox-notify (but would need to be a bit careful
> with co-installed PBS and PVE to not create accidental conflicts).
mhmm. In PBS we install the templates in /usr/share/proxmox-backup/... (there
it makes sense,
because the templates are part of the .deb with the same name).
Using `proxmox-notify` would IMO be not so good. We might have similar
notification templates in
both products (e.g. package-updates) which are not necessarily 100% compatible.
Of course we could unify them, but I kinda prefer the flexibility of
having these separate.
A name based on libpve-notify implies that the templates belong to
PVE, which is good. However, that only shifts the problem:
pve-{ha-,}manager still install templates to a location which appears
to be owned by another package, this time libpve-notify...
Thinking about the options, and also about 'user experience' when
adding overridable templates, I think we could just keep on
using 'proxmox-ve' here.
>
>> In terms of versions:
>> pve-{ha}-manager needs to pull in a bumped libpve-notify-perl
>> libpve-notify-perl needs to pull in bumped libpve-rs-perl/libproxmox-rs-perl
>> libpve-rs-perl needs to pull in bumped librust-proxmox-notify
>>
>> I really wish the dep-chain was a bit easier, yet here we are.
>>
>
> But there also is a need for versioned breaks, right? Because installing
> new libpve-notify-perl (using new proxmox-perl-rs using new
> proxmox-notify) without also upgrading pve-manager and pve-ha-manager
> will be broken or am I missing something?
Ah yes, sorry, my packaging knowledge is not yet the best.
A versioned break will be necessary due to the API changes (passing a template
name
instead of a title/body) and due to the fact that the template
files must be present.
--
- Lukas
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH cluster 15/19] notify: use named template instead of passing template strings
Am 09.04.24 um 15:25 schrieb Lukas Wagner:
> The notification system will now load template files from a defined
> location. The template to use is now passed to proxmox_notify, instead
> of separate template strings for subject/body.
>
> Signed-off-by: Lukas Wagner
> Tested-by: Folke Gleumes
> ---
> src/PVE/Notify.pm | 29 -
> 1 file changed, 12 insertions(+), 17 deletions(-)
>
> diff --git a/src/PVE/Notify.pm b/src/PVE/Notify.pm
> index 872eb25..c514111 100644
> --- a/src/PVE/Notify.pm
> +++ b/src/PVE/Notify.pm
> @@ -58,17 +58,16 @@ sub write_config {
> }
>
> my $send_notification = sub {
> -my ($severity, $title, $message, $template_data, $fields, $config) = @_;
> +my ($severity, $template_name, $template_data, $fields, $config) = @_;
> $config = read_config() if !defined($config);
> -$config->send($severity, $title, $message, $template_data, $fields);
> +$config->send($severity, $template_name, $template_data, $fields);
> };
>
> sub notify {
> -my ($severity, $title, $message, $template_data, $fields, $config) = @_;
> +my ($severity, $template_name, $template_data, $fields, $config) = @_;
And there's another versioned breaks required for packages with the
callers of all these functions here. Old version of package with caller
will be broken with new pve-cluster.
> $send_notification->(
> $severity,
> -$title,
> -$message,
> +$template_name,
> $template_data,
> $fields,
> $config
> @@ -76,11 +75,10 @@ sub notify {
> }
>
> sub info {
> -my ($title, $message, $template_data, $fields, $config) = @_;
> +my ($template_name, $template_data, $fields, $config) = @_;
> $send_notification->(
> 'info',
> -$title,
> -$message,
> +$template_name,
> $template_data,
> $fields,
> $config
> @@ -88,11 +86,10 @@ sub info {
> }
>
> sub notice {
> -my ($title, $message, $template_data, $fields, $config) = @_;
> +my ($template_name, $template_data, $fields, $config) = @_;
> $send_notification->(
> 'notice',
> -$title,
> -$message,
> +$template_name,
> $template_data,
> $fields,
> $config
> @@ -100,11 +97,10 @@ sub notice {
> }
>
> sub warning {
> -my ($title, $message, $template_data, $fields, $config) = @_;
> +my ($template_name, $template_data, $fields, $config) = @_;
> $send_notification->(
> 'warning',
> -$title,
> -$message,
> +$template_name,
> $template_data,
> $fields,
> $config
> @@ -112,11 +108,10 @@ sub warning {
> }
>
> sub error {
> -my ($title, $message, $template_data, $fields, $config) = @_;
> +my ($template_name, $template_data, $fields, $config) = @_;
> $send_notification->(
> 'error',
> -$title,
> -$message,
> +$template_name,
> $template_data,
> $fields,
> $config
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [RFC PATCH docs 12/13] installation: use new 'installation-flow' partial from common docs
Signed-off-by: Christoph Heiss
---
pve-installation.adoc | 172 +-
1 file changed, 1 insertion(+), 171 deletions(-)
diff --git a/pve-installation.adoc b/pve-installation.adoc
index f492ad0..2f4642e 100644
--- a/pve-installation.adoc
+++ b/pve-installation.adoc
@@ -52,177 +52,7 @@ The installer ISO image includes the following:
* Web-based management interface
-NOTE: All existing data on the selected drives will be removed during the
-installation process. The installer does not add boot menu entries for other
-operating systems.
-
-Please insert the xref:installation_prepare_media[prepared installation media]
-(for example, USB flash drive or CD-ROM) and boot from it.
-
-TIP: Make sure that booting from the installation medium (for example, USB) is
-enabled in your server's firmware settings. Secure boot needs to be disabled
-when booting an installer prior to {pve} version 8.1.
-
-[thumbnail="screenshot/pve-grub-menu.png"]
-
-After choosing the correct entry (for example, 'Boot from USB') the {pve} menu
-will be displayed, and one of the following options can be selected:
-
-Install {pve} (Graphical)::
-
-Starts the normal installation.
-
-TIP: It's possible to use the installation wizard with a keyboard only. Buttons
-can be clicked by pressing the `ALT` key combined with the underlined character
-from the respective button. For example, `ALT + N` to press a `Next` button.
-
-Install {pve} (Terminal UI)::
-
-Starts the terminal-mode installation wizard. It provides the same overall
-installation experience as the graphical installer, but has generally better
-compatibility with very old and very new hardware.
-
-Install {pve} (Terminal UI, Serial Console)::
-
-Starts the terminal-mode installation wizard, additionally setting up the Linux
-kernel to use the (first) serial port of the machine for in- and output. This
-can be used if the machine is completely headless and only has a serial console
-available.
-
-[thumbnail="screenshot/pve-tui-installer.png"]
-
-Both modes use the same code base for the actual installation process to
-benefit from more than a decade of bug fixes and ensure feature parity.
-
-TIP: The 'Terminal UI' option can be used in case the graphical installer does
-not work correctly, due to e.g. driver issues. See also
-xref:nomodeset_kernel_param[adding the `nomodeset` kernel parameter].
-
-Advanced Options: Install {pve} (Graphical, Debug Mode)::
-
-Starts the installation in debug mode. A console will be opened at several
-installation steps. This helps to debug the situation if something goes wrong.
-To exit a debug console, press `CTRL-D`. This option can be used to boot a live
-system with all basic tools available. You can use it, for example, to
-xref:chapter_zfs[repair a degraded ZFS 'rpool'] or fix the
-xref:sysboot[bootloader] for an existing {pve} setup.
-
-Advanced Options: Install {pve} (Terminal UI, Debug Mode)::
-
-Same as the graphical debug mode, but preparing the system to run the
-terminal-based installer instead.
-
-Advanced Options: Install {pve} (Serial Console Debug Mode)::
-
-Same the terminal-based debug mode, but additionally sets up the Linux kernel
to
-use the (first) serial port of the machine for in- and output.
-
-Advanced Options: Rescue Boot::
-
-With this option you can boot an existing installation. It searches all
attached
-hard disks. If it finds an existing installation, it boots directly into that
-disk using the Linux kernel from the ISO. This can be useful if there are
-problems with the bootloader (GRUB/`systemd-boot`) or the BIOS/UEFI is unable
to
-read the boot block from the disk.
-
-Advanced Options: Test Memory (memtest86+)::
-
-Runs `memtest86+`. This is useful to check if the memory is functional and free
-of errors. Secure Boot must be turned off in the UEFI firmware setup utility to
-run this option.
-
-You normally select *Install {pve} (Graphical)* to start the installation.
-
-[thumbnail="screenshot/pve-select-target-disk.png"]
-
-The first step is to read our EULA (End User License Agreement). Following
this,
-you can select the target hard disk(s) for the installation.
-
-CAUTION: By default, the whole server is used and all existing data is removed.
-Make sure there is no important data on the server before proceeding with the
-installation.
-
-The `Options` button lets you select the target file system, which
-defaults to `ext4`. The installer uses LVM if you select
-`ext4` or `xfs` as a file system, and offers additional options to
-restrict LVM space (see xref:advanced_lvm_options[below]).
-
-{pve} can also be installed on ZFS. As ZFS offers several software RAID levels,
-this is an option for systems that don't have a hardware RAID controller. The
-target disks must be selected in the `Options` dialog. More ZFS specific
-settings can be changed under xref:advanced_zfs_options[`Advanced Options`].
-
-WARNING: ZFS on top of any hardware RAID is not supported and can result i
[pve-devel] [RFC PATCH docs 13/13] installation: use new 'advanced-installation' partial from common docs
Signed-off-by: Christoph Heiss
---
pve-installation.adoc | 126 +-
1 file changed, 1 insertion(+), 125 deletions(-)
diff --git a/pve-installation.adoc b/pve-installation.adoc
index 2f4642e..0a0dc76 100644
--- a/pve-installation.adoc
+++ b/pve-installation.adoc
@@ -78,131 +78,7 @@ web interface for further configuration.
. Check your xref:chapter_pve_firewall[Firewall settings].
-[[advanced_lvm_options]]
-Advanced LVM Configuration Options
-~~
-
-The installer creates a Volume Group (VG) called `pve`, and additional Logical
-Volumes (LVs) called `root`, `data`, and `swap`, if `ext4` or `xfs` is used. To
-control the size of these volumes use:
-
-`hdsize`::
-
-Defines the total hard disk size to be used. This way you can reserve free
space
-on the hard disk for further partitioning (for example for an additional PV and
-VG on the same hard disk that can be used for LVM storage).
-
-`swapsize`::
-
-Defines the size of the `swap` volume. The default is the size of the installed
-memory, minimum 4 GB and maximum 8 GB. The resulting value cannot be greater
-than `hdsize/8`.
-+
-NOTE: If set to `0`, no `swap` volume will be created.
-
-`maxroot`::
-
-Defines the maximum size of the `root` volume, which stores the operation
-system. The maximum limit of the `root` volume size is `hdsize/4`.
-
-`maxvz`::
-
-Defines the maximum size of the `data` volume. The actual size of the `data`
-volume is:
-+
-`datasize = hdsize - rootsize - swapsize - minfree`
-+
-Where `datasize` cannot be bigger than `maxvz`.
-+
-NOTE: In case of LVM thin, the `data` pool will only be created if `datasize`
is
-bigger than 4GB.
-+
-NOTE: If set to `0`, no `data` volume will be created and the storage
-configuration will be adapted accordingly.
-
-`minfree`::
-
-Defines the amount of free space that should be left in the LVM volume group
-`pve`. With more than 128GB storage available, the default is 16GB, otherwise
-`hdsize/8` will be used.
-+
-NOTE: LVM requires free space in the VG for snapshot creation (not required for
-lvmthin snapshots).
-
-[[advanced_zfs_options]]
-Advanced ZFS Configuration Options
-~~
-The installer creates the ZFS pool `rpool`, if ZFS is used. No swap space is
-created but you can reserve some unpartitioned space on the install disks for
-swap. You can also create a swap zvol after the installation, although this can
-lead to problems (see xref:zfs_swap[ZFS swap notes]).
-
-`ashift`::
-
-Defines the `ashift` value for the created pool. The `ashift` needs to be set
at
-least to the sector-size of the underlying disks (2 to the power of `ashift` is
-the sector-size), or any disk which might be put in the pool (for example the
-replacement of a defective disk).
-
-`compress`::
-
-Defines whether compression is enabled for `rpool`.
-
-`checksum`::
-
-Defines which checksumming algorithm should be used for `rpool`.
-
-`copies`::
-
-Defines the `copies` parameter for `rpool`. Check the `zfs(8)` manpage for the
-semantics, and why this does not replace redundancy on disk-level.
-
-`ARC max size`::
-
-Defines the maximum size the ARC can grow to and thus limits the amount of
-memory ZFS will use. See also the section on
-xref:sysadmin_zfs_limit_memory_usage[how to limit ZFS memory usage] for more
-details.
-
-`hdsize`::
-
-Defines the total hard disk size to be used. This is useful to save free space
-on the hard disk(s) for further partitioning (for example to create a
-swap-partition). `hdsize` is only honored for bootable disks, that is only the
-first disk or mirror for RAID0, RAID1 or RAID10, and all disks in RAID-Z[123].
-
-
-ZFS Performance Tips
-
-
-ZFS works best with a lot of memory. If you intend to use ZFS make sure to have
-enough RAM available for it. A good calculation is 4GB plus 1GB RAM for each TB
-RAW disk space.
-
-ZFS can use a dedicated drive as write cache, called the ZFS Intent Log (ZIL).
-Use a fast drive (SSD) for it. It can be added after installation with the
-following command:
-
-
-# zpool add log
-
-
-[[nomodeset_kernel_param]]
-Adding the `nomodeset` Kernel Parameter
-~
-
-Problems may arise on very old or very new hardware due to graphics drivers. If
-the installation hangs during boot, you can try adding the `nomodeset`
-parameter. This prevents the Linux kernel from loading any graphics drivers and
-forces it to continue using the BIOS/UEFI-provided framebuffer.
-
-On the {pve} bootloader menu, navigate to 'Install {pve} (Terminal UI)' and
-press `e` to edit the entry. Using the arrow keys, navigate to the line
starting
-with `linux`, move the cursor to the end of that line and add the
-parameter `nomodeset`, separated by a space from the pre-existing last
-parameter.
-
-Then press `Ctrl-X` or `F10` to boot the configuration.
+include::proxmox-docs-common/partials/advanced-installation.adoc[]
ifndef::wik
[pve-devel] [RFC PATCH docs 07/13] gitmodules: add proxmox-docs-common
Signed-off-by: Christoph Heiss --- .gitmodules | 3 +++ proxmox-docs-common | 1 + 2 files changed, 4 insertions(+) create mode 100644 .gitmodules create mode 16 proxmox-docs-common diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 000..eff8adf --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "proxmox-docs-common"] + path = proxmox-docs-common + url = ../proxmox-docs-common diff --git a/proxmox-docs-common b/proxmox-docs-common new file mode 16 index 000..a042676 --- /dev/null +++ b/proxmox-docs-common @@ -0,0 +1 @@ +Subproject commit a042676729ea0b87a03844eb4c468028bb45f55d -- 2.44.0 ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [RFC PATCH docs-common 06/13] partials: advanced-installation: adapt from pve-docs
Signed-off-by: Christoph Heiss
---
partials/advanced-installation.adoc | 18 +-
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/partials/advanced-installation.adoc
b/partials/advanced-installation.adoc
index ed2709f..271533b 100644
--- a/partials/advanced-installation.adoc
+++ b/partials/advanced-installation.adoc
@@ -2,9 +2,9 @@
Advanced LVM Configuration Options
~~
-The installer creates a Volume Group (VG) called `pve`, and additional Logical
-Volumes (LVs) called `root`, `data`, and `swap`, if `ext4` or `xfs` is used. To
-control the size of these volumes use:
+The installer creates a Volume Group (VG) called +{product-short}+, and
+additional Logical Volumes (LVs) called `root`, `data`, and `swap`, if `ext4`
or
+`xfs` is used. To control the size of these volumes use:
`hdsize`::
@@ -43,8 +43,8 @@ configuration will be adapted accordingly.
`minfree`::
Defines the amount of free space that should be left in the LVM volume group
-`pve`. With more than 128GB storage available, the default is 16GB, otherwise
-`hdsize/8` will be used.
++{product-short}+. With more than 128GB storage available, the default is 16GB,
+otherwise `hdsize/8` will be used.
+
NOTE: LVM requires free space in the VG for snapshot creation (not required for
lvmthin snapshots).
@@ -109,16 +109,16 @@ following command:
[[nomodeset_kernel_param]]
Adding the `nomodeset` Kernel Parameter
-~
+~~~
Problems may arise on very old or very new hardware due to graphics drivers. If
the installation hangs during boot, you can try adding the `nomodeset`
parameter. This prevents the Linux kernel from loading any graphics drivers and
forces it to continue using the BIOS/UEFI-provided framebuffer.
-On the {pve} bootloader menu, navigate to 'Install {pve} (Terminal UI)' and
-press `e` to edit the entry. Using the arrow keys, navigate to the line
starting
-with `linux`, move the cursor to the end of that line and add the
+On the {product} bootloader menu, navigate to 'Install {product} (Terminal UI)'
+and press `e` to edit the entry. Using the arrow keys, navigate to the line
+starting with `linux`, move the cursor to the end of that line and add the
parameter `nomodeset`, separated by a space from the pre-existing last
parameter.
--
2.44.0
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [RFC PATCH docs 10/13] asciidoc: conf: add iso-url variable
Signed-off-by: Christoph Heiss --- asciidoc/asciidoc-pve.conf | 1 + 1 file changed, 1 insertion(+) diff --git a/asciidoc/asciidoc-pve.conf b/asciidoc/asciidoc-pve.conf index 47139b8..faa190a 100644 --- a/asciidoc/asciidoc-pve.conf +++ b/asciidoc/asciidoc-pve.conf @@ -6,6 +6,7 @@ pve=Proxmox VE product=Proxmox VE product-short=pve pricing-url=https://proxmox.com/en/proxmox-virtual-environment/pricing +iso-url=https://proxmox.com/downloads/proxmox-virtual-environment website=https://www.proxmox.com/ forum-url=https://forum.proxmox.com/ forum=https://forum.proxmox.com/[Proxmox VE Community Forum] -- 2.44.0 ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [RFC PATCH docs-common 04/13] partials: installation-flow: adapt from pve-docs
A trivial s/{pve}/{product}/g again and a additional
s/screenshot\/pve-/screenshot\//g to fix the screenshot paths.
Signed-off-by: Christoph Heiss
---
partials/installation-flow.adoc | 47 +
1 file changed, 24 insertions(+), 23 deletions(-)
diff --git a/partials/installation-flow.adoc b/partials/installation-flow.adoc
index b796676..9a7d1b3 100644
--- a/partials/installation-flow.adoc
+++ b/partials/installation-flow.adoc
@@ -7,14 +7,14 @@ Please insert the xref:installation_prepare_media[prepared
installation media]
TIP: Make sure that booting from the installation medium (for example, USB) is
enabled in your server's firmware settings. Secure boot needs to be disabled
-when booting an installer prior to {pve} version 8.1.
+when booting an installer prior to {product} version 8.1.
-[thumbnail="screenshot/pve-grub-menu.png"]
+[thumbnail="screenshot/grub-menu.png"]
-After choosing the correct entry (for example, 'Boot from USB') the {pve} menu
-will be displayed, and one of the following options can be selected:
+After choosing the correct entry (for example, 'Boot from USB') the {product}
+menu will be displayed, and one of the following options can be selected:
-Install {pve} (Graphical)::
+Install {product} (Graphical)::
Starts the normal installation.
@@ -22,20 +22,20 @@ TIP: It's possible to use the installation wizard with a
keyboard only. Buttons
can be clicked by pressing the `ALT` key combined with the underlined character
from the respective button. For example, `ALT + N` to press a `Next` button.
-Install {pve} (Terminal UI)::
+Install {product} (Terminal UI)::
Starts the terminal-mode installation wizard. It provides the same overall
installation experience as the graphical installer, but has generally better
compatibility with very old and very new hardware.
-Install {pve} (Terminal UI, Serial Console)::
+Install {product} (Terminal UI, Serial Console)::
Starts the terminal-mode installation wizard, additionally setting up the Linux
kernel to use the (first) serial port of the machine for in- and output. This
can be used if the machine is completely headless and only has a serial console
available.
-[thumbnail="screenshot/pve-tui-installer.png"]
+[thumbnail="screenshot/tui-installer.png"]
Both modes use the same code base for the actual installation process to
benefit from more than a decade of bug fixes and ensure feature parity.
@@ -44,21 +44,21 @@ TIP: The 'Terminal UI' option can be used in case the
graphical installer does
not work correctly, due to e.g. driver issues. See also
xref:nomodeset_kernel_param[adding the `nomodeset` kernel parameter].
-Advanced Options: Install {pve} (Graphical, Debug Mode)::
+Advanced Options: Install {product} (Graphical, Debug Mode)::
Starts the installation in debug mode. A console will be opened at several
installation steps. This helps to debug the situation if something goes wrong.
To exit a debug console, press `CTRL-D`. This option can be used to boot a live
system with all basic tools available. You can use it, for example, to
xref:chapter_zfs[repair a degraded ZFS 'rpool'] or fix the
-xref:sysboot[bootloader] for an existing {pve} setup.
+xref:sysboot[bootloader] for an existing {product} setup.
-Advanced Options: Install {pve} (Terminal UI, Debug Mode)::
+Advanced Options: Install {product} (Terminal UI, Debug Mode)::
Same as the graphical debug mode, but preparing the system to run the
terminal-based installer instead.
-Advanced Options: Install {pve} (Serial Console Debug Mode)::
+Advanced Options: Install {product} (Serial Console Debug Mode)::
Same the terminal-based debug mode, but additionally sets up the Linux kernel
to
use the (first) serial port of the machine for in- and output.
@@ -77,9 +77,9 @@ Runs `memtest86+`. This is useful to check if the memory is
functional and free
of errors. Secure Boot must be turned off in the UEFI firmware setup utility to
run this option.
-You normally select *Install {pve} (Graphical)* to start the installation.
+You normally select *Install {product} (Graphical)* to start the installation.
-[thumbnail="screenshot/pve-select-target-disk.png"]
+[thumbnail="screenshot/select-target-disk.png"]
The first step is to read our EULA (End User License Agreement). Following
this,
you can select the target hard disk(s) for the installation.
@@ -93,15 +93,16 @@ defaults to `ext4`. The installer uses LVM if you select
`ext4` or `xfs` as a file system, and offers additional options to
restrict LVM space (see xref:advanced_lvm_options[below]).
-{pve} can also be installed on ZFS. As ZFS offers several software RAID levels,
-this is an option for systems that don't have a hardware RAID controller. The
-target disks must be selected in the `Options` dialog. More ZFS specific
-settings can be changed under xref:advanced_zfs_options[`Advanced Options`].
+{product} can also be installed on ZFS. As ZFS offers several software RAID
+le
[pve-devel] [RFC PATCH docs 11/13] installation-media: move to common docs
Signed-off-by: Christoph Heiss
---
pve-installation-media.adoc | 132
pve-installation.adoc | 2 +-
2 files changed, 1 insertion(+), 133 deletions(-)
delete mode 100644 pve-installation-media.adoc
diff --git a/pve-installation-media.adoc b/pve-installation-media.adoc
deleted file mode 100644
index a1c9402..000
--- a/pve-installation-media.adoc
+++ /dev/null
@@ -1,132 +0,0 @@
-[[installation_prepare_media]]
-Prepare Installation Media
---
-ifdef::wiki[]
-:pve-toplevel:
-endif::wiki[]
-
-Download the installer ISO image from:
{website}en/downloads/proxmox-virtual-environment/iso
-
-The {pve} installation media is a hybrid ISO image. It works in two ways:
-
-* An ISO image file ready to burn to a CD or DVD.
-
-* A raw sector (IMG) image file ready to copy to a USB flash drive (USB stick).
-
-Using a USB flash drive to install {pve} is the recommended way because it is
-the faster option.
-
-Prepare a USB Flash Drive as Installation Medium
-
-
-The flash drive needs to have at least 1 GB of storage available.
-
-NOTE: Do not use UNetbootin. It does not work with the {pve} installation
image.
-
-IMPORTANT: Make sure that the USB flash drive is not mounted and does not
-contain any important data.
-
-
-Instructions for GNU/Linux
-~~
-
-On Unix-like operating system use the `dd` command to copy the ISO image to the
-USB flash drive. First find the correct device name of the USB flash drive (see
-below). Then run the `dd` command.
-
-
-# dd bs=1M conv=fdatasync if=./proxmox-ve_*.iso of=/dev/XYZ
-
-
-NOTE: Be sure to replace /dev/XYZ with the correct device name and adapt the
-input filename ('if') path.
-
-CAUTION: Be very careful, and do not overwrite the wrong disk!
-
-
-Find the Correct USB Device Name
-
-There are two ways to find out the name of the USB flash drive. The first one
is
-to compare the last lines of the `dmesg` command output before and after
-plugging in the flash drive. The second way is to compare the output of the
-`lsblk` command. Open a terminal and run:
-
-
-# lsblk
-
-
-Then plug in your USB flash drive and run the command again:
-
-
-# lsblk
-
-
-A new device will appear. This is the one you want to use. To be on the extra
-safe side check if the reported size matches your USB flash drive.
-
-
-Instructions for macOS
-~~
-
-Open the terminal (query Terminal in Spotlight).
-
-Convert the `.iso` file to `.dmg` format using the convert option of `hdiutil`,
-for example:
-
-
-# hdiutil convert proxmox-ve_*.iso -format UDRW -o proxmox-ve_*.dmg
-
-
-TIP: macOS tends to automatically add '.dmg' to the output file name.
-
-To get the current list of devices run the command:
-
-
-# diskutil list
-
-
-Now insert the USB flash drive and run this command again to determine which
-device node has been assigned to it. (e.g., /dev/diskX).
-
-
-# diskutil list
-# diskutil unmountDisk /dev/diskX
-
-
-NOTE: replace X with the disk number from the last command.
-
-
-# sudo dd if=proxmox-ve_*.dmg bs=1M of=/dev/rdiskX
-
-
-NOTE: 'rdiskX', instead of 'diskX', in the last command is intended. It will
-increase the write speed.
-
-Instructions for Windows
-
-
-Using Etcher
-
-
-Etcher works out of the box. Download Etcher from https://etcher.io. It will
-guide you through the process of selecting the ISO and your USB flash drive.
-
-Using Rufus
-^^^
-
-Rufus is a more lightweight alternative, but you need to use the *DD mode* to
-make it work. Download Rufus from https://rufus.ie/. Either install it or use
-the portable version. Select the destination drive and the {pve} ISO file.
-
-IMPORTANT: Once you 'Start' you have to click 'No' on the dialog asking to
-download a different version of GRUB. In the next dialog select the 'DD' mode.
-
-ifdef::wiki[]
-Boot your Server from the USB Flash Drive
-~
-
-Connect the USB flash drive to your server and make sure that booting from USB
-is enabled (check your servers firmware settings). Then follow the steps in the
-xref:chapter_installation[installation wizard].
-
-endif::wiki[]
diff --git a/pve-installation.adoc b/pve-installation.adoc
index 274d9ad..f492ad0 100644
--- a/pve-installation.adoc
+++ b/pve-installation.adoc
@@ -29,7 +29,7 @@ ifndef::wiki[]
include::pve-system-requirements.adoc[]
-include::pve-installation-media.adoc[]
+include::proxmox-docs-common/installation-media.adoc[]
endif::wiki[]
--
2.44.0
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [RFC PATCH docs 09/13] images: strip `pve-` prefix from screenshots used in common docs
Signed-off-by: Christoph Heiss
---
.../{pve-grub-menu.png => grub-menu.png} | Bin
.../{pve-grub-menu.ppm => grub-menu.ppm} | Bin
...ve-install-summary.png => install-summary.png} | Bin
...ve-install-summary.ppm => install-summary.ppm} | Bin
.../{pve-installation.png => installation.png}| Bin
.../{pve-installation.ppm => installation.ppm}| Bin
...ve-select-location.png => select-location.png} | Bin
...ve-select-location.ppm => select-location.ppm} | Bin
...ect-target-disk.png => select-target-disk.png} | Bin
...ect-target-disk.ppm => select-target-disk.ppm} | Bin
.../{pve-set-password.png => set-password.png}| Bin
.../{pve-set-password.ppm => set-password.ppm}| Bin
.../{pve-setup-network.png => setup-network.png} | Bin
.../{pve-setup-network.ppm => setup-network.ppm} | Bin
.../{pve-tui-installer.png => tui-installer.png} | Bin
.../{pve-tui-installer.ppm => tui-installer.ppm} | Bin
png-verify.pl | 14 --
17 files changed, 8 insertions(+), 6 deletions(-)
rename images/screenshot/{pve-grub-menu.png => grub-menu.png} (100%)
rename images/screenshot/{pve-grub-menu.ppm => grub-menu.ppm} (100%)
rename images/screenshot/{pve-install-summary.png => install-summary.png}
(100%)
rename images/screenshot/{pve-install-summary.ppm => install-summary.ppm}
(100%)
rename images/screenshot/{pve-installation.png => installation.png} (100%)
rename images/screenshot/{pve-installation.ppm => installation.ppm} (100%)
rename images/screenshot/{pve-select-location.png => select-location.png}
(100%)
rename images/screenshot/{pve-select-location.ppm => select-location.ppm}
(100%)
rename images/screenshot/{pve-select-target-disk.png =>
select-target-disk.png} (100%)
rename images/screenshot/{pve-select-target-disk.ppm =>
select-target-disk.ppm} (100%)
rename images/screenshot/{pve-set-password.png => set-password.png} (100%)
rename images/screenshot/{pve-set-password.ppm => set-password.ppm} (100%)
rename images/screenshot/{pve-setup-network.png => setup-network.png} (100%)
rename images/screenshot/{pve-setup-network.ppm => setup-network.ppm} (100%)
rename images/screenshot/{pve-tui-installer.png => tui-installer.png} (100%)
rename images/screenshot/{pve-tui-installer.ppm => tui-installer.ppm} (100%)
diff --git a/images/screenshot/pve-grub-menu.png
b/images/screenshot/grub-menu.png
similarity index 100%
rename from images/screenshot/pve-grub-menu.png
rename to images/screenshot/grub-menu.png
diff --git a/images/screenshot/pve-grub-menu.ppm
b/images/screenshot/grub-menu.ppm
similarity index 100%
rename from images/screenshot/pve-grub-menu.ppm
rename to images/screenshot/grub-menu.ppm
diff --git a/images/screenshot/pve-install-summary.png
b/images/screenshot/install-summary.png
similarity index 100%
rename from images/screenshot/pve-install-summary.png
rename to images/screenshot/install-summary.png
diff --git a/images/screenshot/pve-install-summary.ppm
b/images/screenshot/install-summary.ppm
similarity index 100%
rename from images/screenshot/pve-install-summary.ppm
rename to images/screenshot/install-summary.ppm
diff --git a/images/screenshot/pve-installation.png
b/images/screenshot/installation.png
similarity index 100%
rename from images/screenshot/pve-installation.png
rename to images/screenshot/installation.png
diff --git a/images/screenshot/pve-installation.ppm
b/images/screenshot/installation.ppm
similarity index 100%
rename from images/screenshot/pve-installation.ppm
rename to images/screenshot/installation.ppm
diff --git a/images/screenshot/pve-select-location.png
b/images/screenshot/select-location.png
similarity index 100%
rename from images/screenshot/pve-select-location.png
rename to images/screenshot/select-location.png
diff --git a/images/screenshot/pve-select-location.ppm
b/images/screenshot/select-location.ppm
similarity index 100%
rename from images/screenshot/pve-select-location.ppm
rename to images/screenshot/select-location.ppm
diff --git a/images/screenshot/pve-select-target-disk.png
b/images/screenshot/select-target-disk.png
similarity index 100%
rename from images/screenshot/pve-select-target-disk.png
rename to images/screenshot/select-target-disk.png
diff --git a/images/screenshot/pve-select-target-disk.ppm
b/images/screenshot/select-target-disk.ppm
similarity index 100%
rename from images/screenshot/pve-select-target-disk.ppm
rename to images/screenshot/select-target-disk.ppm
diff --git a/images/screenshot/pve-set-password.png
b/images/screenshot/set-password.png
similarity index 100%
rename from images/screenshot/pve-set-password.png
rename to images/screenshot/set-password.png
diff --git a/images/screenshot/pve-set-password.ppm
b/images/screenshot/set-password.ppm
similarity index 100%
rename from images/screenshot/pve-set-password.ppm
rename to images/screenshot/set-password.ppm
diff --git a/images/screenshot/pve-setup-network.png
b/images/screenshot/setup-network.png
s
[pve-devel] [RFC PATCH docs-common 05/13] partials: add advanced installation hints from pve-docs
Signed-off-by: Christoph Heiss
---
partials/advanced-installation.adoc | 125
1 file changed, 125 insertions(+)
create mode 100644 partials/advanced-installation.adoc
diff --git a/partials/advanced-installation.adoc
b/partials/advanced-installation.adoc
new file mode 100644
index 000..ed2709f
--- /dev/null
+++ b/partials/advanced-installation.adoc
@@ -0,0 +1,125 @@
+[[advanced_lvm_options]]
+Advanced LVM Configuration Options
+~~
+
+The installer creates a Volume Group (VG) called `pve`, and additional Logical
+Volumes (LVs) called `root`, `data`, and `swap`, if `ext4` or `xfs` is used. To
+control the size of these volumes use:
+
+`hdsize`::
+
+Defines the total hard disk size to be used. This way you can reserve free
space
+on the hard disk for further partitioning (for example for an additional PV and
+VG on the same hard disk that can be used for LVM storage).
+
+`swapsize`::
+
+Defines the size of the `swap` volume. The default is the size of the installed
+memory, minimum 4 GB and maximum 8 GB. The resulting value cannot be greater
+than `hdsize/8`.
++
+NOTE: If set to `0`, no `swap` volume will be created.
+
+`maxroot`::
+
+Defines the maximum size of the `root` volume, which stores the operation
+system. The maximum limit of the `root` volume size is `hdsize/4`.
+
+`maxvz`::
+
+Defines the maximum size of the `data` volume. The actual size of the `data`
+volume is:
++
+`datasize = hdsize - rootsize - swapsize - minfree`
++
+Where `datasize` cannot be bigger than `maxvz`.
++
+NOTE: In case of LVM thin, the `data` pool will only be created if `datasize`
is
+bigger than 4GB.
++
+NOTE: If set to `0`, no `data` volume will be created and the storage
+configuration will be adapted accordingly.
+
+`minfree`::
+
+Defines the amount of free space that should be left in the LVM volume group
+`pve`. With more than 128GB storage available, the default is 16GB, otherwise
+`hdsize/8` will be used.
++
+NOTE: LVM requires free space in the VG for snapshot creation (not required for
+lvmthin snapshots).
+
+[[advanced_zfs_options]]
+Advanced ZFS Configuration Options
+~~
+The installer creates the ZFS pool `rpool`, if ZFS is used. No swap space is
+created but you can reserve some unpartitioned space on the install disks for
+swap. You can also create a swap zvol after the installation, although this can
+lead to problems (see xref:zfs_swap[ZFS swap notes]).
+
+`ashift`::
+
+Defines the `ashift` value for the created pool. The `ashift` needs to be set
at
+least to the sector-size of the underlying disks (2 to the power of `ashift` is
+the sector-size), or any disk which might be put in the pool (for example the
+replacement of a defective disk).
+
+`compress`::
+
+Defines whether compression is enabled for `rpool`.
+
+`checksum`::
+
+Defines which checksumming algorithm should be used for `rpool`.
+
+`copies`::
+
+Defines the `copies` parameter for `rpool`. Check the `zfs(8)` manpage for the
+semantics, and why this does not replace redundancy on disk-level.
+
+`ARC max size`::
+
+Defines the maximum size the ARC can grow to and thus limits the amount of
+memory ZFS will use. See also the section on
+xref:sysadmin_zfs_limit_memory_usage[how to limit ZFS memory usage] for more
+details.
+
+`hdsize`::
+
+Defines the total hard disk size to be used. This is useful to save free space
+on the hard disk(s) for further partitioning (for example to create a
+swap-partition). `hdsize` is only honored for bootable disks, that is only the
+first disk or mirror for RAID0, RAID1 or RAID10, and all disks in RAID-Z[123].
+
+
+ZFS Performance Tips
+
+
+ZFS works best with a lot of memory. If you intend to use ZFS make sure to have
+enough RAM available for it. A good calculation is 4GB plus 1GB RAM for each TB
+RAW disk space.
+
+ZFS can use a dedicated drive as write cache, called the ZFS Intent Log (ZIL).
+Use a fast drive (SSD) for it. It can be added after installation with the
+following command:
+
+
+# zpool add log
+
+
+[[nomodeset_kernel_param]]
+Adding the `nomodeset` Kernel Parameter
+~
+
+Problems may arise on very old or very new hardware due to graphics drivers. If
+the installation hangs during boot, you can try adding the `nomodeset`
+parameter. This prevents the Linux kernel from loading any graphics drivers and
+forces it to continue using the BIOS/UEFI-provided framebuffer.
+
+On the {pve} bootloader menu, navigate to 'Install {pve} (Terminal UI)' and
+press `e` to edit the entry. Using the arrow keys, navigate to the line
starting
+with `linux`, move the cursor to the end of that line and add the
+parameter `nomodeset`, separated by a space from the pre-existing last
+parameter.
+
+Then press `Ctrl-X` or `F10` to boot the configuration.
--
2.44.0
___
pve-devel mailing list
pve-devel@lists.proxm
[pve-devel] [RFC PATCH docs 08/13] scan-adoc-refs: enable building pages from proxmox-docs-common/ subdir
Signed-off-by: Christoph Heiss
---
Makefile | 6 --
asciidoc/asciidoc-pve.conf | 2 ++
pve-doc-generator.mk.in| 6 ++
scan-adoc-refs | 25 -
4 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/Makefile b/Makefile
index 801a2a3..5b14e3d 100644
--- a/Makefile
+++ b/Makefile
@@ -26,9 +26,11 @@ all: index.html
verify-images:
for i in ./images/screenshot/*.png; do ./png-verify.pl $$i; done
-ADOC_SOURCES_GUESS=$(filter-out %-synopsis.adoc %-opts.adoc %-table.adoc,
$(wildcard *.adoc))
+ADOC_WILDCARD_SOURCES := *.adoc proxmox-docs-common/*.adoc
proxmox-docs-common/partials/*.adoc
+ADOC_SOURCES_GUESS=$(filter-out %-synopsis.adoc %-opts.adoc %-table.adoc,
$(wildcard $(ADOC_WILDCARD_SOURCES)))
+
.pve-doc-depends link-refs.json: $(ADOC_SOURCES_GUESS) scan-adoc-refs
- ./scan-adoc-refs *.adoc --depends .pve-doc-depends.tmp >
link-refs.json.tmp
+ ./scan-adoc-refs $(ADOC_WILDCARD_SOURCES) --depends
.pve-doc-depends.tmp > link-refs.json.tmp
@cmp --quiet .pve-doc-depends .pve-doc-depends.tmp || mv
.pve-doc-depends.tmp .pve-doc-depends
@cmp --quiet link-refs.json link-refs.json.tmp || mv link-refs.json.tmp
link-refs.json
diff --git a/asciidoc/asciidoc-pve.conf b/asciidoc/asciidoc-pve.conf
index 0c28298..47139b8 100644
--- a/asciidoc/asciidoc-pve.conf
+++ b/asciidoc/asciidoc-pve.conf
@@ -3,6 +3,8 @@
proxmoxGmbh=Proxmox Server Solutions GmbH
copyright=Proxmox Server Solutions GmbH
pve=Proxmox VE
+product=Proxmox VE
+product-short=pve
pricing-url=https://proxmox.com/en/proxmox-virtual-environment/pricing
website=https://www.proxmox.com/
forum-url=https://forum.proxmox.com/
diff --git a/pve-doc-generator.mk.in b/pve-doc-generator.mk.in
index 77fd4f6..a733966 100644
--- a/pve-doc-generator.mk.in
+++ b/pve-doc-generator.mk.in
@@ -71,9 +71,15 @@ endif
%-plain.html: %.adoc ${PVE_COMMON_DOC_SOURCES}
${ASCIIDOC_PVE} compile-wiki -o $@ $*.adoc
+%-plain.html: proxmox-docs-common/%.adoc ${PVE_COMMON_DOC_SOURCES}
+ ${ASCIIDOC_PVE} compile-wiki -o $@ proxmox-docs-common/$*.adoc
+
chapter-%.html: %.adoc ${PVE_COMMON_DOC_SOURCES}
${ASCIIDOC_PVE} compile-chapter -o $@ $*.adoc
+chapter-%.html: proxmox-docs-common/%.adoc ${PVE_COMMON_DOC_SOURCES}
+ ${ASCIIDOC_PVE} compile-chapter -o $@ proxmox-docs-common/$*.adoc
+
%.1: %.adoc %.1-synopsis.adoc ${PVE_COMMON_DOC_SOURCES}
${ASCIIDOC_PVE} compile-man -o $@ $*.adoc
test -z "$${PVE_DOC_INSTANTVIEW}" || man -l $@
diff --git a/scan-adoc-refs b/scan-adoc-refs
index 9252701..fd74c20 100755
--- a/scan-adoc-refs
+++ b/scan-adoc-refs
@@ -5,6 +5,8 @@ use warnings;
use Getopt::Long;
use IO::File;
use JSON;
+use Cwd qw(abs_path);
+use File::Spec;
use Data::Dumper;
@@ -19,6 +21,7 @@ my $environments = {
wiki => 1,
manvolnum => 1,
pvelogo => 0, # ignore
+pve => 0,
};
my $fileinfo = {};
@@ -76,6 +79,12 @@ sub pop_environment {
sub register_include {
my ($filename, $include_filename, $env_list) = @_;
+# get containing directory of the file that includes ..
+my $reldir = (File::Spec->splitpath($filename))[1];
+# .. and resolve the included filename relative to that
+$include_filename = abs_path(File::Spec->join($reldir, $include_filename))
+ if $reldir;
+
foreach my $e (@$env_list) {
$fileinfo->{include}->{$e}->{$filename}->{$include_filename} = 1;
}
@@ -99,6 +108,7 @@ sub register_title {
# fixme: what about other macros?
$title =~ s/\{pve\}/Proxmox VE/g;
+$title =~ s/\{product\}/Proxmox VE/g;
$title =~ s!http://\S+\[(.*?)\]!$1!g;
# register document title (onyl once)
@@ -269,23 +279,19 @@ foreach my $e (@$start_env) {
my $toplevel_hash = $fileinfo->{toplevel}->{$e};
foreach my $fn (sort keys %$toplevel_hash) {
my $mansection = $fileinfo->{mansection}->{manvolnum}->{$fn};
+ my $realfn = $fn;
+ $realfn =~ s/^proxmox-docs-common\///;
+ $realfn =~ s/\.adoc$//;
+
if ($e eq 'wiki') {
- my $realfn = $fn;
- $realfn =~ s/\.adoc$//;
if (defined($mansection) && ($mansection == 5)) {
$realfn .= ".$mansection";
}
$realfn = "$realfn-plain.html";
- $fileinfo->{outfile}->{$e}->{$fn} = $realfn;
} elsif ($e eq 'manvolnum') {
- my $realfn = $fn;
- $realfn =~ s/\.adoc$//;
die "toplevel file '$fn' is not marked as manual page!" if
!$mansection;
$realfn .= ".$mansection";
- $fileinfo->{outfile}->{$e}->{$fn} = $realfn;
} elsif ($e eq 'default') {
- my $realfn = $fn;
- $realfn =~ s/\.adoc$//;
if (defined($mansection) && ($mansection == 5)) {
$realfn .= ".$mansection";
$realfn = "$realfn.html";
@@ -297,8 +303,9 @@ foreach my $e (@$start_env) {
$realfn = "$realfn.html";
[pve-devel] [RFC PATCH docs-common 03/13] partials: add installation flow from pve-docs
Signed-off-by: Christoph Heiss
---
partials/installation-flow.adoc | 170
1 file changed, 170 insertions(+)
create mode 100644 partials/installation-flow.adoc
diff --git a/partials/installation-flow.adoc b/partials/installation-flow.adoc
new file mode 100644
index 000..b796676
--- /dev/null
+++ b/partials/installation-flow.adoc
@@ -0,0 +1,170 @@
+NOTE: All existing data on the selected drives will be removed during the
+installation process. The installer does not add boot menu entries for other
+operating systems.
+
+Please insert the xref:installation_prepare_media[prepared installation media]
+(for example, USB flash drive or CD-ROM) and boot from it.
+
+TIP: Make sure that booting from the installation medium (for example, USB) is
+enabled in your server's firmware settings. Secure boot needs to be disabled
+when booting an installer prior to {pve} version 8.1.
+
+[thumbnail="screenshot/pve-grub-menu.png"]
+
+After choosing the correct entry (for example, 'Boot from USB') the {pve} menu
+will be displayed, and one of the following options can be selected:
+
+Install {pve} (Graphical)::
+
+Starts the normal installation.
+
+TIP: It's possible to use the installation wizard with a keyboard only. Buttons
+can be clicked by pressing the `ALT` key combined with the underlined character
+from the respective button. For example, `ALT + N` to press a `Next` button.
+
+Install {pve} (Terminal UI)::
+
+Starts the terminal-mode installation wizard. It provides the same overall
+installation experience as the graphical installer, but has generally better
+compatibility with very old and very new hardware.
+
+Install {pve} (Terminal UI, Serial Console)::
+
+Starts the terminal-mode installation wizard, additionally setting up the Linux
+kernel to use the (first) serial port of the machine for in- and output. This
+can be used if the machine is completely headless and only has a serial console
+available.
+
+[thumbnail="screenshot/pve-tui-installer.png"]
+
+Both modes use the same code base for the actual installation process to
+benefit from more than a decade of bug fixes and ensure feature parity.
+
+TIP: The 'Terminal UI' option can be used in case the graphical installer does
+not work correctly, due to e.g. driver issues. See also
+xref:nomodeset_kernel_param[adding the `nomodeset` kernel parameter].
+
+Advanced Options: Install {pve} (Graphical, Debug Mode)::
+
+Starts the installation in debug mode. A console will be opened at several
+installation steps. This helps to debug the situation if something goes wrong.
+To exit a debug console, press `CTRL-D`. This option can be used to boot a live
+system with all basic tools available. You can use it, for example, to
+xref:chapter_zfs[repair a degraded ZFS 'rpool'] or fix the
+xref:sysboot[bootloader] for an existing {pve} setup.
+
+Advanced Options: Install {pve} (Terminal UI, Debug Mode)::
+
+Same as the graphical debug mode, but preparing the system to run the
+terminal-based installer instead.
+
+Advanced Options: Install {pve} (Serial Console Debug Mode)::
+
+Same the terminal-based debug mode, but additionally sets up the Linux kernel
to
+use the (first) serial port of the machine for in- and output.
+
+Advanced Options: Rescue Boot::
+
+With this option you can boot an existing installation. It searches all
attached
+hard disks. If it finds an existing installation, it boots directly into that
+disk using the Linux kernel from the ISO. This can be useful if there are
+problems with the bootloader (GRUB/`systemd-boot`) or the BIOS/UEFI is unable
to
+read the boot block from the disk.
+
+Advanced Options: Test Memory (memtest86+)::
+
+Runs `memtest86+`. This is useful to check if the memory is functional and free
+of errors. Secure Boot must be turned off in the UEFI firmware setup utility to
+run this option.
+
+You normally select *Install {pve} (Graphical)* to start the installation.
+
+[thumbnail="screenshot/pve-select-target-disk.png"]
+
+The first step is to read our EULA (End User License Agreement). Following
this,
+you can select the target hard disk(s) for the installation.
+
+CAUTION: By default, the whole server is used and all existing data is removed.
+Make sure there is no important data on the server before proceeding with the
+installation.
+
+The `Options` button lets you select the target file system, which
+defaults to `ext4`. The installer uses LVM if you select
+`ext4` or `xfs` as a file system, and offers additional options to
+restrict LVM space (see xref:advanced_lvm_options[below]).
+
+{pve} can also be installed on ZFS. As ZFS offers several software RAID levels,
+this is an option for systems that don't have a hardware RAID controller. The
+target disks must be selected in the `Options` dialog. More ZFS specific
+settings can be changed under xref:advanced_zfs_options[`Advanced Options`].
+
+WARNING: ZFS on top of any hardware RAID is not supported and can result in
data
+loss.
+
+[th
[pve-devel] [RFC PATCH docs{, -common} 0/13] introduce common documentation base
tl;dr: Introduce a separate repository for shared documentation between
all three products.
This proposes the introduction of a - aptly named - proxmox-docs-common
repo, which can be used for all documentation not specific to a single
product.
Marked RFC to gather feedback from other people. After talking to some
people off-list, there definitely is the need for something like this in
the long run.
For now, this is pve-docs only, with only one page and two partials (see
also "Organization"). But it shows the purpose and general approach well
enough to gather some first feedback.
Motivation
==
A lot of pages and sections esp. from pve-docs apply to all three
products. To name a few; package repos, (firmware) updates, network
config, ZFS host administration, bootloader etc. etc.
Further, the documentation for the auto-installer from Aaron (now in the
wiki) and for the notification system from Lukas would also be good fit
for proxmox-docs-common.
The plan would be to gradually move them all into proxmox-docs-common as
appropriate, thus de-duplicating and having them available on all three
products without the headaches of copying & keeping them in sync.
Most of the common documentation is now done via the wiki, but this
approach would allow "codifying" these things - generating wiki pages
from our documentation is a solved problem already.
Organization
Pretty simple; all files in the repo root of proxmox-docs-common
represent "full pages", i.e. starting with a level-1 heading, much like
in pve-docs. These must still each be included somewhere, e.g. for
pve-docs the appropriate subpage or in `pve-admin-guide.adoc`.
Additionally, there is the `partials/` folder, which contains exactly
that - these are sections of pages (aka. with no level-1 heading) that
can be shared, but where the whole page might might not be suitable for
sharing. These can be then included as usual using
include::proxmox-docs-common/partials/foo.adoc[]
The distinction is only for organization in the repo itself, but
otherwise has no special/hidden meaning. So this can easily be
changed/dropped if desired.
Future works
- Integrating into pmg-docs
- Integrating ReStructuredText
- Integrating into proxmox-backup documentation
- Moving more pages and sections into it, of course - lots to be done
there
- Unifying the `scan-adoc-refs` tool from {pve,pmg}-docs in the common
repo
AsciiDoc vs. ReStructuredText
=
Currently, this is AsciiDoc only.
My proposal would be to keep all exisiting AsciiDoc pages as such and
only adapt as necessary. Since it is possible to automatically convert
between AsciiDoc and ReStructuredText, this means we can reuse
existing AsciiDoc-written pages/partials in PBS and write new pages in
ReStructuredText.
The latter is not included in this series yet, but would be a follow-up
in the future, to keep this initial change as simple as possible.
[ Of course, one could also think about mechanically converting all
AsciiDoc-written pages to ReStructuredText, but that's a whole
different story and harder to get right. ]
Diffstats
=
Looks a bit scary, but the bulk of it are simply renames & pulling
sections out into different files. For the latter, there is always a
commit for copying the content 1:1 and one adapting small things as
necessary.
pve-docs:
Christoph Heiss (7):
gitmodules: add proxmox-docs-common
scan-adoc-refs: enable building pages from proxmox-docs-common/ subdir
images: strip `pve-` prefix from screenshots used in common docs
asciidoc: conf: add iso-url variable
installation-media: move to common docs
installation: use new 'installation-flow' partial from common docs
installation: use new 'advanced-installation' partial from common docs
.gitmodules | 3 +
Makefile | 6 +-
asciidoc/asciidoc-pve.conf| 3 +
.../{pve-grub-menu.png => grub-menu.png} | Bin
.../{pve-grub-menu.ppm => grub-menu.ppm} | Bin
...nstall-summary.png => install-summary.png} | Bin
...nstall-summary.ppm => install-summary.ppm} | Bin
...{pve-installation.png => installation.png} | Bin
...{pve-installation.ppm => installation.ppm} | Bin
...elect-location.png => select-location.png} | Bin
...elect-location.ppm => select-location.ppm} | Bin
...target-disk.png => select-target-disk.png} | Bin
...target-disk.ppm => select-target-disk.ppm} | Bin
...{pve-set-password.png => set-password.png} | Bin
...{pve-set-password.ppm => set-password.ppm} | Bin
...ve-setup-network.png => setup-network.png} | Bin
...ve-setup-network.ppm => setup-network.ppm} | Bin
...ve-tui-installer.png => tui-installer.png} | Bin
...ve-tui-installer.ppm => tui-installer.ppm} | Bin
png-verify.pl | 14 +-
proxmox-docs-common | 1 +
pve-doc-generator.mk.in | 6 +
pve-inst
[pve-devel] [RFC PATCH docs-common 01/13] installation-media: move page from pve-docs here
Small adaptions were necessary; mostly a s/{pve}/{product}/g and
replacing the ISO URL with the {iso-url} variable.
Signed-off-by: Christoph Heiss
---
installation-media.adoc | 132
1 file changed, 132 insertions(+)
create mode 100644 installation-media.adoc
diff --git a/installation-media.adoc b/installation-media.adoc
new file mode 100644
index 000..a1c9402
--- /dev/null
+++ b/installation-media.adoc
@@ -0,0 +1,132 @@
+[[installation_prepare_media]]
+Prepare Installation Media
+--
+ifdef::wiki[]
+:pve-toplevel:
+endif::wiki[]
+
+Download the installer ISO image from:
{website}en/downloads/proxmox-virtual-environment/iso
+
+The {pve} installation media is a hybrid ISO image. It works in two ways:
+
+* An ISO image file ready to burn to a CD or DVD.
+
+* A raw sector (IMG) image file ready to copy to a USB flash drive (USB stick).
+
+Using a USB flash drive to install {pve} is the recommended way because it is
+the faster option.
+
+Prepare a USB Flash Drive as Installation Medium
+
+
+The flash drive needs to have at least 1 GB of storage available.
+
+NOTE: Do not use UNetbootin. It does not work with the {pve} installation
image.
+
+IMPORTANT: Make sure that the USB flash drive is not mounted and does not
+contain any important data.
+
+
+Instructions for GNU/Linux
+~~
+
+On Unix-like operating system use the `dd` command to copy the ISO image to the
+USB flash drive. First find the correct device name of the USB flash drive (see
+below). Then run the `dd` command.
+
+
+# dd bs=1M conv=fdatasync if=./proxmox-ve_*.iso of=/dev/XYZ
+
+
+NOTE: Be sure to replace /dev/XYZ with the correct device name and adapt the
+input filename ('if') path.
+
+CAUTION: Be very careful, and do not overwrite the wrong disk!
+
+
+Find the Correct USB Device Name
+
+There are two ways to find out the name of the USB flash drive. The first one
is
+to compare the last lines of the `dmesg` command output before and after
+plugging in the flash drive. The second way is to compare the output of the
+`lsblk` command. Open a terminal and run:
+
+
+# lsblk
+
+
+Then plug in your USB flash drive and run the command again:
+
+
+# lsblk
+
+
+A new device will appear. This is the one you want to use. To be on the extra
+safe side check if the reported size matches your USB flash drive.
+
+
+Instructions for macOS
+~~
+
+Open the terminal (query Terminal in Spotlight).
+
+Convert the `.iso` file to `.dmg` format using the convert option of `hdiutil`,
+for example:
+
+
+# hdiutil convert proxmox-ve_*.iso -format UDRW -o proxmox-ve_*.dmg
+
+
+TIP: macOS tends to automatically add '.dmg' to the output file name.
+
+To get the current list of devices run the command:
+
+
+# diskutil list
+
+
+Now insert the USB flash drive and run this command again to determine which
+device node has been assigned to it. (e.g., /dev/diskX).
+
+
+# diskutil list
+# diskutil unmountDisk /dev/diskX
+
+
+NOTE: replace X with the disk number from the last command.
+
+
+# sudo dd if=proxmox-ve_*.dmg bs=1M of=/dev/rdiskX
+
+
+NOTE: 'rdiskX', instead of 'diskX', in the last command is intended. It will
+increase the write speed.
+
+Instructions for Windows
+
+
+Using Etcher
+
+
+Etcher works out of the box. Download Etcher from https://etcher.io. It will
+guide you through the process of selecting the ISO and your USB flash drive.
+
+Using Rufus
+^^^
+
+Rufus is a more lightweight alternative, but you need to use the *DD mode* to
+make it work. Download Rufus from https://rufus.ie/. Either install it or use
+the portable version. Select the destination drive and the {pve} ISO file.
+
+IMPORTANT: Once you 'Start' you have to click 'No' on the dialog asking to
+download a different version of GRUB. In the next dialog select the 'DD' mode.
+
+ifdef::wiki[]
+Boot your Server from the USB Flash Drive
+~
+
+Connect the USB flash drive to your server and make sure that booting from USB
+is enabled (check your servers firmware settings). Then follow the steps in the
+xref:chapter_installation[installation wizard].
+
+endif::wiki[]
--
2.44.0
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [RFC PATCH docs-common 02/13] installation-media: adapt for common usage
Small adaptions were necessary; mostly a trivial s/{pve}/{product}/g and
replacing the ISO URL with the {iso-url} variable.
Signed-off-by: Christoph Heiss
---
installation-media.adoc | 21 -
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/installation-media.adoc b/installation-media.adoc
index a1c9402..bcef51c 100644
--- a/installation-media.adoc
+++ b/installation-media.adoc
@@ -1,27 +1,30 @@
[[installation_prepare_media]]
Prepare Installation Media
--
+ifdef::pve[]
ifdef::wiki[]
:pve-toplevel:
endif::wiki[]
+endif::pve[]
-Download the installer ISO image from:
{website}en/downloads/proxmox-virtual-environment/iso
+Download the installer ISO image from: {iso-url}
-The {pve} installation media is a hybrid ISO image. It works in two ways:
+The {product} installation media is a hybrid ISO image. It works in two ways:
* An ISO image file ready to burn to a CD or DVD.
* A raw sector (IMG) image file ready to copy to a USB flash drive (USB stick).
-Using a USB flash drive to install {pve} is the recommended way because it is
-the faster option.
+Using a USB flash drive to install {product} is the recommended way because it
+is the faster option.
Prepare a USB Flash Drive as Installation Medium
The flash drive needs to have at least 1 GB of storage available.
-NOTE: Do not use UNetbootin. It does not work with the {pve} installation
image.
+NOTE: Do not use UNetbootin. It does not work with the {product} installation
+image.
IMPORTANT: Make sure that the USB flash drive is not mounted and does not
contain any important data.
@@ -35,7 +38,7 @@ USB flash drive. First find the correct device name of the
USB flash drive (see
below). Then run the `dd` command.
-# dd bs=1M conv=fdatasync if=./proxmox-ve_*.iso of=/dev/XYZ
+# dd bs=1M conv=fdatasync if=./proxmox-*.iso of=/dev/XYZ
NOTE: Be sure to replace /dev/XYZ with the correct device name and adapt the
@@ -74,7 +77,7 @@ Convert the `.iso` file to `.dmg` format using the convert
option of `hdiutil`,
for example:
-# hdiutil convert proxmox-ve_*.iso -format UDRW -o proxmox-ve_*.dmg
+# hdiutil convert proxmox-*.iso -format UDRW -o proxmox-ve_*.dmg
TIP: macOS tends to automatically add '.dmg' to the output file name.
@@ -96,7 +99,7 @@ device node has been assigned to it. (e.g., /dev/diskX).
NOTE: replace X with the disk number from the last command.
-# sudo dd if=proxmox-ve_*.dmg bs=1M of=/dev/rdiskX
+# sudo dd if=proxmox-*.dmg bs=1M of=/dev/rdiskX
NOTE: 'rdiskX', instead of 'diskX', in the last command is intended. It will
@@ -116,7 +119,7 @@ Using Rufus
Rufus is a more lightweight alternative, but you need to use the *DD mode* to
make it work. Download Rufus from https://rufus.ie/. Either install it or use
-the portable version. Select the destination drive and the {pve} ISO file.
+the portable version. Select the destination drive and the {product} ISO file.
IMPORTANT: Once you 'Start' you have to click 'No' on the dialog asking to
download a different version of GRUB. In the next dialog select the 'DD' mode.
--
2.44.0
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system
Am 19.04.24 um 10:45 schrieb Lukas Wagner:
>> Who adds the template files? I don't see a patch for proxmox-ve in this
>> series. Does this series require some versioned breaks to some package?
>
> The pve-manager and pve-ha-manager (for fencing notifications) patches add
> the templates.
> I can't use `/usr/share/pve-manager` and `/usr/share/pve-ha-manager` because
> proxmox_notify needs to have a single base directory from where to load
> template files.
> Maybe we should use some other base dir to avoid confusion with the
> `proxmox-ve` metapackage?
>
Ah, I see. Yes, maybe a directory named based on libpve-notify-perl
would be better or proxmox-notify (but would need to be a bit careful
with co-installed PBS and PVE to not create accidental conflicts).
> In terms of versions:
> pve-{ha}-manager needs to pull in a bumped libpve-notify-perl
> libpve-notify-perl needs to pull in bumped libpve-rs-perl/libproxmox-rs-perl
> libpve-rs-perl needs to pull in bumped librust-proxmox-notify
>
> I really wish the dep-chain was a bit easier, yet here we are.
>
But there also is a need for versioned breaks, right? Because installing
new libpve-notify-perl (using new proxmox-perl-rs using new
proxmox-notify) without also upgrading pve-manager and pve-ha-manager
will be broken or am I missing something?
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH manager 7/7] report: add recent boot timestamps which may show fencing/crash events
On 4/18/24 17:45, Alexander Zeidler wrote:
> On Thu, 2024-04-18 at 12:43 +0200, Mira Limbeck wrote:
>> On 4/18/24 11:16, Alexander Zeidler wrote:
>>> Successful boots which crashed somehow and sometime afterwards, will
>>> show the same "until" value ("still running" or timestamp) as the next
>>> following boot(s). The most recent boot from such a sequence of
>>> duplicated "until" lines, has not been crashed or not yet.
>>>
>>> Example output where only the boot from 16:25:41 crashed:
>>> reboot system boot 6.5.11-7-pve Thu Apr 11 16:31:24 2024 still running
>>> reboot system boot 6.5.11-7-pve Thu Apr 11 16:29:17 2024 - Thu Apr 11
>>> 16:31:12 2024 (00:01)
>>> reboot system boot 6.5.11-7-pve Thu Apr 11 16:25:41 2024 - Thu Apr 11
>>> 16:31:12 2024 (00:05)
>>> ...
>>>
>>> Furthermore, it shows the booted/crashed/problematic kernel version.
>>>
>>> `last` is also used since currently `journalctl --list-boots` can take
>>> 10 seconds or even longer on some systems, with no option to limit the
>>> amount of reported boot lines.
>>>
>>> Signed-off-by: Alexander Zeidler
>>> ---
>>> v2:
>>> * move away from dmesg base
>>> * list also recent (5) boot timestamps with additional information
>>>
>>> v1: https://lists.proxmox.com/pipermail/pve-devel/2024-March/062342.html
>>>
>>>
>>> PVE/Report.pm | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/PVE/Report.pm b/PVE/Report.pm
>>> index d9f81a0f..c3abb776 100644
>>> --- a/PVE/Report.pm
>>> +++ b/PVE/Report.pm
>>> @@ -32,6 +32,7 @@ my $init_report_cmds = sub {
>>> 'hostname',
>>> 'date -R',
>>> 'cat /proc/cmdline',
>>> + 'last reboot -F -n5',
>>> 'pveversion --verbose',
>>> 'cat /etc/hosts',
>>> 'pvesubscription get',
>>
>> Do we want the reboot info that far up, even above the version output?
>> I'd say it's less interesting most of the time than the `pveversion` output.
>
> I'm not sure if it really fits better with your suggestion. Because, while
> the pveversion output can be considered as often more relevant, I have placed
> it as it is because it fits well with the surrounding information:
>
> * You can see/compare the booted kernel versions to the kernel command line
> and pveversion output.
>
> * For the kernel command line it makes rather sense to have it at the
> beginning of the report.
The kernel command line makes sense up there. I agree.
But the reboots are often less interesting/important than the pveversion
output.
So I'd prefer the pveversion output to stay as far up as possible (after
hostname, date and cmdline).
>> * Also it may be interesting how frequent the host is rebooted (e.g. after
> kernel updates)
>
>
> Btw. the "wtmp begins ..." output does not have to be the installation date.
> In case we do not store this information somewhere, currently something like
>
> stat / | grep Birth
>
> could be used if needed>
>>
>> And for uptime, we do have /cluster/resources and `top` which both show it.
>> Maybe it could be moved a bit further down? After /cluster/resources
>> could perhaps be a nice spot since it is (currently) followed by `top`?
>
>
>
> ___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system
On 2024-04-19 10:14, Fiona Ebner wrote:
> Am 09.04.24 um 15:25 schrieb Lukas Wagner:
>> Instead of passing the template strings for subject and body when
>> constructing a notification, we pass only the name of a template.
>> When rendering the template, the name of the template is used to find
>> corresponding template files. For PVE, they are located at
>> /usr/share/proxmox-ve/templates/default. The `default` part is
>> the 'template namespace', which is a preparation for user-customizable
>> and/or translatable notifications.
>>
>
> Is the plan to create different namespaces there ourselves or tell users
> they can put their custom templates there? In the latter case, I'm not
> sure /usr/share is the best place, rather than some place under /etc/
The idea would be to implement translations as other namespaces, e.g. `de` or
`fr`
instead of `default`.
For user-overridable templates we would extend the implementation to search for
a template
in another location first and then fall back to the templates provided by us in
'/usr/share/...`
I have not made up my mind yet on where these user-provided templates would be
located, either
in /usr/local/share/ or somewhere in /etc (/etc/pve would ensure that we
have the same templates
on all nodes, but I'm not sure if it is a good idea to put custom, user-created
files in there...)
Combining both ideas: assuming that we want to render a fencing notification
translated to German, assuming
that the user-override is in /usr/local/share:
- First try to load /usr/local/share/proxmox-ve/templates/de/fencing.txt.hbs
- If not found, try loading the shipped template at
/usr/share/proxmox-ve/templates/de/...
- If that one also does not exist, fall back to `default` namespace
- first user-location
- finally shipped template
>
> Who adds the template files? I don't see a patch for proxmox-ve in this
> series. Does this series require some versioned breaks to some package?
The pve-manager and pve-ha-manager (for fencing notifications) patches add the
templates.
I can't use `/usr/share/pve-manager` and `/usr/share/pve-ha-manager` because
proxmox_notify needs to have a single base directory from where to load
template files.
Maybe we should use some other base dir to avoid confusion with the
`proxmox-ve` metapackage?
In terms of versions:
pve-{ha}-manager needs to pull in a bumped libpve-notify-perl
libpve-notify-perl needs to pull in bumped libpve-rs-perl/libproxmox-rs-perl
libpve-rs-perl needs to pull in bumped librust-proxmox-notify
I really wish the dep-chain was a bit easier, yet here we are.
--
- Lukas
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 09/19] notify: derive Deserialize/Serialize for Notification struct
Nit: I always like a quick sentence for who needs it for such changes.
Am 09.04.24 um 15:25 schrieb Lukas Wagner:
> Signed-off-by: Lukas Wagner
> ---
> proxmox-notify/src/lib.rs | 10 +++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/proxmox-notify/src/lib.rs b/proxmox-notify/src/lib.rs
> index 91c0b61..8d4dc63 100644
> --- a/proxmox-notify/src/lib.rs
> +++ b/proxmox-notify/src/lib.rs
> @@ -159,11 +159,13 @@ pub trait Endpoint {
> fn disabled(&self) -> bool;
> }
>
> -#[derive(Debug, Clone)]
> +#[derive(Debug, Clone, Serialize, Deserialize)]
> +#[serde(rename_all = "kebab-case")]
> pub enum Content {
> /// Title and body will be rendered as a template
> Template {
> /// Name of the used template
> +#[serde(rename = "template-name")]
So I guess this is here, because the rename_all above is not recursive?
Should we put rename_all on top of Template and ForwardedMail (if that
even works), so we are sure not to forget it for potential future fields?
> template_name: String,
> /// Data that can be used for template rendering.
> data: Value,
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 07/19] notify: api: add get_targets
Am 09.04.24 um 15:25 schrieb Lukas Wagner:
> +#[api]
> +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq, PartialOrd)]
> +#[serde(rename_all = "kebab-case")]
> +/// Target information
> +pub struct Target {
> +/// Name of the endpoint
> +name: String,
> +/// Origin of the endpoint
> +origin: Origin,
> +/// Type of the endpoint
> +#[serde(rename = "type")]
> +endpoint_type: EndpointType,
> +/// Target is disabled
Oh, and don't we want:
#[serde(skip_serializing_if = "Option::is_none")]
here?
> +disable: Option,
> +/// Comment
> +comment: Option,
> +}
> +
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 07/19] notify: api: add get_targets
Am 09.04.24 um 15:25 schrieb Lukas Wagner:
> +/// Get a list of all notification targets.
> +pub fn get_targets(config: &Config) -> Result, HttpError> {
> +let mut targets = Vec::new();
> +
> +#[cfg(feature = "gotify")]
> +for endpoint in gotify::get_endpoints(config)? {
> +targets.push(Target {
> +name: endpoint.name,
> +origin: endpoint.origin.unwrap_or(Origin::UserCreated),
> +endpoint_type: EndpointType::Gotify,
> +disable: endpoint.disable,
> +comment: endpoint.comment,
> +})
Would it make sense to have into() functions for
{Gotify,Sendmail,Smtp}Config -> Target ?
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 05/19] notify: make the `mail-forwarder` feature depend on proxmox-sys
Am 09.04.24 um 15:25 schrieb Lukas Wagner:
> It uses proxmox_sys::nodename - the dep is needed, otherwise the code
> does not compile in some feature flag permutations.
>
> Signed-off-by: Lukas Wagner
> Tested-by: Folke Gleumes
> ---
> proxmox-notify/Cargo.toml | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
> index 52a466e..185b50a 100644
> --- a/proxmox-notify/Cargo.toml
> +++ b/proxmox-notify/Cargo.toml
> @@ -17,13 +17,13 @@ log.workspace = true
> mail-parser = { workspace = true, optional = true }
> openssl.workspace = true
> regex.workspace = true
> -serde = { workspace = true, features = ["derive"]}
> +serde = { workspace = true, features = ["derive"] }
Nit: unrelated whitespace changes should be their own patch
> serde_json.workspace = true
>
> proxmox-http = { workspace = true, features = ["client-sync"], optional =
> true }
> proxmox-http-error.workspace = true
> proxmox-human-byte.workspace = true
> -proxmox-schema = { workspace = true, features = ["api-macro", "api-types"]}
> +proxmox-schema = { workspace = true, features = ["api-macro", "api-types"] }
> proxmox-section-config = { workspace = true }
> proxmox-serde.workspace = true
> proxmox-sys = { workspace = true, optional = true }
> @@ -31,7 +31,7 @@ proxmox-time.workspace = true
>
> [features]
> default = ["sendmail", "gotify", "smtp"]
> -mail-forwarder = ["dep:mail-parser"]
> +mail-forwarder = ["dep:mail-parser", "dep:proxmox-sys"]
> sendmail = ["dep:proxmox-sys"]
> gotify = ["dep:proxmox-http"]
> pve-context = ["dep:proxmox-sys"]
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH proxmox 01/19] notify: switch to file-based templating system
Am 09.04.24 um 15:25 schrieb Lukas Wagner: > Instead of passing the template strings for subject and body when > constructing a notification, we pass only the name of a template. > When rendering the template, the name of the template is used to find > corresponding template files. For PVE, they are located at > /usr/share/proxmox-ve/templates/default. The `default` part is > the 'template namespace', which is a preparation for user-customizable > and/or translatable notifications. > Is the plan to create different namespaces there ourselves or tell users they can put their custom templates there? In the latter case, I'm not sure /usr/share is the best place, rather than some place under /etc/ Who adds the template files? I don't see a patch for proxmox-ve in this series. Does this series require some versioned breaks to some package? ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH proxmox-firewall v3 34/39] firewall: add integration test
Signed-off-by: Stefan Hanreich
---
.gitignore|1 +
debian/control|1 +
proxmox-firewall/Cargo.toml |4 +
proxmox-firewall/tests/input/100.conf | 10 +
proxmox-firewall/tests/input/100.fw | 22 +
proxmox-firewall/tests/input/101.conf | 11 +
proxmox-firewall/tests/input/101.fw | 19 +
proxmox-firewall/tests/input/chains.json | 427 ++
proxmox-firewall/tests/input/cluster.fw | 26 +
proxmox-firewall/tests/input/host.fw | 23 +
proxmox-firewall/tests/integration_tests.rs | 90 +
.../integration_tests__firewall.snap | 3530 +
12 files changed, 4164 insertions(+)
create mode 100644 proxmox-firewall/tests/input/100.conf
create mode 100644 proxmox-firewall/tests/input/100.fw
create mode 100644 proxmox-firewall/tests/input/101.conf
create mode 100644 proxmox-firewall/tests/input/101.fw
create mode 100644 proxmox-firewall/tests/input/chains.json
create mode 100644 proxmox-firewall/tests/input/cluster.fw
create mode 100644 proxmox-firewall/tests/input/host.fw
create mode 100644 proxmox-firewall/tests/integration_tests.rs
create mode 100644
proxmox-firewall/tests/snapshots/integration_tests__firewall.snap
diff --git a/.gitignore b/.gitignore
index 90749ee..c5474ef 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@ proxmox-firewall-*/
*.build
*.buildinfo
*.changes
+*.snap.new
diff --git a/debian/control b/debian/control
index 97f9e89..845b84d 100644
--- a/debian/control
+++ b/debian/control
@@ -20,6 +20,7 @@ Build-Depends: cargo:native,
librust-thiserror-dev,
librust-libc-0.2+default-dev,
librust-proxmox-schema-3+default-dev,
+ librust-insta-dev,
libstd-rust-dev,
netbase,
python3,
diff --git a/proxmox-firewall/Cargo.toml b/proxmox-firewall/Cargo.toml
index bec7552..163ab17 100644
--- a/proxmox-firewall/Cargo.toml
+++ b/proxmox-firewall/Cargo.toml
@@ -22,3 +22,7 @@ signal-hook = "0.3"
proxmox-nftables = { path = "../proxmox-nftables", features = ["config-ext"] }
proxmox-ve-config = { path = "../proxmox-ve-config" }
+
+[dev-dependencies]
+insta = { version = "1.21", features = ["json"] }
+proxmox-sys = "0.5.3"
diff --git a/proxmox-firewall/tests/input/100.conf
b/proxmox-firewall/tests/input/100.conf
new file mode 100644
index 000..495f899
--- /dev/null
+++ b/proxmox-firewall/tests/input/100.conf
@@ -0,0 +1,10 @@
+arch: amd64
+cores: 1
+features: nesting=1
+hostname: host1
+memory: 512
+net1:
name=eth0,bridge=simple1,firewall=1,hwaddr=BC:24:11:4D:B0:FF,ip=dhcp,ip6=fd80::1234/64,type=veth
+ostype: debian
+rootfs: local-lvm:vm-90001-disk-0,size=2G
+swap: 512
+unprivileged: 1
diff --git a/proxmox-firewall/tests/input/100.fw
b/proxmox-firewall/tests/input/100.fw
new file mode 100644
index 000..6cf9fff
--- /dev/null
+++ b/proxmox-firewall/tests/input/100.fw
@@ -0,0 +1,22 @@
+[OPTIONS]
+
+enable: 1
+ndp: 1
+ipfilter: 1
+dhcp: 1
+log_level_in: crit
+log_level_out: alert
+policy_in: DROP
+policy_out: REJECT
+macfilter: 0
+
+[IPSET ipfilter-net1]
+
+dc/network1
+
+[RULES]
+
+GROUP network1 -i net1
+IN ACCEPT -source 192.168.0.1/24,127.0.0.1-127.255.255.0,172.16.0.1 -dport
123,222:333 -sport http -p tcp
+IN DROP --icmp-type echo-request --proto icmp --log info
+
diff --git a/proxmox-firewall/tests/input/101.conf
b/proxmox-firewall/tests/input/101.conf
new file mode 100644
index 000..394e2e4
--- /dev/null
+++ b/proxmox-firewall/tests/input/101.conf
@@ -0,0 +1,11 @@
+boot: order=ide2
+cores: 2
+cpu: x86-64-v2-AES
+memory: 2048
+meta: creation-qemu=8.1.5,ctime=1712322773
+numa: 0
+ostype: l26
+scsihw: virtio-scsi-single
+smbios1: uuid=78ec7794-78f7-4c03-bf08-18b721a6
+sockets: 1
+vmgenid: ec7d4834-cd0a-4376-9c1d-af8a82da8d54
diff --git a/proxmox-firewall/tests/input/101.fw
b/proxmox-firewall/tests/input/101.fw
new file mode 100644
index 000..c77cb5a
--- /dev/null
+++ b/proxmox-firewall/tests/input/101.fw
@@ -0,0 +1,19 @@
+[OPTIONS]
+
+ndp: 0
+enable: 1
+dhcp: 1
+radv: 0
+policy_out: ACCEPT
+
+[ALIASES]
+
+analias 123.123.123.123
+
+[IPSET testing]
+
+
+[RULES]
+
+IN ACCEPT -source guest/analias -dest dc/network2 -log nolog
+
diff --git a/proxmox-firewall/tests/input/chains.json
b/proxmox-firewall/tests/input/chains.json
new file mode 100644
index 000..aabfc6e
--- /dev/null
+++ b/proxmox-firewall/tests/input/chains.json
@@ -0,0 +1,427 @@
+{
+ "nftables": [
+{
+ "metainfo": {
+"version": "1.0.6",
+"release_name": "Lester Gooch #5",
+"json_schema_version": 1
+ }
+},
+{
+ "chain": {
+"family": "inet",
+"table": "proxmox-firewall",
+"name": "do-reject",
+"handle": 1
+ }
+},
+{
+ "chain": {
+"family": "inet",
+"table": "proxmox-firewall",
+"name": "accept-mana
[pve-devel] [PATCH pve-firewall v3 37/39] add configuration option for new nftables firewall
Introduces new nftables configuration option that en/disables the new
nftables firewall.
pve-firewall reads this option and only generates iptables rules when
nftables is set to `0`. Conversely proxmox-firewall only generates
nftables rules when the option is set to `1`.
Signed-off-by: Stefan Hanreich
---
src/PVE/Firewall.pm | 20
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 81a8798..b39843d 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -1408,6 +1408,12 @@ our $host_option_properties = {
default => 0,
optional => 1
},
+nftables => {
+ description => "Enable nftables based firewall",
+ type => 'boolean',
+ default => 0,
+ optional => 1,
+},
};
our $vm_option_properties = {
@@ -2929,7 +2935,7 @@ sub parse_hostfw_option {
my $loglevels = "emerg|alert|crit|err|warning|notice|info|debug|nolog";
-if ($line =~
m/^(enable|nosmurfs|tcpflags|ndp|log_nf_conntrack|nf_conntrack_allow_invalid|protection_synflood):\s*(0|1)\s*$/i)
{
+if ($line =~
m/^(enable|nosmurfs|tcpflags|ndp|log_nf_conntrack|nf_conntrack_allow_invalid|protection_synflood|nftables):\s*(0|1)\s*$/i)
{
$opt = lc($1);
$value = int($2);
} elsif ($line =~
m/^(log_level_in|log_level_out|tcp_flags_log_level|smurf_log_level):\s*(($loglevels)\s*)?$/i)
{
@@ -4676,7 +4682,11 @@ sub remove_pvefw_chains_ebtables {
sub init {
my $cluster_conf = load_clusterfw_conf();
my $cluster_options = $cluster_conf->{options};
-my $enable = $cluster_options->{enable};
+
+my $host_conf = load_hostfw_conf($cluster_conf);
+my $host_options = $host_conf->{options};
+
+my $enable = $cluster_options->{enable} && !$host_options->{nftables};
return if !$enable;
@@ -4689,12 +4699,14 @@ sub update {
my $cluster_conf = load_clusterfw_conf();
my $cluster_options = $cluster_conf->{options};
- if (!$cluster_options->{enable}) {
+ my $hostfw_conf = load_hostfw_conf($cluster_conf);
+ my $host_options = $hostfw_conf->{options};
+
+ if (!$cluster_options->{enable} || $host_options->{nftables}) {
PVE::Firewall::remove_pvefw_chains();
return;
}
- my $hostfw_conf = load_hostfw_conf($cluster_conf);
my ($ruleset, $ipset_ruleset, $rulesetv6, $ebtables_ruleset) =
compile($cluster_conf, $hostfw_conf);
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH proxmox-firewall v3 19/39] nftables: expression: add types
Adds an enum containing most of the expressions defined in the
nftables-json schema [1].
[1]
https://manpages.debian.org/bookworm/libnftables1/libnftables-json.5.en.html#EXPRESSIONS
Reviewed-by: Lukas Wagner
Reviewed-by: Max Carrara
Co-authored-by: Wolfgang Bumiller
Signed-off-by: Stefan Hanreich
---
proxmox-nftables/Cargo.toml| 2 +-
proxmox-nftables/src/expression.rs | 268 +
proxmox-nftables/src/lib.rs| 4 +
proxmox-nftables/src/types.rs | 53 ++
4 files changed, 326 insertions(+), 1 deletion(-)
create mode 100644 proxmox-nftables/src/expression.rs
create mode 100644 proxmox-nftables/src/types.rs
diff --git a/proxmox-nftables/Cargo.toml b/proxmox-nftables/Cargo.toml
index ebece9d..909869b 100644
--- a/proxmox-nftables/Cargo.toml
+++ b/proxmox-nftables/Cargo.toml
@@ -17,4 +17,4 @@ serde = { version = "1", features = [ "derive" ] }
serde_json = "1"
serde_plain = "1"
-proxmox-ve-config = { path = "../proxmox-ve-config", optional = true }
+proxmox-ve-config = { path = "../proxmox-ve-config" }
diff --git a/proxmox-nftables/src/expression.rs
b/proxmox-nftables/src/expression.rs
new file mode 100644
index 000..5478291
--- /dev/null
+++ b/proxmox-nftables/src/expression.rs
@@ -0,0 +1,268 @@
+use crate::types::{ElemConfig, Verdict};
+use serde::{Deserialize, Serialize};
+use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
+
+use crate::helper::NfVec;
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+#[serde(rename_all = "lowercase")]
+pub enum Expression {
+Concat(Vec),
+Set(Vec),
+Range(Box<(Expression, Expression)>),
+Map(Box),
+Prefix(Prefix),
+Payload(Payload),
+Meta(Meta),
+Ct(Ct),
+Elem(Box),
+
+#[serde(rename = "|")]
+Or(Box<(Expression, Expression)>),
+#[serde(rename = "&")]
+And(Box<(Expression, Expression)>),
+#[serde(rename = "^")]
+Xor(Box<(Expression, Expression)>),
+#[serde(rename = "<<")]
+ShiftLeft(Box<(Expression, Expression)>),
+#[serde(rename = ">>")]
+ShiftRight(Box<(Expression, Expression)>),
+
+#[serde(untagged)]
+List(Vec),
+
+#[serde(untagged)]
+Verdict(Verdict),
+
+#[serde(untagged)]
+Bool(bool),
+#[serde(untagged)]
+Number(i64),
+#[serde(untagged)]
+String(String),
+}
+
+impl Expression {
+pub fn set(expressions: impl IntoIterator) -> Self {
+Expression::Set(Vec::from_iter(expressions))
+}
+
+pub fn concat(expressions: impl IntoIterator) -> Self {
+Expression::Concat(Vec::from_iter(expressions))
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(v: bool) -> Self {
+Expression::Bool(v)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(v: i64) -> Self {
+Expression::Number(v)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(v: u16) -> Self {
+Expression::Number(v.into())
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(v: u8) -> Self {
+Expression::Number(v.into())
+}
+}
+
+impl From<&str> for Expression {
+#[inline]
+fn from(v: &str) -> Self {
+Expression::String(v.to_string())
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(v: String) -> Self {
+Expression::String(v)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(meta: Meta) -> Self {
+Expression::Meta(meta)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(ct: Ct) -> Self {
+Expression::Ct(ct)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(payload: Payload) -> Self {
+Expression::Payload(payload)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(prefix: Prefix) -> Self {
+Expression::Prefix(prefix)
+}
+}
+
+impl From for Expression {
+#[inline]
+fn from(value: Verdict) -> Self {
+Expression::Verdict(value)
+}
+}
+
+impl From<&IpAddr> for Expression {
+fn from(value: &IpAddr) -> Self {
+Expression::String(value.to_string())
+}
+}
+
+impl From<&Ipv6Addr> for Expression {
+fn from(address: &Ipv6Addr) -> Self {
+Expression::String(address.to_string())
+}
+}
+
+impl From<&Ipv4Addr> for Expression {
+fn from(address: &Ipv4Addr) -> Self {
+Expression::String(address.to_string())
+}
+}
+
+#[derive(Clone, Copy, Debug, Eq, PartialEq, Deserialize, Serialize)]
+#[serde(rename_all = "lowercase")]
+pub enum IpFamily {
+Ip,
+Ip6,
+}
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+pub struct Meta {
+key: String,
+}
+
+impl Meta {
+pub fn new(key: impl Into) -> Self {
+Self { key: key.into() }
+}
+}
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+pub struct Map {
+key: Expression,
+data: Expression,
+}
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+pub struct Ct {
+key: String,
+#[serde(skip_serializing_if = "Option::is_none")]
+family: Option,
+#[
[pve-devel] [PATCH proxmox-firewall v3 31/39] firewall: add ruleset generation logic
We create the rules from the firewall config by utilizing the
ToNftRules and ToNftObjects traits to convert the firewall config
structs to nftables objects/chains/rules.
Reviewed-by: Lukas Wagner
Reviewed-by: Max Carrara
Co-authored-by: Wolfgang Bumiller
Signed-off-by: Stefan Hanreich
---
proxmox-firewall/Cargo.toml | 5 +
proxmox-firewall/src/firewall.rs | 899 +++
proxmox-firewall/src/main.rs | 1 +
3 files changed, 905 insertions(+)
create mode 100644 proxmox-firewall/src/firewall.rs
diff --git a/proxmox-firewall/Cargo.toml b/proxmox-firewall/Cargo.toml
index 431e71a..bec7552 100644
--- a/proxmox-firewall/Cargo.toml
+++ b/proxmox-firewall/Cargo.toml
@@ -15,5 +15,10 @@ log = "0.4"
env_logger = "0.10"
anyhow = "1"
+serde = { version = "1", features = [ "derive" ] }
+serde_json = "1"
+
+signal-hook = "0.3"
+
proxmox-nftables = { path = "../proxmox-nftables", features = ["config-ext"] }
proxmox-ve-config = { path = "../proxmox-ve-config" }
diff --git a/proxmox-firewall/src/firewall.rs b/proxmox-firewall/src/firewall.rs
new file mode 100644
index 000..2195a07
--- /dev/null
+++ b/proxmox-firewall/src/firewall.rs
@@ -0,0 +1,899 @@
+use std::collections::BTreeMap;
+use std::fs;
+
+use anyhow::Error;
+
+use proxmox_nftables::command::{Add, Commands, Delete, Flush};
+use proxmox_nftables::expression::{Meta, Payload};
+use proxmox_nftables::helper::NfVec;
+use proxmox_nftables::statement::{AnonymousLimit, Log, LogLevel, Match, Set,
SetOperation};
+use proxmox_nftables::types::{
+AddElement, AddRule, ChainPart, MapValue, RateTimescale, SetName,
TableFamily, TableName,
+TablePart, Verdict,
+};
+use proxmox_nftables::{Expression, Statement};
+
+use proxmox_ve_config::firewall::ct_helper::get_cthelper;
+use proxmox_ve_config::firewall::guest::Config as GuestConfig;
+use proxmox_ve_config::firewall::host::Config as HostConfig;
+
+use proxmox_ve_config::firewall::types::address::Ipv6Cidr;
+use proxmox_ve_config::firewall::types::ipset::{
+Ipfilter, Ipset, IpsetEntry, IpsetName, IpsetScope,
+};
+use proxmox_ve_config::firewall::types::log::{LogLevel as ConfigLogLevel,
LogRateLimit};
+use proxmox_ve_config::firewall::types::rule::{Direction, Verdict as
ConfigVerdict};
+use proxmox_ve_config::firewall::types::Group;
+use proxmox_ve_config::guest::types::Vmid;
+
+use crate::config::FirewallConfig;
+use crate::object::{NftObjectEnv, ToNftObjects};
+use crate::rule::{NftRule, NftRuleEnv};
+
+static CLUSTER_TABLE_NAME: &str = "proxmox-firewall";
+static HOST_TABLE_NAME: &str = "proxmox-firewall";
+static GUEST_TABLE_NAME: &str = "proxmox-firewall-guests";
+
+static NF_CONNTRACK_MAX_FILE: &str =
"/proc/sys/net/netfilter/nf_conntrack_max";
+static NF_CONNTRACK_TCP_TIMEOUT_ESTABLISHED: &str =
+"/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established";
+static NF_CONNTRACK_TCP_TIMEOUT_SYN_RECV: &str =
+"/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_syn_recv";
+static LOG_CONNTRACK_FILE: &str = "/var/lib/pve-firewall/log_nf_conntrack";
+
+#[derive(Default)]
+pub struct Firewall {
+config: FirewallConfig,
+}
+
+impl From for Firewall {
+fn from(config: FirewallConfig) -> Self {
+Self { config }
+}
+}
+
+impl Firewall {
+pub fn new() -> Self {
+Self {
+..Default::default()
+}
+}
+
+pub fn is_enabled(&self) -> bool {
+self.config.is_enabled()
+}
+
+fn cluster_table() -> TablePart {
+TablePart::new(TableFamily::Inet, CLUSTER_TABLE_NAME)
+}
+
+fn host_table() -> TablePart {
+TablePart::new(TableFamily::Inet, HOST_TABLE_NAME)
+}
+
+fn guest_table() -> TablePart {
+TablePart::new(TableFamily::Bridge, GUEST_TABLE_NAME)
+}
+
+fn guest_vmap(dir: Direction) -> SetName {
+SetName::new(Self::guest_table(), format!("vm-map-{dir}"))
+}
+
+fn cluster_chain(dir: Direction) -> ChainPart {
+ChainPart::new(Self::cluster_table(), format!("cluster-{dir}"))
+}
+
+fn host_chain(dir: Direction) -> ChainPart {
+ChainPart::new(Self::host_table(), format!("host-{dir}"))
+}
+
+fn guest_chain(dir: Direction, vmid: Vmid) -> ChainPart {
+ChainPart::new(Self::guest_table(), format!("guest-{vmid}-{dir}"))
+}
+
+fn group_chain(table: TablePart, name: &str, dir: Direction) -> ChainPart {
+ChainPart::new(table, format!("group-{name}-{dir}"))
+}
+
+fn host_conntrack_chain() -> ChainPart {
+ChainPart::new(Self::host_table(), "ct-in".to_string())
+}
+
+fn host_option_chain(dir: Direction) -> ChainPart {
+ChainPart::new(Self::host_table(), format!("option-{dir}"))
+}
+
+fn synflood_limit_chain() -> ChainPart {
+ChainPart::new(Self::host_table(), "ratelimit-synflood")
+}
+
+fn log_invalid_tcp_chain() -> ChainPart {
+ChainPart::new(Self::host_table(), "log-invalid-tcp")
+}
+
+fn log_smurfs_chain() -> ChainPart {
+
[pve-devel] [PATCH proxmox-firewall v3 22/39] nftables: statement: add conversion traits for config types
Some types from the firewall configuration map directly onto nftables
statements. For those we implement conversion traits so we can
conveniently convert between the configuration types and the
respective nftables types.
As with the expressions, those are guarded behind a feature so the
nftables crate can be used standalone without having to pull in the
proxmox-ve-config crate.
Reviewed-by: Lukas Wagner
Reviewed-by: Max Carrara
Co-authored-by: Wolfgang Bumiller
Signed-off-by: Stefan Hanreich
---
proxmox-nftables/src/statement.rs | 71 ++-
1 file changed, 70 insertions(+), 1 deletion(-)
diff --git a/proxmox-nftables/src/statement.rs
b/proxmox-nftables/src/statement.rs
index e6371f6..e89f678 100644
--- a/proxmox-nftables/src/statement.rs
+++ b/proxmox-nftables/src/statement.rs
@@ -1,6 +1,15 @@
use anyhow::{bail, Error};
use serde::{Deserialize, Serialize};
+#[cfg(feature = "config-ext")]
+use proxmox_ve_config::firewall::types::log::LogLevel as ConfigLogLevel;
+#[cfg(feature = "config-ext")]
+use proxmox_ve_config::firewall::types::log::LogRateLimit;
+#[cfg(feature = "config-ext")]
+use proxmox_ve_config::firewall::types::rule::Verdict as ConfigVerdict;
+#[cfg(feature = "config-ext")]
+use proxmox_ve_config::guest::types::Vmid;
+
use crate::expression::Meta;
use crate::helper::{NfVec, Null};
use crate::types::{RateTimescale, RateUnit, Verdict};
@@ -104,7 +113,18 @@ impl> From for Statement {
}
}
-#[derive(Clone, Debug, Deserialize, Serialize)]
+#[cfg(feature = "config-ext")]
+impl From for Statement {
+fn from(value: ConfigVerdict) -> Self {
+match value {
+ConfigVerdict::Accept => Statement::make_accept(),
+ConfigVerdict::Reject => Statement::make_drop(),
+ConfigVerdict::Drop => Statement::make_drop(),
+}
+}
+}
+
+#[derive(Clone, Copy, Debug, Deserialize, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum RejectType {
#[serde(rename = "tcp reset")]
@@ -145,6 +165,22 @@ pub struct Log {
}
impl Log {
+#[cfg(feature = "config-ext")]
+pub fn generate_prefix(
+vmid: impl Into>,
+log_level: LogLevel,
+chain_name: &str,
+verdict: ConfigVerdict,
+) -> String {
+format!(
+":{}:{}:{}: {}: ",
+vmid.into().unwrap_or(Vmid::new(0)),
+log_level.nflog_level(),
+chain_name,
+verdict,
+)
+}
+
pub fn new_nflog(prefix: String, group: i64) -> Self {
Self {
prefix: Some(prefix),
@@ -168,6 +204,25 @@ pub enum LogLevel {
Audit,
}
+#[cfg(feature = "config-ext")]
+impl TryFrom for LogLevel {
+type Error = Error;
+
+fn try_from(value: ConfigLogLevel) -> Result {
+match value {
+ConfigLogLevel::Emergency => Ok(LogLevel::Emerg),
+ConfigLogLevel::Alert => Ok(LogLevel::Alert),
+ConfigLogLevel::Critical => Ok(LogLevel::Crit),
+ConfigLogLevel::Error => Ok(LogLevel::Err),
+ConfigLogLevel::Warning => Ok(LogLevel::Warn),
+ConfigLogLevel::Notice => Ok(LogLevel::Notice),
+ConfigLogLevel::Info => Ok(LogLevel::Info),
+ConfigLogLevel::Debug => Ok(LogLevel::Debug),
+_ => bail!("cannot convert config log level to nftables"),
+}
+}
+}
+
impl LogLevel {
pub fn nflog_level(&self) -> u8 {
match self {
@@ -231,6 +286,20 @@ pub struct AnonymousLimit {
pub inv: Option,
}
+#[cfg(feature = "config-ext")]
+impl From for AnonymousLimit {
+fn from(config: LogRateLimit) -> Self {
+AnonymousLimit {
+rate: config.rate(),
+per: config.per().into(),
+rate_unit: None,
+burst: Some(config.burst()),
+burst_unit: None,
+inv: None,
+}
+}
+}
+
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Vmap {
key: Expression,
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH proxmox-firewall v3 25/39] nftables: add nft client
Add a thin wrapper around nft, which can be used to run commands
defined by the rust types.
Reviewed-by: Lukas Wagner
Reviewed-by: Max Carrara
Co-authored-by: Wolfgang Bumiller
Signed-off-by: Stefan Hanreich
---
proxmox-nftables/src/client.rs | 85 ++
proxmox-nftables/src/lib.rs| 2 +
2 files changed, 87 insertions(+)
create mode 100644 proxmox-nftables/src/client.rs
diff --git a/proxmox-nftables/src/client.rs b/proxmox-nftables/src/client.rs
new file mode 100644
index 000..69e464b
--- /dev/null
+++ b/proxmox-nftables/src/client.rs
@@ -0,0 +1,85 @@
+use std::io::prelude::*;
+use std::process::{Command, Stdio};
+
+use thiserror::Error;
+
+use crate::command::{CommandOutput, Commands};
+
+#[derive(Error, Debug)]
+pub enum NftError {
+#[error("cannot communicate with child process")]
+Io(#[from] std::io::Error),
+#[error("cannot execute nftables commands")]
+Command(String),
+}
+
+pub struct NftClient;
+
+impl NftClient {
+fn execute_nft_commands(json: bool, input: &[u8]) -> Result {
+let mut command = Command::new("nft");
+
+if json {
+command.arg("-j");
+}
+
+let mut child = command
+.arg("-f")
+.arg("-")
+.stdin(Stdio::piped())
+.stdout(Stdio::piped())
+.stderr(Stdio::piped())
+.spawn()
+.map_err(NftError::from)?;
+
+if let Err(error) = child.stdin.take().expect("can get
stdin").write_all(input) {
+return Err(NftError::from(error));
+};
+
+let mut error_output = String::new();
+
+match child
+.stderr
+.take()
+.expect("can get stderr")
+.read_to_string(&mut error_output)
+{
+Ok(_) if !error_output.is_empty() => {
+return Err(NftError::Command(error_output));
+}
+Err(error) => {
+return Err(NftError::from(error));
+}
+_ => (),
+};
+
+let mut output = String::new();
+
+if let Err(error) = child
+.stdout
+.take()
+.expect("can get stdout")
+.read_to_string(&mut output)
+{
+return Err(NftError::from(error));
+};
+
+Ok(output)
+}
+
+pub fn run_json_commands(commands: &Commands) ->
Result, NftError> {
+let json = serde_json::to_vec(commands).expect("can serialize commands
struct");
+let output = Self::execute_nft_commands(true, &json)?;
+
+if !output.is_empty() {
+let parsed_output: Option =
serde_json::from_str(&output).ok();
+return Ok(parsed_output);
+}
+
+Ok(None)
+}
+
+pub fn run_commands(commands: &str) -> Result {
+Self::execute_nft_commands(false, commands.as_bytes())
+}
+}
diff --git a/proxmox-nftables/src/lib.rs b/proxmox-nftables/src/lib.rs
index 60ddb3f..2003e1b 100644
--- a/proxmox-nftables/src/lib.rs
+++ b/proxmox-nftables/src/lib.rs
@@ -1,9 +1,11 @@
+pub mod client;
pub mod command;
pub mod expression;
pub mod helper;
pub mod statement;
pub mod types;
+pub use client::NftClient;
pub use command::Command;
pub use expression::Expression;
pub use statement::Statement;
--
2.39.2
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH proxmox-firewall v3 28/39] firewall: add config loader
We load the firewall configuration from the default paths, as well as
only the guest configurations that are local to the node itself. In
the future we could change this to use pmxcfs directly instead.
We also load information from nftables directly about dynamically
created chains (mostly chains for the guest firewall).
Reviewed-by: Lukas Wagner
Reviewed-by: Max Carrara
Co-authored-by: Wolfgang Bumiller
Signed-off-by: Stefan Hanreich
---
proxmox-firewall/Cargo.toml| 2 +
proxmox-firewall/src/config.rs | 283 +
proxmox-firewall/src/main.rs | 3 +
3 files changed, 288 insertions(+)
create mode 100644 proxmox-firewall/src/config.rs
diff --git a/proxmox-firewall/Cargo.toml b/proxmox-firewall/Cargo.toml
index b59d973..431e71a 100644
--- a/proxmox-firewall/Cargo.toml
+++ b/proxmox-firewall/Cargo.toml
@@ -11,6 +11,8 @@ description = "Proxmox VE nftables firewall implementation"
license = "AGPL-3"
[dependencies]
+log = "0.4"
+env_logger = "0.10"
anyhow = "1"
proxmox-nftables = { path = "../proxmox-nftables", features = ["config-ext"] }
diff --git a/proxmox-firewall/src/config.rs b/proxmox-firewall/src/config.rs
new file mode 100644
index 000..2cf3e39
--- /dev/null
+++ b/proxmox-firewall/src/config.rs
@@ -0,0 +1,283 @@
+use std::collections::BTreeMap;
+use std::default::Default;
+use std::fs::File;
+use std::io::{self, BufReader};
+use std::sync::OnceLock;
+
+use anyhow::Error;
+
+use proxmox_ve_config::firewall::cluster::Config as ClusterConfig;
+use proxmox_ve_config::firewall::guest::Config as GuestConfig;
+use proxmox_ve_config::firewall::host::Config as HostConfig;
+use proxmox_ve_config::firewall::types::alias::{Alias, AliasName, AliasScope};
+
+use proxmox_ve_config::guest::types::Vmid;
+use proxmox_ve_config::guest::{GuestEntry, GuestMap};
+
+use proxmox_nftables::command::{CommandOutput, Commands, List, ListOutput};
+use proxmox_nftables::types::ListChain;
+use proxmox_nftables::NftClient;
+
+pub trait FirewallConfigLoader {
+fn cluster(&self) -> Option>;
+fn host(&self) -> Option>;
+fn guest_list(&self) -> GuestMap;
+fn guest_config(&self, vmid: &Vmid, guest: &GuestEntry) -> Option>;
+fn guest_firewall_config(&self, vmid: &Vmid) -> Option>;
+}
+
+#[derive(Default)]
+struct PveFirewallConfigLoader {}
+
+impl PveFirewallConfigLoader {
+pub fn new() -> Self {
+Default::default()
+}
+}
+
+/// opens a configuration file
+///
+/// It returns a file handle to the file or [`None`] if it doesn't exist.
+fn open_config_file(path: &str) -> Result, Error> {
+match File::open(path) {
+Ok(data) => Ok(Some(data)),
+Err(err) if err.kind() == io::ErrorKind::NotFound => {
+log::info!("config file does not exist: {path}");
+Ok(None)
+}
+Err(err) => {
+let context = format!("unable to open configuration file at
{path}");
+Err(anyhow::Error::new(err).context(context))
+}
+}
+}
+
+const CLUSTER_CONFIG_PATH: &str = "/etc/pve/firewall/cluster.fw";
+const HOST_CONFIG_PATH: &str = "/etc/pve/local/host.fw";
+
+impl FirewallConfigLoader for PveFirewallConfigLoader {
+fn cluster(&self) -> Option> {
+log::info!("loading cluster config");
+
+let fd =
+open_config_file(CLUSTER_CONFIG_PATH).expect("able to read cluster
firewall config");
+
+if let Some(file) = fd {
+let buf_reader = Box::new(BufReader::new(file)) as Box;
+return Some(buf_reader);
+}
+
+None
+}
+
+fn host(&self) -> Option> {
+log::info!("loading host config");
+
+let fd = open_config_file(HOST_CONFIG_PATH).expect("able to read host
firewall config");
+
+if let Some(file) = fd {
+let buf_reader = Box::new(BufReader::new(file)) as Box;
+return Some(buf_reader);
+}
+
+None
+}
+
+fn guest_list(&self) -> GuestMap {
+log::info!("loading vmlist");
+GuestMap::new().expect("able to read vmlist")
+}
+
+fn guest_config(&self, vmid: &Vmid, entry: &GuestEntry) -> Option> {
+log::info!("loading guest #{vmid} config");
+
+let fd = open_config_file(&GuestMap::config_path(vmid, entry))
+.expect("able to read guest config");
+
+if let Some(file) = fd {
+let buf_reader = Box::new(BufReader::new(file)) as Box;
+return Some(buf_reader);
+}
+
+None
+}
+
+fn guest_firewall_config(&self, vmid: &Vmid) -> Option> {
+log::info!("loading guest #{vmid} firewall config");
+
+let fd = open_config_file(&GuestMap::firewall_config_path(vmid))
+.expect("able to read guest firewall config");
+
+if let Some(file) = fd {
+let buf_reader = Box::new(BufReader::new(file)) as Box;
+return Some(buf_reader);
+}
+
+None
+}
+}
+
+pub trait NftConfigLoader {
+fn chains(&self) -
[pve-devel] [PATCH proxmox-firewall v3 16/39] config: firewall: add conntrack helper types
Reviewed-by: Lukas Wagner
Reviewed-by: Max Carrara
Co-authored-by: Wolfgang Bumiller
Signed-off-by: Stefan Hanreich
---
proxmox-ve-config/resources/ct_helper.json | 52 +
proxmox-ve-config/src/firewall/ct_helper.rs | 115
proxmox-ve-config/src/firewall/mod.rs | 1 +
3 files changed, 168 insertions(+)
create mode 100644 proxmox-ve-config/resources/ct_helper.json
create mode 100644 proxmox-ve-config/src/firewall/ct_helper.rs
diff --git a/proxmox-ve-config/resources/ct_helper.json
b/proxmox-ve-config/resources/ct_helper.json
new file mode 100644
index 000..5e70a3a
--- /dev/null
+++ b/proxmox-ve-config/resources/ct_helper.json
@@ -0,0 +1,52 @@
+[
+ {
+"name": "amanda",
+"v4": true,
+"v6": true,
+"udp": 10080
+ },
+ {
+"name": "ftp",
+"v4": true,
+"v6": true,
+"tcp": 21
+ } ,
+ {
+"name": "irc",
+"v4": true,
+"tcp": 6667
+ },
+ {
+"name": "netbios-ns",
+"v4": true,
+"udp": 137
+ },
+ {
+"name": "pptp",
+"v4": true,
+"tcp": 1723
+ },
+ {
+"name": "sane",
+"v4": true,
+"v6": true,
+"tcp": 6566
+ },
+ {
+"name": "sip",
+"v4": true,
+"v6": true,
+"udp": 5060
+ },
+ {
+"name": "snmp",
+"v4": true,
+"udp": 161
+ },
+ {
+"name": "tftp",
+"v4": true,
+"v6": true,
+"udp": 69
+ }
+]
diff --git a/proxmox-ve-config/src/firewall/ct_helper.rs
b/proxmox-ve-config/src/firewall/ct_helper.rs
new file mode 100644
index 000..40e4fee
--- /dev/null
+++ b/proxmox-ve-config/src/firewall/ct_helper.rs
@@ -0,0 +1,115 @@
+use anyhow::{bail, Error};
+use serde::Deserialize;
+use std::collections::HashMap;
+use std::sync::OnceLock;
+
+use crate::firewall::types::address::Family;
+use crate::firewall::types::rule_match::{Ports, Protocol, Tcp, Udp};
+
+#[derive(Clone, Debug, Deserialize)]
+pub struct CtHelperMacroJson {
+pub v4: Option,
+pub v6: Option,
+pub name: String,
+pub tcp: Option,
+pub udp: Option,
+}
+
+impl TryFrom for CtHelperMacro {
+type Error = Error;
+
+fn try_from(value: CtHelperMacroJson) -> Result {
+if value.tcp.is_none() && value.udp.is_none() {
+bail!("Neither TCP nor UDP port set in CT helper!");
+}
+
+let family = match (value.v4, value.v6) {
+(Some(true), Some(true)) => None,
+(Some(true), _) => Some(Family::V4),
+(_, Some(true)) => Some(Family::V6),
+_ => bail!("Neither v4 nor v6 set in CT Helper Macro!"),
+};
+
+let mut ct_helper = CtHelperMacro {
+family,
+name: value.name,
+tcp: None,
+udp: None,
+};
+
+if let Some(dport) = value.tcp {
+let ports = Ports::from_u16(None, dport);
+ct_helper.tcp = Some(Tcp::new(ports).into());
+}
+
+if let Some(dport) = value.udp {
+let ports = Ports::from_u16(None, dport);
+ct_helper.udp = Some(Udp::new(ports).into());
+}
+
+Ok(ct_helper)
+}
+}
+
+#[derive(Clone, Debug, Deserialize)]
+#[serde(try_from = "CtHelperMacroJson")]
+pub struct CtHelperMacro {
+family: Option,
+name: String,
+tcp: Option,
+udp: Option,
+}
+
+impl CtHelperMacro {
+fn helper_name(&self, protocol: &str) -> String {
+format!("helper-{}-{protocol}", self.name)
+}
+
+pub fn tcp_helper_name(&self) -> String {
+self.helper_name("tcp")
+}
+
+pub fn udp_helper_name(&self) -> String {
+self.helper_name("udp")
+}
+
+pub fn family(&self) -> Option {
+self.family
+}
+
+pub fn name(&self) -> &str {
+self.name.as_ref()
+}
+
+pub fn tcp(&self) -> Option<&Protocol> {
+self.tcp.as_ref()
+}
+
+pub fn udp(&self) -> Option<&Protocol> {
+self.udp.as_ref()
+}
+}
+
+fn hashmap() -> &'static HashMap {
+const MACROS: &str = include_str!("../../resources/ct_helper.json");
+static HASHMAP: OnceLock> = OnceLock::new();
+
+HASHMAP.get_or_init(|| {
+let macro_data: Vec = match
serde_json::from_str(MACROS) {
+Ok(data) => data,
+Err(err) => {
+log::error!("could not load data for ct helpers: {err}");
+Vec::new()
+}
+};
+
+macro_data
+.into_iter()
+.map(|elem| (elem.name.clone(), elem))
+.collect()
+})
+}
+
+pub fn get_cthelper(name: &str) -> Option<&'static CtHelperMacro> {
+hashmap().get(name)
+}
diff --git a/proxmox-ve-config/src/firewall/mod.rs
b/proxmox-ve-config/src/firewall/mod.rs
index 0f438ca..2cf57e2 100644
--- a/proxmox-ve-config/src/firewall/mod.rs
+++ b/proxmox-ve-config/src/firewall/mod.rs
@@ -1,5 +1,6 @@
pub mod cluster;
pub mod common;
+pub mod ct_helper;
pub mod fw_macros;
pub mod guest;
pub mod host;
--
2.39.2
___
pve-devel mailin
[pve-devel] applied-series: [PATCH cluster/manager/storage/docs 0/9] fix #4886: improve SSH handling
Am 11/01/2024 um 11:51 schrieb Fabian Grünbichler: > this series replaces the old mechanism that used a cluster-wide merged known > hosts file with distributing of each node's host key via pmxcfs, and pinning > the distributed key explicitly for internal SSH connections. > > the main changes in pve-cluster somewhat break the old manager and > storage versions, but only when such a partial upgrade is mixed with a > host key rotation of some sort. > > pve-storage uses a newly introduced helper, so needs a versioned > dependency accordingly. > > the last pve-docs patch has a placeholder for the actual version shipping the > changes which needs to be replaced when applying. > > there's still some potential for follow-ups: > - 'pvecm ssh' wrapper to debug and/or re-use the host key pinning (and other > future changes) > - also add non-RSA host keys > - key (and thus authorized keys) and/or sshd disentangling (this > potentially also affects external access, so might be done on a major > release to give more heads up) > > cluster: > > Fabian Grünbichler (4): > fix #4886: write node SSH hostkey to pmxcfs > fix #4886: SSH: pin node's host key if available > ssh: expose SSH options on their own > pvecm: stop merging SSH known hosts by default > > src/PVE/CLI/pvecm.pm | 10 -- > src/PVE/Cluster/Setup.pm | 24 +--- > src/PVE/SSHInfo.pm | 31 +++ > 3 files changed, 56 insertions(+), 9 deletions(-) > > docs: > > Fabian Grünbichler (2): > ssh: make pitfalls a regular section instead of block > ssh: document PVE-specific setup > > pvecm.adoc | 26 +- > 1 file changed, 21 insertions(+), 5 deletions(-) > > manager: > > Fabian Grünbichler (2): > vnc: use SSH command helper > pvesh: use SSH command helper > > PVE/API2/Nodes.pm | 3 ++- > PVE/CLI/pvesh.pm | 4 ++-- > 2 files changed, 4 insertions(+), 3 deletions(-) > > storage: > > Fabian Grünbichler (1): > upload: use SSH helper to get ssh/scp options > > src/PVE/API2/Storage/Status.pm | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > applied series, thanks! ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
