[pve-devel] applied: [PATCH v2 pve-zsync] parse disks: improve error messages

2024-04-11 Thread Thomas Lamprecht
On 12/09/2023 14:29, Fiona Ebner wrote: > The one with the backup flag was reported in the community forum: > https://forum.proxmox.com/threads/77254/ > > Signed-off-by: Fiona Ebner > --- > > Changes in v2: > * Further improve message as suggested by Sterzy. > > pve-zsync | 5 +++-- > 1

[pve-devel] partially-applied: [PATCH qemu v3 06/22] backup: add minimum cluster size to performance options

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 11:29, Fiona Ebner wrote: > Useful to make discard-source work in the context of backup fleecing > when the fleecing image has a larger granularity than the backup > target. > > Backup/block-copy will use at least this granularity for copy operations > and in particular, discard

[pve-devel] applied: [PATCH guest-common v3 10/22] vzdump: schema: make storage for fleecing semi-optional

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 11:29, Fiona Ebner wrote: > so it doesn't need to be set when explicitly disabling fleecing. Needs > a custom verifier to enforce it being set when enabled. > > Suggested-by: Fabian Grünbichler > Signed-off-by: Fiona Ebner > --- > src/PVE/VZDump/Common.pm | 14 +- > 1

Re: [pve-devel] [PATCH guest-common v3 10/22] vzdump: schema: make storage for fleecing semi-optional

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 11:29, Fiona Ebner wrote: > so it doesn't need to be set when explicitly disabling fleecing. Needs > a custom verifier to enforce it being set when enabled. > > Suggested-by: Fabian Grünbichler > Signed-off-by: Fiona Ebner > --- > src/PVE/VZDump/Common.pm | 14 +- > 1

[pve-devel] applied: Re: [PATCH guest-common v3 09/22] vzdump: schema: add fleecing property string

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 11:29, Fiona Ebner wrote: > It's a property string, because that avoids having an implicit > "enabled" as part of a 'fleecing-storage' property. And there likely > will be more options in the future, e.g. threshold/limit for the > fleecing image size. > > Storage is non-optional, so

[pve-devel] applied: [PATCH common v3 08/22] json schema: add format description for pve-storage-id standard option

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 11:29, Fiona Ebner wrote: > so that the option can be used as part of a property string. > > Signed-off-by: Fiona Ebner > --- > src/PVE/JSONSchema.pm | 1 + > 1 file changed, 1 insertion(+) > > applied, thanks! ___ pve-devel mailing

Re: [pve-devel] [RFC qemu-server v3 17/22] parse config: allow config keys with minus sign

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 11:29, Fiona Ebner wrote: > In preparation for the upcoming 'fleecing-images' key. To avoid mixing > of options with - and options with _, which is not very user-friendly, > it would be nice to add aliases for existing options with _. And > long-term, backup restore handlers could

Re: [pve-devel] [PATCH manager 1/9] report: add kernel command line including boot time

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 19:07, Alexander Zeidler wrote: > On Mon, 2024-03-25 at 08:52 +0100, Thomas Lamprecht wrote: >> journalctl --list-boots > > On slow servers / storage drives this execution can easily take 10 seconds > or longer. But there is an alternative that is quite fast and even shows > the

Re: [pve-devel] [PATCH manager 9/9] report: add microcode info to better assess possible system impacts

2024-04-11 Thread Alexander Zeidler
On Mon, 2024-03-25 at 10:00 +0100, Thomas Lamprecht wrote: > On 22/03/2024 14:59, Alexander Zeidler wrote: > > * list availability and installation status of `*microcode` packages > > * grep for applied "Early OS Microcode Updates" > > * grep for (un)patched CPU vulnerability messages > > > >

Re: [pve-devel] [PATCH manager 7/9] report: add info of the mainboard in use

2024-04-11 Thread Alexander Zeidler
On Fri, 2024-03-22 at 15:26 +0100, Aaron Lauterer wrote: > If we add more info, then dmidecode -t {1,2,3} might be interesting as > well as those deliver motherboard, system and chassis infos. > > If you want to see what you get there, run them on some decent servers > as most consumer boards

Re: [pve-devel] [PATCH manager 6/9] report: switch `dmidecode` to quiet to omit almost never needed info

2024-04-11 Thread Alexander Zeidler
On Mon, 2024-03-25 at 09:20 +0100, Thomas Lamprecht wrote: > On 22/03/2024 14:59, Alexander Zeidler wrote: > > like on this system: > > > > # dmidecode -t bios > > # dmidecode 3.4 > > Getting SMBIOS data from sysfs. > > SMBIOS 3.0.0 present. > > > > Handle 0x, DMI type 0, 24 bytes > >

Re: [pve-devel] [PATCH manager 5/9] report: move `lscpu` & cluster info to more appropriate sections

2024-04-11 Thread Alexander Zeidler
On Mon, 2024-03-25 at 09:11 +0100, Thomas Lamprecht wrote: > And why are those more appropriate? Both fit's the general "always important" > section, so even though they fit the section you moved them too, they also > fit the general one, so some actual reasoning here would be good.. > > On

Re: [pve-devel] [PATCH manager 3/9] report: add list of upgradable packages

2024-04-11 Thread Alexander Zeidler
On Mon, 2024-03-25 at 09:02 +0100, Thomas Lamprecht wrote: > On 22/03/2024 14:59, Alexander Zeidler wrote: > > * to easily see if APT already knows about old packages in use and > > their exact version > > * to reconsider asking for applying updates as a first recommendation > > if the list is

Re: [pve-devel] [PATCH manager 2/9] report: add `jobs.cfg` to debug related network/load/backup/etc issues

2024-04-11 Thread Alexander Zeidler
On Mon, 2024-03-25 at 09:08 +0100, Thomas Lamprecht wrote: > > > > On 22/03/2024 14:59, Alexander Zeidler wrote: > > > > > > > > Suggested-by: Friedrich Weber > > > > > > > > Signed-off-by: Alexander Zeidler > > > > > > > > --- > > > > > > > >  PVE/Report.pm | 1 + > > > > > > > >  1 file

Re: [pve-devel] [PATCH manager 1/9] report: add kernel command line including boot time

2024-04-11 Thread Alexander Zeidler
On Mon, 2024-03-25 at 08:52 +0100, Thomas Lamprecht wrote: > > > > On 22/03/2024 14:59, Alexander Zeidler wrote: > > > > > > > > While using `/proc/cmdline` would already provide an initial > > > > > > > > info for > > > > > > > > debugging passthrough and similar, the use of `dmesg` is an > > >

Re: [pve-devel] [PATCH guest-common v2 2/5] mapping: pci: rework properties check

2024-04-11 Thread Thomas Lamprecht
On 10/04/2024 13:03, Dominik Csapak wrote: > refactors the actual checking out to its own sub, so we can reuse it > later > > Signed-off-by: Dominik Csapak > --- > src/PVE/Mapping/PCI.pm | 43 +- > 1 file changed, 26 insertions(+), 17 deletions(-) > >

[pve-devel] applied: Re: [PATCH guest-common v2 1/5] mapping: pci: fix missing description/default for mdev

2024-04-11 Thread Thomas Lamprecht
On 10/04/2024 13:03, Dominik Csapak wrote: > Signed-off-by: Dominik Csapak > --- > src/PVE/Mapping/PCI.pm | 2 ++ > 1 file changed, 2 insertions(+) > > applied, thanks! ___ pve-devel mailing list pve-devel@lists.proxmox.com

[pve-devel] applied-series: [PATCH-SERIES v2 guest-common] replication improvements

2024-04-11 Thread Thomas Lamprecht
On 13/12/2023 15:17, Fiona Ebner wrote: > Improve error when finding a common base snapshot and fix the check if > a snapshot is needed by replication when there are volumes with > replicate setting turned off. > > First version: >

Re: [pve-devel] [PATCH manager v9 3/3] ui: machine: add viommu ComboBox

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 12:48, Markus Frank wrote: > Added a proxmoxKVComboBox for selecting a vIOMMU implementation for a > VM. If i440fx is selected, a hint tells that q35 is required for Intel vIOMMU. > > The UI also needs to parse the new machine parameter as PropertyString. > > Signed-off-by: Markus

[pve-devel] applied: Re: [PATCH docs v9 2/3] add vIOMMU documentation

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 12:48, Markus Frank wrote: > Signed-off-by: Markus Frank > --- > qm-pci-passthrough.adoc | 50 + > qm.adoc | 1 + > 2 files changed, 51 insertions(+) > > applied, thanks! ___

[pve-devel] applied: Re: [PATCH qemu-server v9 1/3] fix #3784: config: Parameter for guest vIOMMU + test-cases

2024-04-11 Thread Thomas Lamprecht
On 11/04/2024 12:48, Markus Frank wrote: > vIOMMU enables the option to passthrough pci devices to L2 VMs > in L1 VMs via Nested Virtualisation and adds an extra isolation. > > Uses the new property-string from the "config: define machine schema > as property-string"-commit to add the viommu

Re: [pve-devel] [PATCH v3 common/docs/widget-toolkit/manager/firewall 0/6] drop vmbr prefix for bridges

2024-04-11 Thread Fabian Grünbichler
Reviewed-by: Fabian Grünbichler with some small nits for the docs patch, see comment there. the pve-common patch should probably get the bug number (545) in the subject as well. some hint about the inter-dependencies of the patches would be nice, AFAIU: - pve-manager requires

Re: [pve-devel] [PATCH v3 docs 2/6] network: update specification for bridge names

2024-04-11 Thread Fabian Grünbichler
On February 29, 2024 11:41 am, Stefan Hanreich wrote: > Signed-off-by: Stefan Hanreich > --- > pve-network.adoc | 15 +-- > 1 file changed, 9 insertions(+), 6 deletions(-) > > diff --git a/pve-network.adoc b/pve-network.adoc > index d1ec64b..a5ad9b4 100644 > --- a/pve-network.adoc >

Re: [pve-devel] [PATCH v2 container] fix #4846: Avoid the outdated noacl mount option on ext4

2024-04-11 Thread Fabian Grünbichler
On February 23, 2024 11:48 am, Filip Schauer wrote: > Do not use the 'noacl' mount option when mounting a container disk with > an ext4 file system. The option was removed from the kernel in commit > 2d544ec923db > > The ext4 detection is based on $do_format in alloc_disk. > > Signed-off-by:

Re: [pve-devel] [PATCH container] mountpoint mount: activate PVE-managed volumes during preparation

2024-04-11 Thread Fabian Grünbichler
On March 25, 2024 12:00 pm, Fiona Ebner wrote: > Otherwise it was not possible to hotplug a volume that was previously > deactivated and requires activation, e.g. an LVM LV that was detached > after shutting down the container couldn't be hotplugged anymore > later. > > Signed-off-by: Fiona Ebner

[pve-devel] applied: [PATCH container] config list helper: fix return type of vmid for container index API enpoint

2024-04-11 Thread Fabian Grünbichler
On December 5, 2023 12:35 pm, Fiona Ebner wrote: > The schema declares it to be an integer. Done in the config_list() > helper, to ensure consistency for the following existing code in > vmstatus(): > >> my $list = $opt_vmid ? { $opt_vmid => { type => 'lxc', vmid => >> int($opt_vmid) }} :

[pve-devel] applied-series: [PATCH v5 pve-storage, pve-manager 00/11] Fix #4759: Configure Permissions for ceph-crash.service

2024-04-11 Thread Fabian Grünbichler
with a few small follow-ups in pve-manager On April 2, 2024 4:55 pm, Max Carrara wrote: > Fix #4759: Configure Permissions for ceph-crash.service - Version 5 > === > > Notable changes since v4 > > > * The

Re: [pve-devel] [PATCH container] Fix invalid device passthrough being added to config

2024-04-11 Thread Fiona Ebner
Am 29.01.24 um 15:29 schrieb Filip Schauer: > Fix a bug that allows a device passthrough entry to be added to the > config despite the device path not pointing to a device. Previously, > adding an invalid device passthrough entry would throw an error, but the > entry would still be added to the

Re: [pve-devel] [PATCH-SERIES v2 guest-common] replication improvements

2024-04-11 Thread Fiona Ebner
Am 13.12.23 um 15:17 schrieb Fiona Ebner: > Improve error when finding a common base snapshot and fix the check if > a snapshot is needed by replication when there are volumes with > replicate setting turned off. > > First version: >

Re: [pve-devel] [PATCH v2 pve-zsync] parse disks: improve error messages

2024-04-11 Thread Fiona Ebner
Am 12.09.23 um 14:29 schrieb Fiona Ebner: > The one with the backup flag was reported in the community forum: > https://forum.proxmox.com/threads/77254/ > > Signed-off-by: Fiona Ebner Ping ___ pve-devel mailing list pve-devel@lists.proxmox.com

Re: [pve-devel] [PATCH container] config list helper: fix return type of vmid for container index API enpoint

2024-04-11 Thread Fiona Ebner
Am 05.12.23 um 12:35 schrieb Fiona Ebner: > The schema declares it to be an integer. Done in the config_list() > helper, to ensure consistency for the following existing code in > vmstatus(): > >> my $list = $opt_vmid ? { $opt_vmid => { type => 'lxc', vmid => >> int($opt_vmid) }} :

Re: [pve-devel] [RFC kernel-meta] add proxmox-secure-boot-support package

2024-04-11 Thread Fabian Grünbichler
On February 6, 2024 10:40 am, Thomas Lamprecht wrote: > Am 05/02/2024 um 12:45 schrieb Fabian Grünbichler: >> On February 2, 2024 7:23 pm, Thomas Lamprecht wrote: >>> seems OK w.r.t. change, but do we want this to be either part of the shim, >>> or a separate repo? So that we do not need to ship a

Re: [pve-devel] [PATCH-SERIES qemu-server] improve error detection/messages for some block jobs

2024-04-11 Thread Fiona Ebner
Am 12.03.24 um 12:59 schrieb Fiona Ebner: > When auto-dismiss=true (the default), a failed job can disappear very > quickly from the job list and there might not be any chance to see the > error in the result of 'query-block-jobs'. For jobs with $completion > being 'auto', like 'block-stream', it

[pve-devel] [PATCH v2 qemu-server 3/4] live restore: do not auto-dismiss stream job to improve error message and detection

2024-04-11 Thread Fiona Ebner
upon failure. Otherwise, the job would disappear too quickly from the job list and cannot be queried for the actual error anymore. Relevant part of the error in an actual example: Before: > VM 112 qmp command 'blockdev-del' failed - Node 'drive-scsi0-pbs' is busy: > node is used as backing hd

[pve-devel] [PATCH v2 qemu-server 4/4] live import: do not auto-dismiss stream job to improve error message and detection

2024-04-11 Thread Fiona Ebner
upon failure. Otherwise, the job would disappear too quickly from the job list and cannot be queried for the actual error anymore. Relevant part of the error in an actual example: Before: > VM 106 qmp command 'blockdev-del' failed - Node 'drive-scsi0-restore' is > busy: node is used as backing

[pve-devel] [PATCH v2 qemu-server 2/4] mirror: do not auto-dismiss to allow getting error message from job

2024-04-11 Thread Fiona Ebner
upon failure. Otherwise, the job would disappear too quickly from the job list and cannot be queried for the actual error anymore. Relevant part of the error in actual examples (note that the fact that it's a mirror job is already mentioned earlier in the full error, with "block job (mirror)

[pve-devel] [PATCH v2 qemu-server 1/4] blockjob: anticipate jobs with auto-dismiss=false for better error messages and detection

2024-04-11 Thread Fiona Ebner
When auto-dismiss=true (the default), a failed job can disappear very quickly from the job list and there might not be any chance to see the error in the result of 'query-block-jobs'. For jobs with $completion being 'auto', like 'block-stream', it couldn't even be detected that the job failed.

[pve-devel] [PATCH-SERIES v2 qemu-server] improve error detection/messages for some block jobs

2024-04-11 Thread Fiona Ebner
Changes in v2: * Also do not auto-dismiss for the stream job for the new live-import feature. When auto-dismiss=true (the default), a failed job can disappear very quickly from the job list and there might not be any chance to see the error in the result of 'query-block-jobs'. For jobs

[pve-devel] [PATCH qemu-server/docs/manager v9 0/3] vIOMMU-Feature #3784

2024-04-11 Thread Markus Frank
I was able to clarify any ambiguity regarding vIOMMU: https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg02370.html The iommu_platform parameter could be made an optional parameter for VirtIO devices to add more isolation (with the downside of making them less performant) in a follow-up

[pve-devel] [PATCH qemu-server v9 1/3] fix #3784: config: Parameter for guest vIOMMU + test-cases

2024-04-11 Thread Markus Frank
vIOMMU enables the option to passthrough pci devices to L2 VMs in L1 VMs via Nested Virtualisation and adds an extra isolation. Uses the new property-string from the "config: define machine schema as property-string"-commit to add the viommu option to the machine parameter. Currently there are

[pve-devel] [PATCH manager v9 3/3] ui: machine: add viommu ComboBox

2024-04-11 Thread Markus Frank
Added a proxmoxKVComboBox for selecting a vIOMMU implementation for a VM. If i440fx is selected, a hint tells that q35 is required for Intel vIOMMU. The UI also needs to parse the new machine parameter as PropertyString. Signed-off-by: Markus Frank --- www/manager6/qemu/MachineEdit.js | 45

[pve-devel] [PATCH docs v9 2/3] add vIOMMU documentation

2024-04-11 Thread Markus Frank
Signed-off-by: Markus Frank --- qm-pci-passthrough.adoc | 50 + qm.adoc | 1 + 2 files changed, 51 insertions(+) diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc index 3abeb50..bbd6b85 100644 --- a/qm-pci-passthrough.adoc

Re: [pve-devel] [PATCH ksm-control-daemon] ksmtuned: use PSS instead of RSZ for caluculating memory usage

2024-04-11 Thread Roland via pve-devel
--- Begin Message --- hello, nice to see some progress here, but keep in mind that memory calculation is completely wrong on zfs based systems, where arcsize is not taken into account https://bugzilla.proxmox.com/show_bug.cgi?id=3859 regards roland Am 08.04.24 um 15:25 schrieb Stefan Lendl:

[pve-devel] [PATCH ha-manager] d/postinst: make deb-systemd-invoke non-fatal

2024-04-11 Thread Fabian Grünbichler
else this can break an upgrade for unrelated reasons. this also mimics debhelper behaviour more (which we only not use here because of lack of reload support) - restructured the snippet to be more similar with an explicit `if` as well. Signed-off-by: Fabian Grünbichler ---

[pve-devel] [PATCH manager] d/postinst: make deb-systemd-invoke non-fatal

2024-04-11 Thread Fabian Grünbichler
else this can break an upgrade for unrelated reasons (regular debhelper also constructs the restart invocations like this, it even redirects output to /dev/null) Signed-off-by: Fabian Grünbichler --- debian/postinst | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git

Re: [pve-devel] [PATCH container 7/7] update: handle pool limits

2024-04-11 Thread Wolfgang Bumiller
On Thu, Apr 11, 2024 at 09:23:53AM +0200, Fabian Grünbichler wrote: > On April 10, 2024 3:13 pm, Fabian Grünbichler wrote: > > Signed-off-by: Fabian Grünbichler > > --- > > src/PVE/API2/LXC/Config.pm | 21 + > > 1 file changed, 21 insertions(+) > > > > diff --git

[pve-devel] applied: [PATCH] d/postinst: don't treat reload/restart failure as fatal

2024-04-11 Thread Wolfgang Bumiller
applied, thanks ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] d/postinst: don't treat reload/restart failure as fatal

2024-04-11 Thread Fabian Grünbichler
aborting apt invocations because of service failures like that is always painful, and lxcfs is in a lot of our build-dep chains, and build environments might not run systemd as init. Signed-off-by: Fabian Grünbichler --- debian/lxcfs.postinst | 2 +- 1 file changed, 1 insertion(+), 1

[pve-devel] [RFC qemu-server v3 20/22] migration: attempt to clean up potential left-over fleecing images

2024-04-11 Thread Fiona Ebner
Before the guest is migrated to a different node and they'd really become orphaned. Suggested-by: Fabian Grünbichler Signed-off-by: Fiona Ebner --- PVE/QemuMigrate.pm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 8d9b35ae..a190ce03 100644

[pve-devel] [PATCH manager v3 14/22] api: backup/vzdump: add permission check for fleecing storage

2024-04-11 Thread Fiona Ebner
Similar to how Datastore.AllocateSpace is required for the backup storage, it should also be required for the fleecing storage. Removing a fleecing storage from a job does not require more permissions than for modifying the job. Suggested-by: Fabian Grünbichler Signed-off-by: Fiona Ebner ---

[pve-devel] [PATCH qemu v3 07/22] PVE backup: add fleecing option

2024-04-11 Thread Fiona Ebner
When a fleecing option is given, it is expected that each device has a corresponding "-fleecing" block device already attached, except for EFI disk and TPM state, where fleecing is never used. The following graph was adapted from [0] which also contains more details about fleecing. [guest] |

[pve-devel] [RFC qemu-server v3 19/22] vzdump: better cleanup fleecing images after hard errors

2024-04-11 Thread Fiona Ebner
By recording the allocated fleecing images in the VM config, they are not immediately orphaned, should a hard error occur during backup that prevents cleanup. They are attempted to be cleaned up during the next backup run. Suggested-by: Fabian Grünbichler Signed-off-by: Fiona Ebner ---

[pve-devel] [PATCH qemu v3 04/22] qapi: blockdev-backup: add discard-source parameter

2024-04-11 Thread Fiona Ebner
From: Vladimir Sementsov-Ogievskiy Add a parameter that enables discard-after-copy. That is mostly useful in "push backup with fleecing" scheme, when source is snapshot-access format driver node, based on copy-before-write filter snapshot-access API: [guest] [snapshot-access] ~~

[pve-devel] [PATCH manager v3 12/22] vzdump: have property string helpers always return the result

2024-04-11 Thread Fiona Ebner
Previously, the result would only be returned implicitly and if not already parsed. While callers do not strictly need the return value, future callers might mistakenly rely on it and even work by chance in some scenarios, because of the implicit return. Make the code more future proof by

[pve-devel] [RFC qemu-server v3 21/22] destroy vm: clean up potential left-over fleecing images

2024-04-11 Thread Fiona Ebner
To avoid them becoming orphaned. Suggested-by: Fabian Grünbichler Signed-off-by: Fiona Ebner --- PVE/QemuServer.pm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 61804ae6..6edd1692 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@

[pve-devel] [PATCH qemu-server v3 15/22] backup: disk info: also keep track of size

2024-04-11 Thread Fiona Ebner
which will be needed to allocate fleecing images. Signed-off-by: Fiona Ebner --- PVE/VZDump/QemuServer.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm index be7d8e1e..51498dbc 100644 --- a/PVE/VZDump/QemuServer.pm +++

[pve-devel] [RFC guest-common v3 11/22] abstract config: do not copy fleecing images entry for snapshot

2024-04-11 Thread Fiona Ebner
Signed-off-by: Fiona Ebner --- RFC because belongs to the optional "improved cleanup"-part of the series. See qemu-server patches for more info. src/PVE/AbstractConfig.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm index

[pve-devel] [PATCH qemu v3 05/22] copy-before-write: allow specifying minimum cluster size

2024-04-11 Thread Fiona Ebner
Useful to make discard-source work in the context of backup fleecing when the fleecing image has a larger granularity than the backup target. Copy-before-write operations will use at least this granularity and in particular, discard requests to the source node will too. If the granularity is too

[pve-devel] [PATCH common v3 08/22] json schema: add format description for pve-storage-id standard option

2024-04-11 Thread Fiona Ebner
so that the option can be used as part of a property string. Signed-off-by: Fiona Ebner --- src/PVE/JSONSchema.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm index 4bf0e78..115f811 100644 --- a/src/PVE/JSONSchema.pm +++

[pve-devel] [PATCH qemu-server v3 16/22] backup: implement fleecing option

2024-04-11 Thread Fiona Ebner
Management for fleecing images is implemented here. If the fleecing option is set, for each disk (except EFI disk and TPM state) a new fleecing image is allocated on the configured fleecing storage (same storage as original disk by default). The disk is attached to QEMU with the 'size' parameter,

[pve-devel] [RFC qemu-server v3 18/22] schema: add fleecing-images config property

2024-04-11 Thread Fiona Ebner
to be used internally to record volume IDs of fleecing images allocated during backup. Suggested-by: Fabian Grünbichler Signed-off-by: Fiona Ebner --- PVE/API2/Qemu.pm | 9 + PVE/QemuServer.pm| 7 +++ PVE/VZDump/QemuServer.pm | 1 + 3 files changed, 17 insertions(+)

[pve-devel] [PATCH docs v3 22/22] vzdump: add section about backup fleecing

2024-04-11 Thread Fiona Ebner
Signed-off-by: Fiona Ebner --- vzdump.adoc | 38 ++ 1 file changed, 38 insertions(+) diff --git a/vzdump.adoc b/vzdump.adoc index b5bbac7..b01d3c5 100644 --- a/vzdump.adoc +++ b/vzdump.adoc @@ -136,6 +136,44 @@ not included in backups. For volume mount points

[pve-devel] [PATCH guest-common v3 09/22] vzdump: schema: add fleecing property string

2024-04-11 Thread Fiona Ebner
It's a property string, because that avoids having an implicit "enabled" as part of a 'fleecing-storage' property. And there likely will be more options in the future, e.g. threshold/limit for the fleecing image size. Storage is non-optional, so the storage choice needs to be a conscious

[pve-devel] [RFC qemu-server v3 17/22] parse config: allow config keys with minus sign

2024-04-11 Thread Fiona Ebner
In preparation for the upcoming 'fleecing-images' key. To avoid mixing of options with - and options with _, which is not very user-friendly, it would be nice to add aliases for existing options with _. And long-term, backup restore handlers could switch to the modern keys with -. Signed-off-by:

[pve-devel] [PATCH manager v3 13/22] vzdump: handle new 'fleecing' property string

2024-04-11 Thread Fiona Ebner
Signed-off-by: Fiona Ebner --- Changes in v3: * return result from parsing PVE/VZDump.pm | 14 ++ 1 file changed, 14 insertions(+) diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm index 72461f73..812357bd 100644 --- a/PVE/VZDump.pm +++ b/PVE/VZDump.pm @@ -130,6 +130,17 @@ my

[pve-devel] [PATCH qemu v3 06/22] backup: add minimum cluster size to performance options

2024-04-11 Thread Fiona Ebner
Useful to make discard-source work in the context of backup fleecing when the fleecing image has a larger granularity than the backup target. Backup/block-copy will use at least this granularity for copy operations and in particular, discard requests to the backup source will too. If the

[pve-devel] [PATCH guest-common v3 10/22] vzdump: schema: make storage for fleecing semi-optional

2024-04-11 Thread Fiona Ebner
so it doesn't need to be set when explicitly disabling fleecing. Needs a custom verifier to enforce it being set when enabled. Suggested-by: Fabian Grünbichler Signed-off-by: Fiona Ebner --- src/PVE/VZDump/Common.pm | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff

[pve-devel] [PATCH-SERIES v3] fix #4136: implement backup fleecing

2024-04-11 Thread Fiona Ebner
Changes in v3 (thanks to Wolfgang for feedback!): * Fix brittle code for permission check that only worked by chance. Changes in v2 (thanks - not limited to - to Fabian and Alexandre for feedback!): * Use v3 of "discard-source" upstream series (v4 was posted in the meantime

[pve-devel] [PATCH qemu v3 03/22] block/copy-before-write: create block_copy bitmap in filter node

2024-04-11 Thread Fiona Ebner
From: Vladimir Sementsov-Ogievskiy Currently block_copy creates copy_bitmap in source node. But that is in bad relation with .independent_close=true of copy-before-write filter: source node may be detached and removed before .bdrv_close() handler called, which should call

[pve-devel] [PATCH qemu v3 01/22] block/copy-before-write: fix permission

2024-04-11 Thread Fiona Ebner
From: Vladimir Sementsov-Ogievskiy In case when source node does not have any parents, the condition still works as required: backup job do create the parent by block_job_create -> block_job_add_bdrv -> bdrv_root_attach_child Still, in this case checking @perm variable doesn't work, as

[pve-devel] [PATCH qemu v3 02/22] block/copy-before-write: support unligned snapshot-discard

2024-04-11 Thread Fiona Ebner
From: Vladimir Sementsov-Ogievskiy First thing that crashes on unligned access here is bdrv_reset_dirty_bitmap(). Correct way is to align-down the snapshot-discard request. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner --- block/copy-before-write.c | 16

Re: [pve-devel] [PATCH manager 2/4] pvestatd: collect and broadcast pool usage

2024-04-11 Thread Wolfgang Bumiller
On Wed, Apr 10, 2024 at 03:13:08PM +0200, Fabian Grünbichler wrote: > so that other nodes can query it and both block changes that would violate the > limits, and mark pools which are violating it currently accordingly. > > Signed-off-by: Fabian Grünbichler > --- > PVE/Service/pvestatd.pm | 59

Re: [pve-devel] [PATCH container 2/7] status: add pool usage fields

2024-04-11 Thread Wolfgang Bumiller
On Wed, Apr 10, 2024 at 03:13:00PM +0200, Fabian Grünbichler wrote: > these are similar to existing ones, but with slightly different semantics. > > Signed-off-by: Fabian Grünbichler > --- > src/PVE/LXC.pm | 29 + > 1 file changed, 29 insertions(+) > > diff --git

Re: [pve-devel] [PATCH manager 1/4] api: pools: add limits management

2024-04-11 Thread Wolfgang Bumiller
On Wed, Apr 10, 2024 at 03:13:07PM +0200, Fabian Grünbichler wrote: > allow to set/update limits, and return them when querying individual pools. > > Signed-off-by: Fabian Grünbichler > --- > > Notes: > requires bumped pve-access-control > > PVE/API2/Pool.pm | 36

Re: [pve-devel] [PATCH guest-common 1/1] helpers: add pool limit/usage helpers

2024-04-11 Thread Wolfgang Bumiller
On Wed, Apr 10, 2024 at 03:13:06PM +0200, Fabian Grünbichler wrote: > one for combining the per-node broadcasted values, one for checking a pool's > limit, and one specific helper for checking guest-related actions such as > starting a VM. > > Signed-off-by: Fabian Grünbichler > --- >

[pve-devel] [PATCH access-control 0/1] Unstable Sorting of Items in Config Files in /etc/pve

2024-04-11 Thread Daniel Krambrock via pve-devel
--- Begin Message --- As explaind in #5335: In the user.cfg file, any modifications lead to a reordering of ACL entries within the file. Adding a `sort` in AccessControl.pm fixes this. Daniel Krambrock (1): fix #5335: sort ACL entries in user.cfg src/PVE/AccessControl.pm | 2 +- 1 file

[pve-devel] [PATCH firewall 1/1] fix #5335: sort cluster.fw entries in ALIASES section

2024-04-11 Thread Daniel Krambrock via pve-devel
--- Begin Message --- --- src/PVE/Firewall.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 77cbaf4..81a8798 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -3360,7 +3360,7 @@ my $format_aliases = sub { my

[pve-devel] [PATCH access-control 1/1] fix #5335: sort ACL entries in user.cfg

2024-04-11 Thread Daniel Krambrock via pve-devel
--- Begin Message --- --- src/PVE/AccessControl.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm index 21f93ff..47f2d38 100644 --- a/src/PVE/AccessControl.pm +++ b/src/PVE/AccessControl.pm @@ -951,7 +951,7 @@ sub

[pve-devel] [PATCH firewall 0/1] Unstable Sorting of Items in Config Files in /etc/pve

2024-04-11 Thread Daniel Krambrock via pve-devel
--- Begin Message --- As explaind in #5335: The ALIASES section within the firewall/cluster.fw file suffers from sorting instability. Each modification to the file results in a reshuffling of ALIASES entries. Adding a `sort` in Firewall.pm fixes this. Daniel Krambrock (1): fix #5335: sort

Re: [pve-devel] [PATCH qemu-server v8 2/4] fix #3784: Parameter for guest vIOMMU + test-cases

2024-04-11 Thread Thomas Lamprecht
Am 24/01/2024 um 10:49 schrieb Markus Frank: > vIOMMU is the emulation of a hardware IOMMU within a virtual machine, > providing improved memory access control and security for virtualized I/O > devices. > vIOMMU also enables the option to passthrough pci devices to L2 VMs > in L1 VMs via Nested

[pve-devel] applied: [PATCH qemu-server v8 1/4] machine as property-string

2024-04-11 Thread Thomas Lamprecht
Am 24/01/2024 um 10:49 schrieb Markus Frank: > Convert the machine parameter to a property-string and use the > machine type as the default key for backward compatibility. > > Signed-off-by: Markus Frank > --- > PVE/API2/Qemu.pm | 9 +++-- > PVE/QemuConfig.pm | 3 ++- >

Re: [pve-devel] [RFC container/firewall/manager/proxmox-firewall/qemu-server 00/37] proxmox firewall nftables implementation

2024-04-11 Thread Stefan Hanreich
On 4/11/24 09:34, Thomas Lamprecht wrote: > Am 11/04/2024 um 07:21 schrieb Stefan Hanreich: >>> Since `Command` is serializable anyway, we could have a nice test suite of >>> firewall/VM config files and expected commands as JSON dumps. >>> This will be tedious to setup at first, but will help to

Re: [pve-devel] [PATCH manager 1/2] ui: dc: backup: improve UX for the different 'notification-mode's

2024-04-11 Thread Thomas Lamprecht
Am 14/12/2023 um 11:26 schrieb Maximiliano Sandoval: > > Lukas Wagner writes: > >> - Switch order of 'mailto' and 'mailnotification' field >> - When mode is 'auto', disable 'mailtnotification' field >> - When mode is 'auto' and 'mailto' is empty, show >> hint that the notification

Re: [pve-devel] [RFC container/firewall/manager/proxmox-firewall/qemu-server 00/37] proxmox firewall nftables implementation

2024-04-11 Thread Thomas Lamprecht
Am 11/04/2024 um 07:21 schrieb Stefan Hanreich: >> Since `Command` is serializable anyway, we could have a nice test suite of >> firewall/VM config files and expected commands as JSON dumps. >> This will be tedious to setup at first, but will help to detect any unwanted >> regressions in the

Re: [pve-devel] [PATCH container 7/7] update: handle pool limits

2024-04-11 Thread Fabian Grünbichler
On April 10, 2024 3:13 pm, Fabian Grünbichler wrote: > Signed-off-by: Fabian Grünbichler > --- > src/PVE/API2/LXC/Config.pm | 21 + > 1 file changed, 21 insertions(+) > > diff --git a/src/PVE/API2/LXC/Config.pm b/src/PVE/API2/LXC/Config.pm > index e6c0980..3fb3885 100644 >