Re: [pve-devel] Roadmap - improved SDN support

2019-07-10 Thread Naumann, Thomas
Hi,

at datacenter of university we are working on a saas-platform to
automaticly deploy VMs, container, networks, services via web-gui for
students and employies. So, we are searching for a technicly basis for
this project. 
Because "Proxmox" runs several Clusters in our datacenter and does a
great jobs we wonder if it could be a basis for this project. We known
there is also "Openstack" but I think it might be some kind of
overkill.
What do you think about that?

-- 
Thomas Naumann

Abteilung Netze und Kommunikation
Otto-von-Guericke Universität Magdeburg
Universitätsrechenzentrum
Universitätsplatz 2
39106 Magdeburg

fon: +49 391 67-58563
email: thomas.naum...@ovgu.de

On Wed, 2019-05-22 at 18:25 +0200, Alexandre DERUMIER wrote:
> Hi,
> 
> I'm working on it.
> 
> The main idea is to be able to define bridge/network at datacenter
> level,
> then deploy them to hosts and manage dynamic network reloading.
> 
> Theses bridge network will support vlan but also vxlan.
> (I'll try to finish this for proxmox6)
> 
> Then I'm working to implement vxlan with bgp evpn too (with anycast
> routing, full layer3 distributed vm network), to have a full sdn,
> with routing through frr.
> 
> When it'll be done, I think it'll be possible to add more services on
> top on 
> theses networks (dhcp server, nat,).
> 
> 
> Do you have a special need with sdn?
> 
> 
> - Mail original -
> De: "Naumann, Thomas" 
> À: "pve-devel" 
> Envoyé: Mardi 21 Mai 2019 08:46:51
> Objet: [pve-devel] Roadmap - improved SDN support
> 
> Hi, 
> 
> there is an extra point "improved SDN support" under roadmap in 
> official proxmox-wiki. Who can give a hint what this means in
> detail? 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH installer] use pve-efiboot-tool for systemd boot ESP preparation

2019-07-10 Thread Thomas Lamprecht
Signed-off-by: Thomas Lamprecht 
---
 debian/control | 12 ++--
 proxinstall| 20 ++--
 2 files changed, 12 insertions(+), 20 deletions(-)

diff --git a/debian/control b/debian/control
index 0bb7caa..97bf20b 100644
--- a/debian/control
+++ b/debian/control
@@ -12,12 +12,20 @@ Homepage: https://www.proxmox.com
 
 Package: pve-installer
 Architecture: all
-Depends: geoip-bin, libgtk3-webkit2-perl, squashfs-tools, ${perl:Depends}
+Depends: geoip-bin,
+ libgtk3-webkit2-perl,
+ pve-kernel-helper,
+ squashfs-tools,
+ ${perl:Depends},
 Description: Proxmox VE Installer
  This package contains the graphical installer for Proxmox VE.
 
 Package: pmg-installer
 Architecture: all
-Depends: geoip-bin, libgtk3-webkit2-perl, squashfs-tools, ${perl:Depends}
+Depends: geoip-bin,
+ libgtk3-webkit2-perl,
+ pve-kernel-helper,
+ squashfs-tools,
+ ${perl:Depends},
 Description: Proxmox Mail Gateway Installer
  This package contains the graphical installer for Proxmox Mail Gateway.
diff --git a/proxinstall b/proxinstall
index aff6c4c..b06da13 100755
--- a/proxinstall
+++ b/proxinstall
@@ -1154,24 +1154,8 @@ sub compute_swapsize {
 sub prepare_systemd_boot_esp {
 my ($espdev, $targetdir) = @_;
 
-my $espuuid = find_dev_by_uuid($espdev);
-my $espmp = "var/tmp/$espuuid";
-mkdir "$targetdir/$espmp";
-
-syscmd("mount -n $espdev -t vfat $targetdir/$espmp") == 0 ||
-   die "unable to mount ESP $espdev\n";
-
-File::Path::make_path("$targetdir/$espmp/EFI/proxmox") ||
-   die "unable to create directory $targetdir/$espmp/EFI/proxmox\n";
-
-syscmd("chroot $targetdir bootctl --path /$espmp install") == 0 ||
-   die "unable to install systemd-boot loader\n";
-write_config("timeout 3\ndefault proxmox-*\n",
-   "$targetdir/$espmp/loader/loader.conf");
-
-syscmd("umount $targetdir/$espmp") == 0 ||
-   die "unable to umount ESP $targetdir/$espmp\n";
-
+syscmd("chroot $targetdir pve-efiboot-tool init $espdev") == 0 ||
+   die "unable to init ESP and install systemd-boot loader on '$espdev'\n";
 }
 
 sub prepare_grub_efi_boot_esp {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager] fix #2265: enable discard for virtio-blk

2019-07-10 Thread Nick Chevsky
On Wed, Jul 10, 2019, 5:29 AM Stefan Reiter  wrote:

> The 'SSD emulation' checkbox is interesting, since 'rotation_rate' (which
> it changes) is not only not supported on virtio-blk, but seemingly only
> works on virtio-scsi and IDE. QEMU's SATA implementation has no mention
> that I could find, but it also doesn't seem to break it, so I left it as is
> for now.


QEMU's "ide-hd" code path (which is really a misnomer for ATA) implements
this for both "IDE" and "SATA" drives as we refer to them in Proxmox.

I did extensive research on this when I implemented the "SSD emulation"
flag. For background, see the links and code referenced in:

https://git.proxmox.com/?p=qemu-server.git;a=commit;h=6c875f9f315f2dc7a0a3a988622c2efdf089b171

Nick
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [RFC proxmox-ve/kernel-meta 00/15] ESP sync improvements

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 5:04 PM, Fabian Grünbichler wrote:
> move ESP sync and kernel autoremove snippets to pve-kernel-meta and
> introduce a new binary package there, to allow easier sharing with
> future PMG versions.
> 
> add a new pveesptool for formatting, initializing and refreshing synced
> ESPs. various small improvements and cleanups to the hooks as well.
> 
> proxmox-ve needs a bump accordingly, see last patch for pve-kernel-meta.
> 
> There is probably lots of room for improvement, some areas for potential
> follow-ups:
> - 'pveesptool mount' and 'umount' commands (for manual configuration,
>   cleanup, ...)
> - 'pveesptool scan' to list ESPs that might be added to the list of
>   synced ones
> - 'pveesptool remove' to remove an ESP from the list (and clean up its
>   contents?)
> - more helpers in the functions file
> - handle more edge/error case correctly in zz-pve-efiboot (there are
>   surely still some missing)
> - mktemp or something fancier for mountpoint directory creation, instead
>   of easy-to-guess hardcoded ones
> 
> CC: Wolfgang Bumiller 
> 
> (especially for the last two points, which should be looked at before the
> final release)
> 

applied, with minor followups, i.e., rename 'pveesptool' to 'pve-efiboot-tool'
and add 'help' to its usage output. The above list can be addressed one-by-one
now the infrastructure is really in place.

Much thanks to you and Stoiko!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [RFC proxmox-ve/kernel-meta 00/15] ESP sync improvements

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 6:12 PM, Thomas Lamprecht wrote:
> On 7/10/19 5:04 PM, Fabian Grünbichler wrote:
>> - mktemp or something fancier for mountpoint directory creation, instead
>>   of easy-to-guess hardcoded ones
> 
> We could just use a mount namespace, e.g., with
> # unshare --mount
> 
> Then the outside would not see our mounts, at least unpriv. users..

E.g., something like the following:

8<
diff --git a/bin/pveesptool b/bin/pveesptool
index 6bbf679..e4b3928 100755
--- a/bin/pveesptool
+++ b/bin/pveesptool
@@ -92,6 +92,28 @@ format() {
exit 0
 }
 
+do_esp_install() {
+   part="$1"
+   UUID="$2"
+
+   esp_mp="/var/tmp/espmounts/$UUID"
+
+   mkdir -p "$esp_mp"
+   echo "Mounting '$part' on '$esp_mp'."
+   mount -t vfat "$part" "$esp_mp"
+
+   echo "Installing systemd-boot.."
+   mkdir -p "$esp_mp/$PMX_ESP_DIR"
+   bootctl --path "$esp_mp" install
+
+   echo "Configuring systemd-boot.."
+   echo "timeout 3" > "$esp_mp/$PMX_LOADER_CONF.tmp"
+   echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp"
+   mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF"
+   echo "Unmounting '$part'."
+   umount "$part"
+}
+
 init() {
part="$1"
 
@@ -112,22 +134,8 @@ init() {
exit 1
fi
 
-   esp_mp="/var/tmp/espmounts/$UUID"
-
-   mkdir -p "$esp_mp"
-   echo "Mounting '$part' on '$esp_mp'."
-   mount -t vfat "$part" "$esp_mp"
-
-   echo "Installing systemd-boot.."
-   mkdir -p "$esp_mp/$PMX_ESP_DIR"
-   bootctl --path "$esp_mp" install
-
-   echo "Configuring systemd-boot.."
-   echo "timeout 3" > "$esp_mp/$PMX_LOADER_CONF.tmp"
-   echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp"
-   mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF"
-   echo "Unmounting '$part'."
-   umount "$part"
+   echo "Do real ESP initialization in mount namespace.."
+   unshare --mount --propagation private "$0" "do-esp-install" "$part" 
"$UUID"
 
echo "Adding '$part' to list of synced ESPs.."
if [ -e "$ESP_LIST" ]; then
@@ -199,6 +207,16 @@ case "$1" in
init "$@"
exit 0
;;
+   'do-esp-install')
+   shift
+   if [ -z "$1" ] || [ -z "$2" ]; then
+   warn "E:  and  are mandatory."
+   warn ""
+   exit 1
+   fi
+   do_esp_install "$@"
+   exit 0
+   ;;
'refresh')
shift
refresh
--


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [RFC proxmox-ve/kernel-meta 00/15] ESP sync improvements

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 5:04 PM, Fabian Grünbichler wrote:
> move ESP sync and kernel autoremove snippets to pve-kernel-meta and
> introduce a new binary package there, to allow easier sharing with
> future PMG versions.
> 
> add a new pveesptool for formatting, initializing and refreshing synced
> ESPs. various small improvements and cleanups to the hooks as well.
> 
> proxmox-ve needs a bump accordingly, see last patch for pve-kernel-meta.
> 
> There is probably lots of room for improvement, some areas for potential
> follow-ups:
> - 'pveesptool mount' and 'umount' commands (for manual configuration,
>   cleanup, ...)
> - 'pveesptool scan' to list ESPs that might be added to the list of
>   synced ones
> - 'pveesptool remove' to remove an ESP from the list (and clean up its
>   contents?)
> - more helpers in the functions file
> - handle more edge/error case correctly in zz-pve-efiboot (there are
>   surely still some missing)
> - mktemp or something fancier for mountpoint directory creation, instead
>   of easy-to-guess hardcoded ones

We could just use a mount namespace, e.g., with
# unshare --mount

Then the outside would not see our mounts, at least unpriv. users..

> 
> CC: Wolfgang Bumiller 
> 
> (especially for the last two points, which should be looked at before the
> final release)


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 06/13] buildsys: minor cleanups

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 64d98db..d432599 100644
--- a/Makefile
+++ b/Makefile
@@ -3,8 +3,6 @@ include /usr/share/dpkg/pkg-info.mk
 export KERNEL_VER=5.0
 export KERNEL_ABI=5.0.15-1-pve
 
-PACKAGE=pve-kernel-${KERNEL_VER}
-
 GITVERSION:=$(shell git rev-parse HEAD)
 
 KERNEL_DEB=pve-kernel-${KERNEL_VER}_${DEB_VERSION_UPSTREAM_REVISION}_all.deb
@@ -15,6 +13,8 @@ BUILD_DIR=build
 DEBS=${KERNEL_DEB} ${HEADERS_DEB}
 
 all: deb
+
+.PHONY: deb
 deb: ${DEBS}
 
 ${HEADERS_DEB}: ${KERNEL_DEB}
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 07/13] pve-kernel-helper: add new package

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

with (sh) pveesptool to format, initialize and refresh synced ESPs, as
well as the hooks previously shipped via proxmox-ve.

pveesptool based on a never-released version written in perl by Stoiko
Ivanov.

Co-Developed-By: Stoiko Ivanov 

Signed-off-by: Fabian Grünbichler 
---
 Makefile |  17 ++-
 bin/Makefile |  12 ++
 efiboot/Makefile |  18 +++
 debian/control.in|  14 ++
 bin/pveesptool   | 219 +++
 debian/pve-kernel-helper.install |   3 +
 debian/rules |  15 +--
 7 files changed, 281 insertions(+), 17 deletions(-)
 create mode 100644 bin/Makefile
 create mode 100644 efiboot/Makefile
 create mode 100755 bin/pveesptool
 create mode 100644 debian/pve-kernel-helper.install

diff --git a/Makefile b/Makefile
index d432599..74527eb 100644
--- a/Makefile
+++ b/Makefile
@@ -7,12 +7,17 @@ GITVERSION:=$(shell git rev-parse HEAD)
 
 KERNEL_DEB=pve-kernel-${KERNEL_VER}_${DEB_VERSION_UPSTREAM_REVISION}_all.deb
 HEADERS_DEB=pve-headers-${KERNEL_VER}_${DEB_VERSION_UPSTREAM_REVISION}_all.deb
+HELPER_DEB=pve-kernel-helper_${DEB_VERSION_UPSTREAM_REVISION}_all.deb
 
 BUILD_DIR=build
 
-DEBS=${KERNEL_DEB} ${HEADERS_DEB}
+DEBS=${KERNEL_DEB} ${HEADERS_DEB} ${HELPER_DEB}
 
-all: deb
+SUBDIRS = efiboot bin
+
+.PHONY: all
+all: ${SUBDIRS}
+   set -e && for i in ${SUBDIRS}; do ${MAKE} -C $$i; done
 
 .PHONY: deb
 deb: ${DEBS}
@@ -21,11 +26,15 @@ ${HEADERS_DEB}: ${KERNEL_DEB}
 ${KERNEL_DEB}: debian
rm -rf ${BUILD_DIR}
mkdir -p ${BUILD_DIR}/debian
-   cp -ar debian/* ${BUILD_DIR}/debian/
+   rsync -a * ${BUILD_DIR}/
cd ${BUILD_DIR}; debian/rules debian/control
echo "git clone git://git.proxmox.com/git/pve-kernel-meta.git\\ngit 
checkout ${GITVERSION}" > ${BUILD_DIR}/debian/SOURCE
cd ${BUILD_DIR}; dpkg-buildpackage -b -uc -us
-   lintian ${KERNEL_DEB} ${HEADERS_DEB}
+   lintian ${DEBS}
+
+.PHONY: install
+install: ${SUBDIRS}
+   set -e && for i in ${SUBDIRS}; do ${MAKE} -C $$i $@; done
 
 .PHONY: upload
 upload: ${DEBS}
diff --git a/bin/Makefile b/bin/Makefile
new file mode 100644
index 000..c5ac674
--- /dev/null
+++ b/bin/Makefile
@@ -0,0 +1,12 @@
+SBINDIR=${DESTDIR}/usr/sbin
+
+.PHONY: all
+all:
+
+install:
+   install -d ${SBINDIR}
+   install -m 0755 pveesptool ${SBINDIR}/
+
+.PHONY: clean distclean
+distclean:
+clean:
diff --git a/efiboot/Makefile b/efiboot/Makefile
new file mode 100644
index 000..dfda9b4
--- /dev/null
+++ b/efiboot/Makefile
@@ -0,0 +1,18 @@
+KERNEL_HOOKSCRIPTS = pve-auto-removal zz-pve-efiboot
+SHARE_FILES = functions
+
+HOOKDIR = ${DESTDIR}/etc/kernel/postinst.d
+SHARE_SCRIPTDIR = ${DESTDIR}/usr/share/pve-kernel-helper/scripts
+
+.PHONY: all
+all:
+
+install:
+   install -d ${HOOKDIR}
+   install -m 0755 ${KERNEL_HOOKSCRIPTS} ${HOOKDIR}
+   install -d ${SHARE_SCRIPTDIR}
+   install -m 0755 ${SHARE_FILES} ${SHARE_SCRIPTDIR}
+
+.PHONY: clean distclean
+distclean:
+clean:
diff --git a/debian/control.in b/debian/control.in
index 4972ea7..f6d9f4d 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -25,3 +25,17 @@ Depends: pve-firmware,
 Description: Latest Proxmox VE Kernel Image
  This is a virtual package which will install the latest available
  proxmox kernel from the @KERNEL_VER@ series.
+
+Package: pve-kernel-helper
+Architecture: all
+Section: admin
+Priority: optional
+Depends: dosfstools,
+ gdisk,
+ systemd,
+ udev,
+ ${misc:Depends},
+Description: Function for various kernel maintenance tasks.
+ .
+ This package includes kernel-hooks for marking certain kernels as
+ NeverAutoRemove and helpers for systemd-boot
diff --git a/bin/pveesptool b/bin/pveesptool
new file mode 100755
index 000..601c1e9
--- /dev/null
+++ b/bin/pveesptool
@@ -0,0 +1,219 @@
+#!/bin/sh
+
+set -e
+
+. /usr/share/pve-kernel-helper/scripts/functions
+
+
+_get_partition_info() {
+   if [ ! -e "$1" ]; then
+   warn "E: '$1' does not exist!"
+   exit 1
+   fi
+   bdev=$(realpath "$1")
+   if [ ! -b "$bdev" ]; then
+   warn "E: '$bdev' is not a block device!"
+   exit 1
+   fi
+
+   bdev_info=$( \
+ lsblk \
+ --bytes \
+ --pairs \
+ -o 'UUID,SIZE,FSTYPE,PARTTYPE,PKNAME,MOUNTPOINT' \
+ "$bdev" \
+ )
+   if [ -z "$bdev_info" ]; then
+   warn "E: unable to get information about block device '$1'!"
+   exit 1
+   fi
+
+   count=$(echo "$bdev_info" | grep -c '^')
+   if [ "$count" -ne '1' ]; then
+   echo "$bdev_info"
+   warn "E: block device '$1' has children!"
+   exit 1
+   fi
+
+   echo "$bdev_info"
+   eval "$bdev_info"
+
+   if [ -z "$PKNAME" ]; then
+   warn "E: cannot 

[pve-devel] [PATCH kernel-meta 04/13] zz-pve-efiboot: adapt to static list of ESPs

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

read a list of vfat UUIDs from /etc/kernel/pve-efiboot-uuids instead of
taking all blockdevices with vfat and the proper partition type

Otherwise all guests having an ESP would get it mounted and potentially
modified
---
 efiboot/zz-pve-efiboot | 33 +
 1 file changed, 25 insertions(+), 8 deletions(-)

diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index ad78caf..1ba0538 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -6,6 +6,7 @@ set -e
 # https://kernel-team.pages.debian.net/kernel-handbook/ch-update-hooks.html
 
 # relative to the ESP mountpoint
+ESP_LIST="/etc/kernel/pve-efiboot-uuids"
 PMX_ESP_DIR="EFI/proxmox"
 
 MOUNTROOT="${TMPDIR:-/var/tmp}/espmounts"
@@ -42,19 +43,35 @@ else
CMDLINE="$(cat /proc/cmdline)"
 fi
 
+potential_esps(){
+   lsblk --list -o PATH,UUID,FSTYPE,PARTTYPE,MOUNTPOINT |
+   awk '$3 == "vfat" && $4 == "c12a7328-f81f-11d2-ba4b-00a0c93ec93b" && $5 
== "" {print $1,$2}'
+}
 
 update_esps() {
-   esps="$(lsblk --list -o PATH,UUID,FSTYPE,PARTTYPE,MOUNTPOINT |
-   awk -v OFS=';' '$3 == "vfat" && $4 == 
"c12a7328-f81f-11d2-ba4b-00a0c93ec93b" && $5 == "" {print $1,$2}')"
+   if [ ! -f "${ESP_LIST}" ]; then
+   warn "No ${ESP_LIST} found cannot update installed kernels - 
exiting"
+   esps="$(potential_esps)"
+   warn "add the UUIDs of one of the following devices to ${ESP_LIST}:"
+   warn "${esps}"
+   exit 1
+   fi
+   cat "${ESP_LIST}" | while IFS= read -r uuid; do
+   if ! (echo "${uuid}" | grep -qE 
'[0-9a-fA-F]{4}-[0-9a-fA-F]{4}'); then
+   warn "WARN: ${uuid} read from ${ESP_LIST} does not look 
like a VFAT-UUID - skipping"
+   continue
+   fi
 
-   for esp in ${esps}; do
-   path="$(echo "${esp}" | cut -d ';' -f1)"
-   uuid="$(echo "${esp}" | cut -d ';' -f2)"
-   mountpoint="${MOUNTROOT}/${uuid}"
+   path="/dev/disk/by-uuid/$uuid"
+   if [ ! -e "${path}" ]; then
+   warn "WARN: ${path} does not exist - clean ${ESP_LIST}! 
- skipping"
+   continue
+   fi
 
+   mountpoint="${MOUNTROOT}/${uuid}"
mkdir -p "${mountpoint}"
mount "${path}" "${mountpoint}" || \
-   { warn "mount of ${esp} failed - skipping"; continue; }
+   { warn "mount of ${path} failed - skipping"; continue; }
if [ ! -f "${mountpoint}/loader/loader.conf" ]; then
warn "${path} contains no loader.conf - skipping"
continue
@@ -69,7 +86,7 @@ update_esps() {
remove_old_kernels "${mountpoint}"
 
umount "${mountpoint}" || \
-   { warn "umount of ${esp} failed - failure"; exit 2; }
+   { warn "umount of ${path} failed - failure"; exit 2; }
 
rmdir "${mountpoint}"
done
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 08/13] esptool: move more variables to functions

2019-07-10 Thread Fabian Grünbichler
which is sourced by all scripts anyway.

Signed-off-by: Fabian Grünbichler 
---
 efiboot/functions  |  7 +++
 efiboot/zz-pve-efiboot | 11 ---
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/efiboot/functions b/efiboot/functions
index 1327bb4..cb6cd62 100755
--- a/efiboot/functions
+++ b/efiboot/functions
@@ -1,6 +1,13 @@
 #! /bin/sh
 set -e
 
+ESP_LIST="/etc/kernel/pve-efiboot-uuids"
+ESPTYPE='c12a7328-f81f-11d2-ba4b-00a0c93ec93b'
+
+# relative to the ESP mountpoint
+PMX_ESP_DIR="EFI/proxmox"
+PMX_LOADER_CONF="loader/loader.conf"
+
 # adapted from /etc/kernel/postinst.d/apt-auto-removal as present in
 # debian's apt package:
 #
diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index 8d5d625..560856f 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -5,9 +5,6 @@ set -e
 # /usr/lib/kernel/install.d/90-loaderentry.install, see also
 # https://kernel-team.pages.debian.net/kernel-handbook/ch-update-hooks.html
 
-# relative to the ESP mountpoint
-ESP_LIST="/etc/kernel/pve-efiboot-uuids"
-PMX_ESP_DIR="EFI/proxmox"
 
 MOUNTROOT="${TMPDIR:-/var/tmp}/espmounts"
 
@@ -49,7 +46,7 @@ potential_esps(){
 
 update_esps() {
if [ ! -f "${ESP_LIST}" ]; then
-   warn "No ${ESP_LIST} found cannot update installed kernels - 
exiting"
+   warn "No ${ESP_LIST} found, cannot update installed kernels - 
exiting"
esps="$(potential_esps)"
warn "add the UUIDs of one of the following devices to ${ESP_LIST}:"
warn "${esps}"
@@ -71,12 +68,12 @@ update_esps() {
mkdir -p "${mountpoint}"
mount "${path}" "${mountpoint}" || \
{ warn "mount of ${path} failed - skipping"; continue; }
-   if [ ! -f "${mountpoint}/loader/loader.conf" ]; then
+   if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
warn "${path} contains no loader.conf - skipping"
continue
fi
-   if [ ! -d "${mountpoint}/EFI/proxmox" ]; then
-   warn "${path} contains no EFI/proxmox - skipping"
+   if [ ! -d "${mountpoint}/$PMX_ESP_DIR" ]; then
+   warn "${path}/$PMX_ESP_DIR does not exist- skipping"
continue
fi
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 13/13] add B=R on proxmox-ve << 6.0-2~

2019-07-10 Thread Fabian Grünbichler
since we took over the efiboot files from there.

Signed-off-by: Fabian Grünbichler 
---
 debian/control.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/debian/control.in b/debian/control.in
index f6d9f4d..d40015c 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -35,6 +35,8 @@ Depends: dosfstools,
  systemd,
  udev,
  ${misc:Depends},
+Breaks: proxmox-ve (<< 6.0-2~)
+Replaces: proxmox-ve (<< 6.0-2~)
 Description: Function for various kernel maintenance tasks.
  .
  This package includes kernel-hooks for marking certain kernels as
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 12/13] efiboot: call on kernel removal as well

2019-07-10 Thread Fabian Grünbichler
but don't pass in newly 'installed' kernel version

Signed-off-by: Fabian Grünbichler 
---
 efiboot/Makefile |  9 ++---
 debian/pve-kernel-helper.install |  1 +
 efiboot/zz-pve-efiboot   | 12 
 3 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/efiboot/Makefile b/efiboot/Makefile
index dfda9b4..766faca 100644
--- a/efiboot/Makefile
+++ b/efiboot/Makefile
@@ -1,15 +1,18 @@
 KERNEL_HOOKSCRIPTS = pve-auto-removal zz-pve-efiboot
 SHARE_FILES = functions
 
-HOOKDIR = ${DESTDIR}/etc/kernel/postinst.d
+POSTINSTHOOKDIR = ${DESTDIR}/etc/kernel/postinst.d
+POSTRMHOOKDIR = ${DESTDIR}/etc/kernel/postrm.d
 SHARE_SCRIPTDIR = ${DESTDIR}/usr/share/pve-kernel-helper/scripts
 
 .PHONY: all
 all:
 
 install:
-   install -d ${HOOKDIR}
-   install -m 0755 ${KERNEL_HOOKSCRIPTS} ${HOOKDIR}
+   install -d ${POSTINSTHOOKDIR}
+   install -m 0755 ${KERNEL_HOOKSCRIPTS} ${POSTINSTHOOKDIR}
+   install -d ${POSTRMHOOKDIR}
+   install -m 0755 ${KERNEL_HOOKSCRIPTS} ${POSTRMHOOKDIR}
install -d ${SHARE_SCRIPTDIR}
install -m 0755 ${SHARE_FILES} ${SHARE_SCRIPTDIR}
 
diff --git a/debian/pve-kernel-helper.install b/debian/pve-kernel-helper.install
index f79ab7f..283459d 100644
--- a/debian/pve-kernel-helper.install
+++ b/debian/pve-kernel-helper.install
@@ -1,3 +1,4 @@
 etc/kernel/postinst.d/*
+etc/kernel/postrm.d/*
 usr/sbin/pveesptool
 usr/share/pve-kernel-helper/scripts/functions
diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index 06a1926..954dbec 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -31,8 +31,6 @@ trap cleanup EXIT INT TERM QUIT
 
 . /usr/share/pve-kernel-helper/scripts/functions
 
-BOOT_KVERS="$(boot_kernel_list "$@")"
-
 potential_esps(){
lsblk --list -o PATH,UUID,FSTYPE,PARTTYPE,MOUNTPOINT |
awk '$3 == "vfat" && $4 == "c12a7328-f81f-11d2-ba4b-00a0c93ec93b" && $5 
== "" {print $1,$2}'
@@ -152,8 +150,14 @@ case $0:$mode in
# time by calling update-grub multiple times on upgrade and removal.
# Also run if we have no DEB_MAINT_PARAMS, in order to work with old
# kernel packages.
-   
*/postinst.d/*:|*/postinst.d/*:configure|*/postrm.d/*:|*/postrm.d/*:remove)
-   update_esps
+   */postinst.d/*:|*/postinst.d/*:configure)
+   BOOT_KVERS="$(boot_kernel_list "$@")"
+   update_esps
+   ;;
+*/postrm.d/*:|*/postrm.d/*:remove)
+   # no newly installed kernel
+   BOOT_KVERS="$(boot_kernel_list)"
+   update_esps
;;
 esac
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 09/13] efiboot: move kernel CMDLINE check

2019-07-10 Thread Fabian Grünbichler
from global scope to where it is actually used, otherwise this gets
called/printed too often/early.

Signed-off-by: Fabian Grünbichler 
---
 efiboot/zz-pve-efiboot | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index 560856f..134d4ab 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -32,13 +32,6 @@ trap cleanup EXIT INT TERM QUIT
 
 BOOT_KVERS="$(boot_kernel_list "$@")"
 
-if [ -f /etc/kernel/cmdline ]; then
-   CMDLINE="$(cat /etc/kernel/cmdline)"
-else
-   warn "No /etc/kernel/cmdline found - falling back to /proc/cmdline"
-   CMDLINE="$(cat /proc/cmdline)"
-fi
-
 potential_esps(){
lsblk --list -o PATH,UUID,FSTYPE,PARTTYPE,MOUNTPOINT |
awk '$3 == "vfat" && $4 == "c12a7328-f81f-11d2-ba4b-00a0c93ec93b" && $5 
== "" {print $1,$2}'
@@ -52,6 +45,13 @@ update_esps() {
warn "${esps}"
exit 1
fi
+   if [ -f /etc/kernel/cmdline ]; then
+   CMDLINE="$(cat /etc/kernel/cmdline)"
+   else
+   warn "No /etc/kernel/cmdline found - falling back to 
/proc/cmdline"
+   CMDLINE="$(cat /proc/cmdline)"
+   fi
+
cat "${ESP_LIST}" | while IFS= read -r uuid; do
if ! (echo "${uuid}" | grep -qE 
'[0-9a-fA-F]{4}-[0-9a-fA-F]{4}'); then
warn "WARN: ${uuid} read from ${ESP_LIST} does not look 
like a VFAT-UUID - skipping"
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 03/13] zz-pve-efiboot: remove outdated comments

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

---
 efiboot/zz-pve-efiboot | 8 
 1 file changed, 8 deletions(-)

diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index b5e03b5..ad78caf 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -10,16 +10,8 @@ PMX_ESP_DIR="EFI/proxmox"
 
 MOUNTROOT="${TMPDIR:-/var/tmp}/espmounts"
 
-# TODO:
-# - no mount on /boot/efi - mount all available esps on /var/tmp/esp-UUID
-#   and copy the stuff for all of them (or copy onto first and sync for the
-#   others - or don't copy if unchanged
-# - trap error-conditions and make sure stuff gets unmounted
 # - cleanup - gently delete all kernels not in kernel-keep-list
 
-#[ -f "${LOADERDIR}/loader.conf" ] || exit 0
-#[ -d "${ESPMOUNT}/${PMX_ESP_DIR}" ] || exit 0
-
 if command -V systemd-detect-virt >/dev/null 2>&1 &&
systemd-detect-virt --quiet --container; then
exit 0
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 10/13] efiboot: be less verbose if UUID file does not exist

2019-07-10 Thread Fabian Grünbichler
this is the default for upgraded systems after all.

Signed-off-by: Fabian Grünbichler 
---
 efiboot/zz-pve-efiboot | 7 ++-
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index 134d4ab..5e4d6dc 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -39,11 +39,8 @@ potential_esps(){
 
 update_esps() {
if [ ! -f "${ESP_LIST}" ]; then
-   warn "No ${ESP_LIST} found, cannot update installed kernels - 
exiting"
-   esps="$(potential_esps)"
-   warn "add the UUIDs of one of the following devices to ${ESP_LIST}:"
-   warn "${esps}"
-   exit 1
+   warn "No ${ESP_LIST} found, skipping ESP sync."
+   exit 0
fi
if [ -f /etc/kernel/cmdline ]; then
CMDLINE="$(cat /etc/kernel/cmdline)"
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 02/13] efiboot: adapt includepaths to new package

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

diff --git a/efiboot/pve-auto-removal b/efiboot/pve-auto-removal
index e06bf0a..8aba624 100755
--- a/efiboot/pve-auto-removal
+++ b/efiboot/pve-auto-removal
@@ -1,7 +1,7 @@
 #! /bin/sh
 set -e
 
-. /usr/share/proxmox-ve/scripts/functions
+. /usr/share/pve-kernel-helper/scripts/functions
 
 eval "$(apt-config shell APT_CONF_D Dir::Etc::parts/d)"
 test -n "${APT_CONF_D}" || APT_CONF_D="/etc/apt/apt.conf.d"
diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index f3c0aee..b5e03b5 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -39,7 +39,7 @@ cleanup() {
 
 trap cleanup EXIT INT TERM QUIT
 
-. /usr/share/proxmox-ve/scripts/functions
+. /usr/share/pve-kernel-helper/scripts/functions
 
 BOOT_KVERS="$(boot_kernel_list "$@")"
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH proxmox-ve 1/2] remove efiboot

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

it got moved along with the new pveesptool to pve-kernel-meta
---
 debian/efiboot/functions|  58 
 debian/efiboot/pve-auto-removal |  39 
 debian/efiboot/zz-pve-efiboot   | 156 
 debian/proxmox-ve.install   |   3 -
 4 files changed, 256 deletions(-)
 delete mode 100755 debian/efiboot/functions
 delete mode 100755 debian/efiboot/pve-auto-removal
 delete mode 100755 debian/efiboot/zz-pve-efiboot

diff --git a/debian/efiboot/functions b/debian/efiboot/functions
deleted file mode 100755
index 1327bb4..000
--- a/debian/efiboot/functions
+++ /dev/null
@@ -1,58 +0,0 @@
-#! /bin/sh
-set -e
-
-# adapted from /etc/kernel/postinst.d/apt-auto-removal as present in
-# debian's apt package:
-#
-# Mark as not-for-autoremoval those kernel packages that are:
-#  - the currently booted version
-#  - the kernel version we've been called for
-#  - the latest kernel version (as determined by debian version number)
-#  - the second-latest kernel version
-#  - the latest kernel version of each series (e.g. 4.13, 4.15, 5.0) by
-#marking the meta-packages
-
-kernel_keep_versions() {
-   eval "$(apt-config shell DPKG Dir::bin::dpkg/f)"
-   test -n "$DPKG" || DPKG="/usr/bin/dpkg"
-
-   list="$("${DPKG}" -l | awk '/^[ih][^nc][ ]+pve-kernel-[0-9]+\./ && $2 
!~ /-dbg(:.*)?$/ && $2 !~ /-dbgsym(:.*)?$/ { print $2; }' \
-  | sed -e 's#^pve-kernel-##' -e 's#:[^:]\+ # #')"
-
-   sorted_list="$(echo "$list" | sort --unique --reverse --version-sort)"
-
-   [ -n "$1" ] && install_version="$1"
-
-   running_version="$(uname -r | tr 'A-Z' 'a-z')"
-
-   # ignore the currently running version if attempting a reproducible 
build
-   if [ -n "${SOURCE_DATE_EPOCH}" ]; then
-   running_version=""
-   fi
-
-   latest_2_versions="$(echo "$sorted_list" | grep -E '^[^ ]+-pve' | head 
-n2 )"
-
-   series_metapackages="$(echo "$sorted_list" | grep -Ev '^[^ ]+-pve')"
-
-   kernels="$(cat <<-EOF
-   $running_version
-   $install_version
-   $latest_2_versions
-   $series_metapackages
-   EOF
-   )"
-
-   echo "$kernels" | sort -u | sed -e '/^$/ d'
-}
-
-#bootable kernels are the same as the no_autoremove ones without the 
meta-package
-boot_kernel_list() {
-   list="$(kernel_keep_versions "$@")"
-
-   echo "$list" | grep -E '^[^ ]+-pve'
-
-}
-
-warn() {
-   echo "$@" 1>&2
-}
diff --git a/debian/efiboot/pve-auto-removal b/debian/efiboot/pve-auto-removal
deleted file mode 100755
index e06bf0a..000
--- a/debian/efiboot/pve-auto-removal
+++ /dev/null
@@ -1,39 +0,0 @@
-#! /bin/sh
-set -e
-
-. /usr/share/proxmox-ve/scripts/functions
-
-eval "$(apt-config shell APT_CONF_D Dir::Etc::parts/d)"
-test -n "${APT_CONF_D}" || APT_CONF_D="/etc/apt/apt.conf.d"
-
-config_file="${APT_CONF_D}/76pveconf"
-
-generate_apt_config() {
-
-   kernels="$(kernel_keep_versions "$@")"
-
-   cat <<- EOF
-   // DO NOT EDIT! File autogenerated by $0
-   APT::NeverAutoRemove
-   {
-   EOF
-   for kernel in $kernels; do
-   escaped_kver="$(echo "$kernel" |  sed -e 's#\([\.\+]\)#\\\1#g')"
-   echo "   \"^pve-kernel-${escaped_kver}$\";"
-   done
-   echo '};'
-   if [ "${APT_AUTO_REMOVAL_KERNELS_DEBUG:-false}" = 'true' ]; then
-   cat <<-EOF
-   /* Debug information:
-   # dpkg list:
-   $(dpkg -l | grep 'pve-kernel')
-   # list of installed kernel packages:
-   $kernels
-   */
-   EOF
-   fi
-}
-
-generate_apt_config "$@" > "${config_file}.dpkg-new"
-mv -f "${config_file}.dpkg-new" "$config_file"
-chmod 444 "$config_file"
diff --git a/debian/efiboot/zz-pve-efiboot b/debian/efiboot/zz-pve-efiboot
deleted file mode 100755
index f3c0aee..000
--- a/debian/efiboot/zz-pve-efiboot
+++ /dev/null
@@ -1,156 +0,0 @@
-#! /bin/sh
-set -e
-
-# adapted from '/etc/kernel/postinst.d/zz-update-grub and
-# /usr/lib/kernel/install.d/90-loaderentry.install, see also
-# https://kernel-team.pages.debian.net/kernel-handbook/ch-update-hooks.html
-
-# relative to the ESP mountpoint
-PMX_ESP_DIR="EFI/proxmox"
-
-MOUNTROOT="${TMPDIR:-/var/tmp}/espmounts"
-
-# TODO:
-# - no mount on /boot/efi - mount all available esps on /var/tmp/esp-UUID
-#   and copy the stuff for all of them (or copy onto first and sync for the
-#   others - or don't copy if unchanged
-# - trap error-conditions and make sure stuff gets unmounted
-# - cleanup - gently delete all kernels not in kernel-keep-list
-
-#[ -f "${LOADERDIR}/loader.conf" ] || exit 0
-#[ -d "${ESPMOUNT}/${PMX_ESP_DIR}" ] || exit 0
-
-if command -V systemd-detect-virt >/dev/null 2>&1 &&
-   systemd-detect-virt --quiet --container; then
-   exit 0
-fi
-
-cleanup() {
-
-   warn "unmounting ESPs"
-   for mount in "${MOUNTROOT}"/* ; do
-   if echo 

[pve-devel] [PATCH kernel-meta 11/13] efiboot: always exit cleanly

2019-07-10 Thread Fabian Grünbichler
otherwise the whole apt process stops.

Signed-off-by: Fabian Grünbichler 
---
 efiboot/zz-pve-efiboot | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index 5e4d6dc..06a1926 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -20,7 +20,8 @@ cleanup() {
for mount in "${MOUNTROOT}"/* ; do
if echo "${mount}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}' 
&& \
mountpoint -q "${mount}"; then
-   umount "${mount}"
+   umount "${mount}" || \
+   { warn "umount of ${mount} failed - failure"; exit 
0; }
fi
done
 
@@ -62,7 +63,8 @@ update_esps() {
fi
 
mountpoint="${MOUNTROOT}/${uuid}"
-   mkdir -p "${mountpoint}"
+   mkdir -p "${mountpoint}" || \
+   { warn "creation of mountpoint ${mountpoint} failed - 
skipping"; continue; }
mount "${path}" "${mountpoint}" || \
{ warn "mount of ${path} failed - skipping"; continue; }
if [ ! -f "${mountpoint}/$PMX_LOADER_CONF" ]; then
@@ -79,9 +81,9 @@ update_esps() {
remove_old_kernels "${mountpoint}"
 
umount "${mountpoint}" || \
-   { warn "umount of ${path} failed - failure"; exit 2; }
+   { warn "umount of ${path} failed - failure"; exit 0; }
 
-   rmdir "${mountpoint}"
+   rmdir "${mountpoint}" || true
done
 
 }
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH kernel-meta 01/13] add efiboot and autoremoval hooks

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

moved from proxmox-ve, in order to provide one package which contains all
relevant helper functions for pve-kernels
---
 efiboot/functions|  58 +++
 efiboot/pve-auto-removal |  39 ++
 efiboot/zz-pve-efiboot   | 156 +++
 3 files changed, 253 insertions(+)
 create mode 100755 efiboot/functions
 create mode 100755 efiboot/pve-auto-removal
 create mode 100755 efiboot/zz-pve-efiboot

diff --git a/efiboot/functions b/efiboot/functions
new file mode 100755
index 000..1327bb4
--- /dev/null
+++ b/efiboot/functions
@@ -0,0 +1,58 @@
+#! /bin/sh
+set -e
+
+# adapted from /etc/kernel/postinst.d/apt-auto-removal as present in
+# debian's apt package:
+#
+# Mark as not-for-autoremoval those kernel packages that are:
+#  - the currently booted version
+#  - the kernel version we've been called for
+#  - the latest kernel version (as determined by debian version number)
+#  - the second-latest kernel version
+#  - the latest kernel version of each series (e.g. 4.13, 4.15, 5.0) by
+#marking the meta-packages
+
+kernel_keep_versions() {
+   eval "$(apt-config shell DPKG Dir::bin::dpkg/f)"
+   test -n "$DPKG" || DPKG="/usr/bin/dpkg"
+
+   list="$("${DPKG}" -l | awk '/^[ih][^nc][ ]+pve-kernel-[0-9]+\./ && $2 
!~ /-dbg(:.*)?$/ && $2 !~ /-dbgsym(:.*)?$/ { print $2; }' \
+  | sed -e 's#^pve-kernel-##' -e 's#:[^:]\+ # #')"
+
+   sorted_list="$(echo "$list" | sort --unique --reverse --version-sort)"
+
+   [ -n "$1" ] && install_version="$1"
+
+   running_version="$(uname -r | tr 'A-Z' 'a-z')"
+
+   # ignore the currently running version if attempting a reproducible 
build
+   if [ -n "${SOURCE_DATE_EPOCH}" ]; then
+   running_version=""
+   fi
+
+   latest_2_versions="$(echo "$sorted_list" | grep -E '^[^ ]+-pve' | head 
-n2 )"
+
+   series_metapackages="$(echo "$sorted_list" | grep -Ev '^[^ ]+-pve')"
+
+   kernels="$(cat <<-EOF
+   $running_version
+   $install_version
+   $latest_2_versions
+   $series_metapackages
+   EOF
+   )"
+
+   echo "$kernels" | sort -u | sed -e '/^$/ d'
+}
+
+#bootable kernels are the same as the no_autoremove ones without the 
meta-package
+boot_kernel_list() {
+   list="$(kernel_keep_versions "$@")"
+
+   echo "$list" | grep -E '^[^ ]+-pve'
+
+}
+
+warn() {
+   echo "$@" 1>&2
+}
diff --git a/efiboot/pve-auto-removal b/efiboot/pve-auto-removal
new file mode 100755
index 000..e06bf0a
--- /dev/null
+++ b/efiboot/pve-auto-removal
@@ -0,0 +1,39 @@
+#! /bin/sh
+set -e
+
+. /usr/share/proxmox-ve/scripts/functions
+
+eval "$(apt-config shell APT_CONF_D Dir::Etc::parts/d)"
+test -n "${APT_CONF_D}" || APT_CONF_D="/etc/apt/apt.conf.d"
+
+config_file="${APT_CONF_D}/76pveconf"
+
+generate_apt_config() {
+
+   kernels="$(kernel_keep_versions "$@")"
+
+   cat <<- EOF
+   // DO NOT EDIT! File autogenerated by $0
+   APT::NeverAutoRemove
+   {
+   EOF
+   for kernel in $kernels; do
+   escaped_kver="$(echo "$kernel" |  sed -e 's#\([\.\+]\)#\\\1#g')"
+   echo "   \"^pve-kernel-${escaped_kver}$\";"
+   done
+   echo '};'
+   if [ "${APT_AUTO_REMOVAL_KERNELS_DEBUG:-false}" = 'true' ]; then
+   cat <<-EOF
+   /* Debug information:
+   # dpkg list:
+   $(dpkg -l | grep 'pve-kernel')
+   # list of installed kernel packages:
+   $kernels
+   */
+   EOF
+   fi
+}
+
+generate_apt_config "$@" > "${config_file}.dpkg-new"
+mv -f "${config_file}.dpkg-new" "$config_file"
+chmod 444 "$config_file"
diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
new file mode 100755
index 000..f3c0aee
--- /dev/null
+++ b/efiboot/zz-pve-efiboot
@@ -0,0 +1,156 @@
+#! /bin/sh
+set -e
+
+# adapted from '/etc/kernel/postinst.d/zz-update-grub and
+# /usr/lib/kernel/install.d/90-loaderentry.install, see also
+# https://kernel-team.pages.debian.net/kernel-handbook/ch-update-hooks.html
+
+# relative to the ESP mountpoint
+PMX_ESP_DIR="EFI/proxmox"
+
+MOUNTROOT="${TMPDIR:-/var/tmp}/espmounts"
+
+# TODO:
+# - no mount on /boot/efi - mount all available esps on /var/tmp/esp-UUID
+#   and copy the stuff for all of them (or copy onto first and sync for the
+#   others - or don't copy if unchanged
+# - trap error-conditions and make sure stuff gets unmounted
+# - cleanup - gently delete all kernels not in kernel-keep-list
+
+#[ -f "${LOADERDIR}/loader.conf" ] || exit 0
+#[ -d "${ESPMOUNT}/${PMX_ESP_DIR}" ] || exit 0
+
+if command -V systemd-detect-virt >/dev/null 2>&1 &&
+   systemd-detect-virt --quiet --container; then
+   exit 0
+fi
+
+cleanup() {
+
+   warn "unmounting ESPs"
+   for mount in "${MOUNTROOT}"/* ; do
+   if echo "${mount}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}' 
&& \
+   

[pve-devel] [PATCH kernel-meta 05/13] zz-pve-efiboot: remove unmount warning

2019-07-10 Thread Fabian Grünbichler
From: Stoiko Ivanov 

it is printed everytime (also on clean exit) and distracting

---
 efiboot/zz-pve-efiboot | 1 -
 1 file changed, 1 deletion(-)

diff --git a/efiboot/zz-pve-efiboot b/efiboot/zz-pve-efiboot
index 1ba0538..8d5d625 100755
--- a/efiboot/zz-pve-efiboot
+++ b/efiboot/zz-pve-efiboot
@@ -20,7 +20,6 @@ fi
 
 cleanup() {
 
-   warn "unmounting ESPs"
for mount in "${MOUNTROOT}"/* ; do
if echo "${mount}" | grep -qE '[0-9a-fA-F]{4}-[0-9a-fA-F]{4}' 
&& \
mountpoint -q "${mount}"; then
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH proxmox-ve 2/2] add dependency on pve-kernel-helper

2019-07-10 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler 
---
 debian/control | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/control b/debian/control
index fcc855f..e9a45c8 100644
--- a/debian/control
+++ b/debian/control
@@ -18,6 +18,7 @@ Depends: apt,
  openssh-client,
  openssh-server,
  pve-kernel-5.0,
+ pve-kernel-helper,
  pve-manager,
  pve-qemu-kvm,
  qemu-server,
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [RFC proxmox-ve/kernel-meta 00/15] ESP sync improvements

2019-07-10 Thread Fabian Grünbichler
move ESP sync and kernel autoremove snippets to pve-kernel-meta and
introduce a new binary package there, to allow easier sharing with
future PMG versions.

add a new pveesptool for formatting, initializing and refreshing synced
ESPs. various small improvements and cleanups to the hooks as well.

proxmox-ve needs a bump accordingly, see last patch for pve-kernel-meta.

There is probably lots of room for improvement, some areas for potential
follow-ups:
- 'pveesptool mount' and 'umount' commands (for manual configuration,
  cleanup, ...)
- 'pveesptool scan' to list ESPs that might be added to the list of
  synced ones
- 'pveesptool remove' to remove an ESP from the list (and clean up its
  contents?)
- more helpers in the functions file
- handle more edge/error case correctly in zz-pve-efiboot (there are
  surely still some missing)
- mktemp or something fancier for mountpoint directory creation, instead
  of easy-to-guess hardcoded ones

CC: Wolfgang Bumiller 

(especially for the last two points, which should be looked at before the
final release)

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager] cleanup: don't copy ceph's admin keyring to /etc/ceph

2019-07-10 Thread Alwin Antreich
As this is done only on the first node that runs pveceph init, it might
be confusing for users to see it only on one node in the cluster. The
client section in the ceph.conf references the key under /etc/pve/priv/,
so all ceph tools will look for it there anyway.

Signed-off-by: Alwin Antreich 
---
Note: after some more thought and discussion, it seems that the copy of
  the keyring to /etc/ceph is not needed

 PVE/Ceph/Tools.pm | 5 -
 1 file changed, 5 deletions(-)

diff --git a/PVE/Ceph/Tools.pm b/PVE/Ceph/Tools.pm
index ff00ba15..de99718e 100644
--- a/PVE/Ceph/Tools.pm
+++ b/PVE/Ceph/Tools.pm
@@ -241,11 +241,6 @@ sub get_or_create_admin_keyring {
"--cap osd 'allow *' " .
"--cap mds 'allow *' " .
"--cap mgr 'allow *' ");
-   # we do not want to overwrite it
-   if (! -f $ckeyring_path) {
-   run_command("cp $pve_ckeyring_path $ckeyring_path");
-   run_command("chown ceph:ceph $ckeyring_path");
-   }
 }
 return $pve_ckeyring_path;
 }
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 2/2] keyring: use ckeyring_path variable in chown cmd

2019-07-10 Thread Alwin Antreich
A fixed path for the ceph.client.admin.keyring was used in the chown
command. This patch uses the ckeyring_path variable instead, to minimize
changes should the path change.

Signed-off-by: Alwin Antreich 
---
 PVE/Ceph/Tools.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/PVE/Ceph/Tools.pm b/PVE/Ceph/Tools.pm
index d6d21ff2..ff00ba15 100644
--- a/PVE/Ceph/Tools.pm
+++ b/PVE/Ceph/Tools.pm
@@ -244,7 +244,7 @@ sub get_or_create_admin_keyring {
# we do not want to overwrite it
if (! -f $ckeyring_path) {
run_command("cp $pve_ckeyring_path $ckeyring_path");
-   run_command("chown ceph:ceph /etc/ceph/ceph.client.admin.keyring");
+   run_command("chown ceph:ceph $ckeyring_path");
}
 }
 return $pve_ckeyring_path;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 1/2] Fix: typo in ckeyring_path

2019-07-10 Thread Alwin Antreich
pveceph init failed, as it couldn't chown the keyring

Signed-off-by: Alwin Antreich 
---
 PVE/Ceph/Tools.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/PVE/Ceph/Tools.pm b/PVE/Ceph/Tools.pm
index f8ec4302..d6d21ff2 100644
--- a/PVE/Ceph/Tools.pm
+++ b/PVE/Ceph/Tools.pm
@@ -19,7 +19,7 @@ my $ceph_cfgpath = "$ceph_cfgdir/$ccname.conf";
 
 my $pve_mon_key_path = "/etc/pve/priv/$ccname.mon.keyring";
 my $pve_ckeyring_path = "/etc/pve/priv/$ccname.client.admin.keyring";
-my $ckeyring_path = "/etc/ceph/ceph.client.admin.keyrign";
+my $ckeyring_path = "/etc/ceph/ceph.client.admin.keyring";
 my $ceph_bootstrap_osd_keyring = "/var/lib/ceph/bootstrap-osd/$ccname.keyring";
 my $ceph_bootstrap_mds_keyring = "/var/lib/ceph/bootstrap-mds/$ccname.keyring";
 my $ceph_mds_data_dir = '/var/lib/ceph/mds';
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied-series: [PATCH docs v2 0/2] add initial documentation for systemd-boot

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 2:40 PM, Stoiko Ivanov wrote:
> v1->v2:
> * incorporated Thomas' feedback (huge thanks!)
> * incorporated Aaron's feedback (huge thanks!)
> 
> Sending the v2, despite still not having addressed Fabian's quite valid point
> about the need to document a disk-change of a bootable disk, because of my
> being on vacation for the next 1.5 weeks. Will gladly pick the remaining task
> up once I'm back.
> 
> With the recent changes to the installer [0] we should adapt the documentation
> to include a short chapter on bootloaders to address the 2 ways of editing the
> kernel command line.
> 
> Once merged and deployed online we should also adapt the relevant wiki-pages
> and link them to the new chapter.
> 
> After a quick search of the wiki the following list of pages might profit from
> an update (mostly as a reminder to myself which pages need updating, but also
> so that others can add pages I missed):
> https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks
> https://pve.proxmox.com/wiki/Recover_From_Grub_Failure
> https://pve.proxmox.com/wiki/Serial_Terminal
> https://pve.proxmox.com/wiki/Cloud-Init_FAQ
> https://pve.proxmox.com/wiki/Booting_a_ZFS_root_file_system_via_UEFI
> 
> I was wondering whether we want to make the sysadmin chapter available for
> reading on the command-line (e.g. by creating a manpage out of it, or by
> copying it to /usr/share/doc/pve-docs as plain-text) - then we could refer to
> the information also in man-pages.

applied, thanks! Followed up with name-spacing all references with "sysboot_".

And yes that makes sense, I pondered already over that when we had the issue
with the ceph repo links (which did not worked from pveceph manpage as the
sysadmin chapter which included it was not rendered to a manpage)..

> 
> Stoiko Ivanov (2):
>   Add documentation on bootloaders (systemd-boot)
>   Refer to the bootloader chapter in remaining docs
> 
>  local-zfs.adoc  |  20 ++---
>  qm-pci-passthrough.adoc |  26 +++---
>  sysadmin.adoc   |   2 +
>  system-booting.adoc | 174 
>  4 files changed, 197 insertions(+), 25 deletions(-)
>  create mode 100644 system-booting.adoc
> 


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 2/2] gui: ceph osd: rewrite osd panel and add scrub button

2019-07-10 Thread Dominik Csapak
this rewrite of the osd panel converts it to a static declaration of
the layout with a controller/viewmodel logic

while doing this, add icons to buttons, and add a scrub/deep scrub entry

Signed-off-by: Dominik Csapak 
---
i guess it would be easiest to review the result and not the diff,
since it is simply a big rewrite of the whole component..

 www/manager6/ceph/OSD.js | 789 +--
 1 file changed, 420 insertions(+), 369 deletions(-)

diff --git a/www/manager6/ceph/OSD.js b/www/manager6/ceph/OSD.js
index 31cd3ea3..b7c88475 100644
--- a/www/manager6/ceph/OSD.js
+++ b/www/manager6/ceph/OSD.js
@@ -167,8 +167,271 @@ Ext.define('PVE.node.CephOsdTree', {
 extend: 'Ext.tree.Panel',
 alias: ['widget.pveNodeCephOsdTree'],
 onlineHelp: 'chapter_pveceph',
+
+viewModel: {
+   data: {
+   nodename: '',
+   flags: [],
+   maxversion: '0',
+   versions: {},
+   isOsd: false,
+   downOsd: false,
+   upOsd: false,
+   inOsd: false,
+   outOsd: false,
+   osdid: '',
+   osdhost: '',
+   }
+},
+
+controller: {
+   xclass: 'Ext.app.ViewController',
+
+   reload: function() {
+   var me = this.getView();
+   var vm = this.getViewModel();
+   var nodename = vm.get('nodename');
+   var sm = me.getSelectionModel();
+   Proxmox.Utils.API2Request({
+url: "/nodes/" + nodename + "/ceph/osd",
+   waitMsgTarget: me,
+   method: 'GET',
+   failure: function(response, opts) {
+   var msg = response.htmlStatus;
+   PVE.Utils.showCephInstallOrMask(me, msg, nodename,
+   function(win){
+   me.mon(win, 'cephInstallWindowClosed', this.reload);
+   }
+   );
+   },
+   success: function(response, opts) {
+   var data = response.result.data;
+   var selected = me.getSelection();
+   var name;
+   if (selected.length) {
+   name = selected[0].data.name;
+   }
+   vm.set('versions', data.versions);
+   // extract max version
+   var maxversion = vm.get('maxversion');
+   Object.values(data.versions || 
{}).forEach(function(version) {
+   if (PVE.Utils.compare_ceph_versions(version, 
maxversion) > 0) {
+   maxversion = version;
+   }
+   });
+   vm.set('maxversion', maxversion);
+   sm.deselectAll();
+   me.setRootNode(data.root);
+   me.expandAll();
+   if (name) {
+   var node = me.getRootNode().findChild('name', name, 
true);
+   if (node) {
+   me.setSelection([node]);
+   }
+   }
+
+   var flags = data.flags.split(',');
+   vm.set('flags', flags);
+   var noout = flags.includes('noout');
+   me.down('#nooutBtn').setText(noout ? gettext("Unset noout") 
: gettext("Set noout"));
+   }
+   });
+   },
+
+   osd_cmd: function(comp) {
+   var me = this;
+   var vm = this.getViewModel();
+   var cmd = comp.cmd;
+   var params = comp.params || {};
+   var osdid = vm.get('osdid');
+
+   var doRequest = function() {
+   Proxmox.Utils.API2Request({
+   url: "/nodes/" + vm.get('osdhost') + "/ceph/osd/" + osdid + 
'/' + cmd,
+   waitMsgTarget: me.getView(),
+   method: 'POST',
+   params: params,
+   success: () => { me.reload(); },
+   failure: function(response, opts) {
+   Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+   }
+   });
+   };
+
+   if (cmd === 'scrub') {
+   Ext.MessageBox.defaultButton = params.deep === 1 ? 2 : 1;
+   Ext.Msg.show({
+   title: gettext('Confirm'),
+   icon: params.deep === 1 ? Ext.Msg.WARNING : 
Ext.Msg.QUESTION,
+   msg: params.deep !== 1 ?
+  Ext.String.format(gettext("Scrub OSD.{0}"), osdid) :
+  Ext.String.format(gettext("Deep Scrub OSD.{0}"), osdid) +
+  "Caution: This can reduce performance while it 
is running.",
+   buttons: Ext.Msg.YESNO,
+   callback: function(btn) {
+   if (btn !== 'yes') {
+   return;
+   }
+   doRequest();
+   }
+

[pve-devel] [PATCH manager 1/2] ceph: add osd scrub api call

2019-07-10 Thread Dominik Csapak
can be called to (deep) scrub a specific osd

Signed-off-by: Dominik Csapak 
---
is there a better permission we could use? (since we do not know the
storage name?)
 PVE/API2/Ceph/OSD.pm | 45 
 1 file changed, 45 insertions(+)

diff --git a/PVE/API2/Ceph/OSD.pm b/PVE/API2/Ceph/OSD.pm
index 85197107..0582d538 100644
--- a/PVE/API2/Ceph/OSD.pm
+++ b/PVE/API2/Ceph/OSD.pm
@@ -650,4 +650,49 @@ __PACKAGE__->register_method ({
return undef;
 }});
 
+__PACKAGE__->register_method ({
+name => 'scrub',
+path => '{osdid}/scrub',
+method => 'POST',
+description => "Instruct the OSD to scrub.",
+proxyto => 'node',
+protected => 1,
+permissions => {
+   check => ['perm', '/', [ 'Sys.Modify' ]],
+},
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   node => get_standard_option('pve-node'),
+   osdid => {
+   description => 'OSD ID',
+   type => 'integer',
+   },
+   deep => {
+   description => 'If set, instructs a deep scrub instead of a 
normal one.',
+   type => 'boolean',
+   optional => 1,
+   default => 0,
+   },
+   },
+},
+returns => { type => "null" },
+code => sub {
+   my ($param) = @_;
+
+   PVE::Ceph::Tools::check_ceph_inited();
+
+   my $osdid = $param->{osdid};
+   my $deep = $param->{deep} // 0;
+
+   my $rados = PVE::RADOS->new();
+
+   my $osdstat = &$get_osd_status($rados, $osdid); # osd exists?
+   my $prefix = $deep ? 'osd deep-scrub' : 'osd scrub';
+
+   $rados->mon_command({ prefix => $prefix, who => $osdid });
+
+   return undef;
+}});
+
 1;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager 2/2] filter unusable lxc mountoptions for rootfs

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 2:26 PM, Oguz Bektas wrote:
> disable nodev and noexec per thomas' suggestion[0].
> 
> [0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/037994.html
> 
> Signed-off-by: Oguz Bektas 
> ---
>  www/manager6/lxc/MPEdit.js | 10 ++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/www/manager6/lxc/MPEdit.js b/www/manager6/lxc/MPEdit.js
> index 7689340d..638cf6e1 100644
> --- a/www/manager6/lxc/MPEdit.js
> +++ b/www/manager6/lxc/MPEdit.js
> @@ -58,6 +58,14 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
>   if (mp.mountoptions) {
>   mp.mountoptions = mp.mountoptions.split(';');
>   }
> +
> + var field = me.down('field[name=mountoptions]');

you only need to get the field in the 'rootfs' case, and .down is not
completely cheap, so maybe only do this in the if branch?

> + var forbidden = ['nodev', 'noexec'];
> + if (this.confid === 'rootfs') {
> + filtered = field.comboItems.filter(e => !forbidden.includes(e[0]));
> + field.setComboItems(filtered);
> + }
> +
>   me.setValues(mp);
>  },
>  
> @@ -363,6 +371,8 @@ Ext.define('PVE.lxc.MountPointEdit', {
>   }
>  
>   ipanel.setMountPoint(mp);
> +
> +
>   me.isValid(); // trigger validation
>   }
>   }
> 


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager 2/2] filter unusable lxc mountoptions for rootfs

2019-07-10 Thread Dominik Csapak

looks good to me (except one comment inline)

two things though:

why do we allow those options in the backend for rootfs ? shouldn't we 
prevent a user from setting 'noexec' on the rootfs in the first place?

(ofc we have to hide it in the gui anyway)

why is nodev on root forbidden?

On 7/10/19 2:26 PM, Oguz Bektas wrote:

disable nodev and noexec per thomas' suggestion[0].

[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/037994.html

Signed-off-by: Oguz Bektas 
---
  www/manager6/lxc/MPEdit.js | 10 ++
  1 file changed, 10 insertions(+)

diff --git a/www/manager6/lxc/MPEdit.js b/www/manager6/lxc/MPEdit.js
index 7689340d..638cf6e1 100644
--- a/www/manager6/lxc/MPEdit.js
+++ b/www/manager6/lxc/MPEdit.js
@@ -58,6 +58,14 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
if (mp.mountoptions) {
mp.mountoptions = mp.mountoptions.split(';');
}
+
+   var field = me.down('field[name=mountoptions]');
+   var forbidden = ['nodev', 'noexec'];
+   if (this.confid === 'rootfs') {
+   filtered = field.comboItems.filter(e => !forbidden.includes(e[0]));
+   field.setComboItems(filtered);
+   }
+
me.setValues(mp);
  },
  
@@ -363,6 +371,8 @@ Ext.define('PVE.lxc.MountPointEdit', {

}
  
  		ipanel.setMountPoint(mp);

+
+


this seems unintentional ;)


me.isValid(); // trigger validation
}
}




___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH docs v2 1/2] Add documentation on bootloaders (systemd-boot)

2019-07-10 Thread Stoiko Ivanov
With the recently added support for booting ZFS on root on EFI systems via
`systemd-boot` the documentation needs adapting (mostly related to editing
the kernel commandline).

This patch adds a short section on Bootloaders to the sysadmin chapter
describing both `grub` and PVE's use of `systemd-boot`

Signed-off-by: Stoiko Ivanov 
---
 sysadmin.adoc   |   2 +
 system-booting.adoc | 174 
 2 files changed, 176 insertions(+)
 create mode 100644 system-booting.adoc

diff --git a/sysadmin.adoc b/sysadmin.adoc
index 21537f1..e045610 100644
--- a/sysadmin.adoc
+++ b/sysadmin.adoc
@@ -74,6 +74,8 @@ include::local-zfs.adoc[]
 
 include::certificate-management.adoc[]
 
+include::system-booting.adoc[]
+
 endif::wiki[]
 
 
diff --git a/system-booting.adoc b/system-booting.adoc
new file mode 100644
index 000..38dacb1
--- /dev/null
+++ b/system-booting.adoc
@@ -0,0 +1,174 @@
+[[system_booting]]
+Bootloaders
+---
+ifdef::wiki[]
+:pve-toplevel:
+endif::wiki[]
+
+{pve} uses one of two bootloaders depending on the disk setup selected in the
+installer.
+
+For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is
+used. All other deployments use the standard `grub` bootloader (this usually
+also applies to systems which are installed on top of Debian).
+
+[[installer_partitioning_scheme]]
+Partitioning scheme used by the installer
+~
+
+The {pve} installer creates 3 partitions on the bootable disks selected for
+installation. The bootable disks are:
+
+* For Installations with `ext4` or `xfs` the selected disk
+
+* For ZFS installations all disks belonging to the first `vdev`:
+** The first disk for RAID0
+** All disks for RAID1, RAIDZ1, RAIDZ2, RAIDZ3
+** The first two disks for RAID10
+
+The created partitions are:
+
+* a 1 MB BIOS Boot Partition (gdisk type EF02)
+
+* a 512 MB EFI System Partition (ESP, gdisk type EF00)
+
+* a third partition spanning the set `hdsize` parameter or the remaining space
+used for the chosen storage type
+
+`grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
+Partition of all bootable disks for supporting older systems.
+
+
+Grub
+
+
+`grub` has been the de-facto standard for booting Linux systems for many years
+and is quite well documented
+footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
+
+The kernel and initrd images are taken from `/boot` and its configuration file
+`/boot/grub/grub.cfg` gets updated by the kernel installation process.
+
+Configuration
+^
+Changes to the `grub` configuration are done via the defaults file
+`/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate
+the `/boot/grub/grub.cfg` after a change to the configuration run:
+
+
+`update-grub`.
+
+
+Systemd-boot
+
+
+`systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd
+images directly from the EFI Service Partition (ESP) where it is installed.
+The main advantage of directly loading the kernel from the ESP is that it does
+not need to reimplement the drivers for accessing the storage.  In the context
+of ZFS as root filesystem this means that you can use all optional features on
+your root pool instead of the subset which is also present in the ZFS
+implementation in `grub` or having to create a separate small boot-pool
+footnote:[Booting ZFS on root with grub 
https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS].
+
+In setups with redundancy (RAID1, RAID10, RAIDZ*) all bootable disks (those
+being part of the first `vdev`) are partitioned with an ESP. This ensures the
+system boots even if the first boot device fails.  The ESPs are kept in sync by
+a kernel postinstall hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. The
+script copies certain kernel versions and the initrd images to `EFI/proxmox/`
+on the root of each ESP and creates the appropriate config files in
+`loader/entries/proxmox-*.conf`.
+
+The following kernel versions are configured by default:
+
+* the currently running kernel
+* the version being newly installed on package updates
+* the two latest kernels
+* the latest version of each kernel series (e.g. 4.15, 5.0).
+
+The ESPs are not kept mounted during regular operation, in contrast to `grub`,
+which keeps an ESP mounted on `/boot/efi`. This helps to prevent filesystem
+corruption to the `vfat` formatted ESPs in case of a system crash, and removes
+the need to manually adapt `/etc/fstab` in case the primary boot device fails.
+
+[[systemd_boot_config]]
+Configuration
+^
+
+`systemd-boot` is configured via the file `loader/loader.conf` in the root
+directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage
+for details.
+
+Each bootloader entry is placed in a file of its own in the directory
+`loader/entries/`
+
+An example entry.conf looks like this (`/` refers to the root of the ESP):
+
+

[pve-devel] [PATCH docs v2 2/2] Refer to the bootloader chapter in remaining docs

2019-07-10 Thread Stoiko Ivanov
Editing the kernel commandline is described centrally in the bootloaders
chapter. Refer to it where appropriate (qm-pci-passthrough.adoc).

Additionally update the documentation on ZFS as rpool to the inclusion of
`systemd-boot`

Signed-off-by: Stoiko Ivanov 
---
 local-zfs.adoc  | 20 ++--
 qm-pci-passthrough.adoc | 26 +++---
 2 files changed, 21 insertions(+), 25 deletions(-)

diff --git a/local-zfs.adoc b/local-zfs.adoc
index 13f6050..aae89e0 100644
--- a/local-zfs.adoc
+++ b/local-zfs.adoc
@@ -154,15 +154,9 @@ rpool/swap4.25G  7.69T64K  -
 Bootloader
 ~~
 
-The default ZFS disk partitioning scheme does not use the first 2048
-sectors. This gives enough room to install a GRUB boot partition. The
-{pve} installer automatically allocates that space, and installs the
-GRUB boot loader there. If you use a redundant RAID setup, it installs
-the boot loader on all disk required for booting. So you can boot
-even if some disks fail.
-
-NOTE: It is not possible to use ZFS as root file system with UEFI
-boot.
+Depending on whether the system is booted in EFI or legacy BIOS mode the
+{pve} installer sets up either `grub` or `systemd-boot` as main bootloader.
+See the chapter on  xref:system_booting[bootladers] for details.
 
 
 ZFS Administration
@@ -255,7 +249,13 @@ can be used as cache.
 
 .Changing a failed device
 
- zpool replace -f   
+ zpool replace -f   
+
+.Changing a failed bootable device when using systemd-boot
+
+ sgdisk  -R 
+ sgdisk -G 
+ zpool replace -f   
 
 
 Activate E-Mail Notification
diff --git a/qm-pci-passthrough.adoc b/qm-pci-passthrough.adoc
index 3895df4..452ecb7 100644
--- a/qm-pci-passthrough.adoc
+++ b/qm-pci-passthrough.adoc
@@ -45,9 +45,10 @@ some configuration to enable PCI(e) passthrough.
 
 .IOMMU
 
-The IOMMU has to be activated on the kernel commandline. The easiest way is to
-enable trough grub. Edit `'/etc/default/grub'' and add the following to the
-'GRUB_CMDLINE_LINUX_DEFAULT' variable:
+The IOMMU has to be activated on the
+xref:edit_kernel_cmdline[kernel commandline].
+
+The command line parameters are:
 
 * for Intel CPUs:
 +
@@ -60,12 +61,6 @@ enable trough grub. Edit `'/etc/default/grub'' and add the 
following to the
  amd_iommu=on
 
 
-[[qm_pci_passthrough_update_grub]]
-To bring this change in effect, make sure you run:
-
-
-# update-grub
-
 
 .Kernel Modules
 
@@ -87,6 +82,9 @@ After changing anything modules related, you need to refresh 
your
 # update-initramfs -u -k all
 
 
+If you are using `systemd-boot` make sure to
+xref:systemd-boot-refresh[sync the new initramfs to the bootable partitions].
+
 .Finish Configuration
 
 Finally reboot to bring the changes into effect and check that it is indeed
@@ -316,10 +314,9 @@ Intels drivers for GVT-g are integrated in the Kernel and 
should work
 with 5th, 6th and 7th generation Intel Core Processors, as well as E3 v4, E3
 v5 and E3 v6 Xeon Processors.
 
-To enable it for Intel Graphcs, you have to make sure to load the module
-'kvmgt' (for example via `/etc/modules`) and to enable it on the Kernel
-commandline. For this you can edit `'/etc/default/grub'' and add the following
-to the 'GRUB_CMDLINE_LINUX_DEFAULT' variable:
+To enable it for Intel Graphics, you have to make sure to load the module
+'kvmgt' (for example via `/etc/modules`) and to enable it on the
+xref:edit_kernel_cmdline[Kernel commandline] and add the following parameter:
 
 
  i915.enable_gvt=1
@@ -327,8 +324,7 @@ to the 'GRUB_CMDLINE_LINUX_DEFAULT' variable:
 
 After that remember to
 xref:qm_pci_passthrough_update_initramfs[update the `initramfs`],
-xref:qm_pci_passthrough_update_grub[update grub] and
-reboot your host.
+and reboot your host.
 
 VM Configuration
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH docs v2 0/2] add initial documentation for systemd-boot

2019-07-10 Thread Stoiko Ivanov
v1->v2:
* incorporated Thomas' feedback (huge thanks!)
* incorporated Aaron's feedback (huge thanks!)

Sending the v2, despite still not having addressed Fabian's quite valid point
about the need to document a disk-change of a bootable disk, because of my
being on vacation for the next 1.5 weeks. Will gladly pick the remaining task
up once I'm back.

With the recent changes to the installer [0] we should adapt the documentation
to include a short chapter on bootloaders to address the 2 ways of editing the
kernel command line.

Once merged and deployed online we should also adapt the relevant wiki-pages
and link them to the new chapter.

After a quick search of the wiki the following list of pages might profit from
an update (mostly as a reminder to myself which pages need updating, but also
so that others can add pages I missed):
https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks
https://pve.proxmox.com/wiki/Recover_From_Grub_Failure
https://pve.proxmox.com/wiki/Serial_Terminal
https://pve.proxmox.com/wiki/Cloud-Init_FAQ
https://pve.proxmox.com/wiki/Booting_a_ZFS_root_file_system_via_UEFI

I was wondering whether we want to make the sysadmin chapter available for
reading on the command-line (e.g. by creating a manpage out of it, or by
copying it to /usr/share/doc/pve-docs as plain-text) - then we could refer to
the information also in man-pages.

Stoiko Ivanov (2):
  Add documentation on bootloaders (systemd-boot)
  Refer to the bootloader chapter in remaining docs

 local-zfs.adoc  |  20 ++---
 qm-pci-passthrough.adoc |  26 +++---
 sysadmin.adoc   |   2 +
 system-booting.adoc | 174 
 4 files changed, 197 insertions(+), 25 deletions(-)
 create mode 100644 system-booting.adoc

-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 2/2] filter unusable lxc mountoptions for rootfs

2019-07-10 Thread Oguz Bektas
disable nodev and noexec per thomas' suggestion[0].

[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/037994.html

Signed-off-by: Oguz Bektas 
---
 www/manager6/lxc/MPEdit.js | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/www/manager6/lxc/MPEdit.js b/www/manager6/lxc/MPEdit.js
index 7689340d..638cf6e1 100644
--- a/www/manager6/lxc/MPEdit.js
+++ b/www/manager6/lxc/MPEdit.js
@@ -58,6 +58,14 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
if (mp.mountoptions) {
mp.mountoptions = mp.mountoptions.split(';');
}
+
+   var field = me.down('field[name=mountoptions]');
+   var forbidden = ['nodev', 'noexec'];
+   if (this.confid === 'rootfs') {
+   filtered = field.comboItems.filter(e => !forbidden.includes(e[0]));
+   field.setComboItems(filtered);
+   }
+
me.setValues(mp);
 },
 
@@ -363,6 +371,8 @@ Ext.define('PVE.lxc.MountPointEdit', {
}
 
ipanel.setMountPoint(mp);
+
+
me.isValid(); // trigger validation
}
}
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH widget-toolkit 1/2] KVComboBox: add setComboItems function

2019-07-10 Thread Oguz Bektas
this allows to change the comboItems of a KVComboBox on the run

Signed-off-by: Oguz Bektas 
---
 form/KVComboBox.js | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/form/KVComboBox.js b/form/KVComboBox.js
index 0970fcf..5e76f6f 100644
--- a/form/KVComboBox.js
+++ b/form/KVComboBox.js
@@ -71,5 +71,12 @@ Ext.define('Proxmox.form.KVComboBox', {
}
 
me.callParent();
+},
+
+setComboItems: function(items) {
+   var me = this;
+
+   me.getStore().setData(items);
 }
+
 });
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] spice new streaming option

2019-07-10 Thread Alexandre DERUMIER
>>looks great, Dominik means that the "high frame speed" detection 
>>may produce some overhead on the host, so maybe we could add it as 
>>opt-in flag for the spice display? 

yes, maybe add an simple option like : streaming=off|filter|all ? 








Alexandre Derumier 
Ingénieur système et stockage 

Manager Infrastructure 


Fixe : +33 3 59 82 20 10 



125 Avenue de la république 
59110 La Madeleine 
[ https://twitter.com/OdisoHosting ] [ https://twitter.com/mindbaz ] [ 
https://www.linkedin.com/company/odiso ] [ 
https://www.viadeo.com/fr/company/odiso ] [ 
https://www.facebook.com/monsiteestlent ] 

[ https://www.monsiteestlent.com/ | MonSiteEstLent.com ] - Blog dédié à la 
webperformance et la gestion de pics de trafic 






De: "Thomas Lamprecht"  
À: "pve-devel" , "aderumier"  
Cc: "Dominik Csapak"  
Envoyé: Mercredi 10 Juillet 2019 13:43:50 
Objet: Re: [pve-devel] spice new streaming option 

On 7/9/19 6:22 PM, Alexandre DERUMIER wrote: 
> Hi, 
> 
> I have open a bugzilla 
> https://bugzilla.proxmox.com/show_bug.cgi?id=2272 
> 
> seem that since spice 0.14.1, they are a new option to stream video from 
> guest directly to spice client. 
> 
> This need a new qemu device 
> 
> -device 
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel1,id=channel1,name=org.spice-space.stream.0
>  \ 
> -chardev spiceport,name=org.spice-space.stream.0,id=charchannel1 
> 
> 
> I don't have tested it yet, but maybe it could be added before proxmox6 final 
> release ? 
> (add it for qemu machine >= 4) 
> 

looks great, Dominik means that the "high frame speed" detection 
may produce some overhead on the host, so maybe we could add it as 
opt-in flag for the spice display? 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] spice new streaming option

2019-07-10 Thread Thomas Lamprecht
On 7/9/19 6:22 PM, Alexandre DERUMIER wrote:
> Hi,
> 
> I have open a bugzilla
> https://bugzilla.proxmox.com/show_bug.cgi?id=2272
> 
> seem that since spice  0.14.1, they are a new option to stream video from 
> guest directly to spice client.
> 
> This need a new qemu device
> 
> -device 
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel1,id=channel1,name=org.spice-space.stream.0
>  \
> -chardev spiceport,name=org.spice-space.stream.0,id=charchannel1
> 
> 
> I don't have tested it yet, but maybe it could be added before proxmox6 final 
> release ?
> (add it for qemu machine >= 4)
> 

looks great, Dominik means that the "high frame speed" detection
may produce some overhead on the host, so maybe we could add it as
opt-in flag for the spice display?

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied-series: [PATCH v2 firewall 1/2] ebtables: remove PVE chains properly

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 11:59 AM, Fabian Grünbichler wrote:
> when globally disabling the FW, or on shutdown of firewall service.
> 
> Signed-off-by: Fabian Grünbichler 
> ---
> Notes:
> v1->v2: drop condition, add comment
> 

applied series, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH widget-toolkit 1/3] node network: hide ip and netmask by default

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 11:30 AM, Dominik Csapak wrote:
> CIDR column displays the same information in only on column,
> no need to duplicate the information (by default)
> 
> Signed-off-by: Dominik Csapak 
> ---
>  node/NetworkView.js | 2 ++
>  1 file changed, 2 insertions(+)
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH widget-toolkit 2/3] node network: refactor bond mode array generation

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 11:30 AM, Dominik Csapak wrote:
> Signed-off-by: Dominik Csapak 
> ---
>  Utils.js | 23 +++
>  form/BondModeSelector.js | 30 +++---
>  2 files changed, 38 insertions(+), 15 deletions(-)
> 
> diff --git a/Utils.js b/Utils.js
> index 7a78fda..929de6e 100644
> --- a/Utils.js
> +++ b/Utils.js
> @@ -99,6 +99,29 @@ Ext.define('Proxmox.Utils', { utilities: {
>   return data;
>  },
>  
> +bond_mode_gettext_map: {
> + '802.3ad': 'LACP (802.3ad)',
> + 'lacp-balance-slb': 'LACP (balance-slb)',
> + 'lacp-balance-tcp': 'LACP (balance-tcp)',
> +},
> +
> +render_bond_mode: function(value) {
> + var val = Proxmox.Utils.bond_mode_gettext_map[value];
> + if (!val) {
> + val = value || '';
> + }
> + return val;
> +},
> +
> +bond_mode_array: function(modes) {
> + var arr = [];
> + var i;
> + for (i = 0; i < modes.length; i++) {
> + arr.push([modes[i], Proxmox.Utils.render_bond_mode(modes[i])]);
> + }
> + return arr;
> +},
> +

applied, with followup below, thanks!

8<
diff --git a/Utils.js b/Utils.js
index 929de6e..9b62bf6 100644
--- a/Utils.js
+++ b/Utils.js
@@ -105,21 +105,10 @@ Ext.define('Proxmox.Utils', { utilities: {
'lacp-balance-tcp': 'LACP (balance-tcp)',
 },
 
-render_bond_mode: function(value) {
-   var val = Proxmox.Utils.bond_mode_gettext_map[value];
-   if (!val) {
-   val = value || '';
-   }
-   return val;
-},
+render_bond_mode: value => Proxmox.Utils.bond_mode_gettext_map[value] || 
value || '',
 
 bond_mode_array: function(modes) {
-   var arr = [];
-   var i;
-   for (i = 0; i < modes.length; i++) {
-   arr.push([modes[i], Proxmox.Utils.render_bond_mode(modes[i])]);
-   }
-   return arr;
+   return modes.map(mode => [mode, Proxmox.Utils.render_bond_mode(mode)]);
 },
 
 getNoSubKeyHtml: function(url) {
--


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH widget-toolkit 3/3] node network: add Bond Mode and Hash Policy columns

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 11:30 AM, Dominik Csapak wrote:
> Signed-off-by: Dominik Csapak 
> ---
>  node/NetworkView.js | 10 ++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/node/NetworkView.js b/node/NetworkView.js
> index 2c4d851..af2da87 100644
> --- a/node/NetworkView.js
> +++ b/node/NetworkView.js
> @@ -323,6 +323,16 @@ Ext.define('Proxmox.node.NetworkView', {
>   dataIndex: 'type',
>   renderer: render_ports
>   },
> + {
> + header: gettext('Bond Mode'),
> + dataIndex: 'bond_mode',
> + renderer: Proxmox.Utils.render_bond_mode,
> + },
> + {
> + header: gettext('Hash Policy'),
> + hidden: true,
> + dataIndex: 'bond_xmit_hash_policy',
> + },
>   {
>   header: gettext('IP address'),
>   sortable: true,
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager] fix #2265: enable discard for virtio-blk

2019-07-10 Thread Stefan Reiter
QEMU 4.0 supports this. Guest needs to have at least Kernel 5.0 for it
to be used.

SSD emulation is still not supported AFAICT, so no change here.

Signed-off-by: Stefan Reiter 
---

The 'SSD emulation' checkbox is interesting, since 'rotation_rate' (which it
changes) is not only not supported on virtio-blk, but seemingly only works on
virtio-scsi and IDE. QEMU's SATA implementation has no mention that I could
find, but it also doesn't seem to break it, so I left it as is for now.

 www/manager6/qemu/HDEdit.js | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/www/manager6/qemu/HDEdit.js b/www/manager6/qemu/HDEdit.js
index 0cabb598..46397d37 100644
--- a/www/manager6/qemu/HDEdit.js
+++ b/www/manager6/qemu/HDEdit.js
@@ -25,10 +25,8 @@ Ext.define('PVE.qemu.HDInputPanel', {
}
 
var virtio = value.match(/^virtio/);
-   this.lookup('discard').setDisabled(virtio);
this.lookup('ssd').setDisabled(virtio);
if (virtio) {
-   this.lookup('discard').setValue(false);
this.lookup('ssd').setValue(false);
}
 
@@ -258,7 +256,6 @@ Ext.define('PVE.qemu.HDInputPanel', {
{
xtype: 'proxmoxcheckbox',
fieldLabel: gettext('Discard'),
-   disabled: me.confid && me.confid.match(/^virtio/),
reference: 'discard',
name: 'discard'
}
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v2 firewall 1/2] ebtables: remove PVE chains properly

2019-07-10 Thread Fabian Grünbichler
when globally disabling the FW, or on shutdown of firewall service.

Signed-off-by: Fabian Grünbichler 
---
Notes:
v1->v2: drop condition, add comment

 src/PVE/Firewall.pm | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 96c45e9..3b67186 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -4269,6 +4269,7 @@ sub remove_pvefw_chains {
 PVE::Firewall::remove_pvefw_chains_iptables("iptables");
 PVE::Firewall::remove_pvefw_chains_iptables("ip6tables");
 PVE::Firewall::remove_pvefw_chains_ipset();
+PVE::Firewall::remove_pvefw_chains_ebtables();
 
 }
 
@@ -4314,6 +4315,11 @@ sub remove_pvefw_chains_ipset {
 ipset_restore_cmdlist($cmdlist) if $cmdlist;
 }
 
+sub remove_pvefw_chains_ebtables {
+# apply empty ruleset = remove all our chains
+ebtables_restore_cmdlist(get_ebtables_cmdlist({}));
+}
+
 sub init {
 my $cluster_conf = load_clusterfw_conf();
 my $cluster_options = $cluster_conf->{options};
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v2 firewall 2/2] ebtables: treat chain deletion as change

2019-07-10 Thread Fabian Grünbichler
since it is one. otherwise, 'pve-firewall compile' will print false
information as long as the deletion has not been applied yet.

Signed-off-by: Fabian Grünbichler 
---
new in v2

 src/PVE/Firewall.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 3b67186..41f48d6 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -4033,8 +4033,8 @@ sub get_ebtables_cmdlist {
 
 foreach my $chain (sort keys %$statushash) {
my $stat = $statushash->{$chain};
-   next if ($stat->{action} eq 'delete');
$changes = 1 if ($stat->{action} !~ 'ignore|exists');
+   next if ($stat->{action} eq 'delete');
 
foreach my $cmd (@{$statushash->{$chain}->{'rules'}}) {
if ($chain eq 'FORWARD' && $cmd eq $append_pve_to_forward) {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH widget-toolkit 1/3] node network: hide ip and netmask by default

2019-07-10 Thread Dominik Csapak
CIDR column displays the same information in only on column,
no need to duplicate the information (by default)

Signed-off-by: Dominik Csapak 
---
 node/NetworkView.js | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/node/NetworkView.js b/node/NetworkView.js
index e3f7e60..2c4d851 100644
--- a/node/NetworkView.js
+++ b/node/NetworkView.js
@@ -327,6 +327,7 @@ Ext.define('Proxmox.node.NetworkView', {
header: gettext('IP address'),
sortable: true,
width: 120,
+   hidden: true,
dataIndex: 'address',
renderer: renderer_generator('address'),
},
@@ -334,6 +335,7 @@ Ext.define('Proxmox.node.NetworkView', {
header: gettext('Subnet mask'),
width: 120,
sortable: true,
+   hidden: true,
dataIndex: 'netmask',
renderer: renderer_generator('netmask'),
},
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH widget-toolkit 3/3] node network: add Bond Mode and Hash Policy columns

2019-07-10 Thread Dominik Csapak
Signed-off-by: Dominik Csapak 
---
 node/NetworkView.js | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/node/NetworkView.js b/node/NetworkView.js
index 2c4d851..af2da87 100644
--- a/node/NetworkView.js
+++ b/node/NetworkView.js
@@ -323,6 +323,16 @@ Ext.define('Proxmox.node.NetworkView', {
dataIndex: 'type',
renderer: render_ports
},
+   {
+   header: gettext('Bond Mode'),
+   dataIndex: 'bond_mode',
+   renderer: Proxmox.Utils.render_bond_mode,
+   },
+   {
+   header: gettext('Hash Policy'),
+   hidden: true,
+   dataIndex: 'bond_xmit_hash_policy',
+   },
{
header: gettext('IP address'),
sortable: true,
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH widget-toolkit 2/3] node network: refactor bond mode array generation

2019-07-10 Thread Dominik Csapak
Signed-off-by: Dominik Csapak 
---
 Utils.js | 23 +++
 form/BondModeSelector.js | 30 +++---
 2 files changed, 38 insertions(+), 15 deletions(-)

diff --git a/Utils.js b/Utils.js
index 7a78fda..929de6e 100644
--- a/Utils.js
+++ b/Utils.js
@@ -99,6 +99,29 @@ Ext.define('Proxmox.Utils', { utilities: {
return data;
 },
 
+bond_mode_gettext_map: {
+   '802.3ad': 'LACP (802.3ad)',
+   'lacp-balance-slb': 'LACP (balance-slb)',
+   'lacp-balance-tcp': 'LACP (balance-tcp)',
+},
+
+render_bond_mode: function(value) {
+   var val = Proxmox.Utils.bond_mode_gettext_map[value];
+   if (!val) {
+   val = value || '';
+   }
+   return val;
+},
+
+bond_mode_array: function(modes) {
+   var arr = [];
+   var i;
+   for (i = 0; i < modes.length; i++) {
+   arr.push([modes[i], Proxmox.Utils.render_bond_mode(modes[i])]);
+   }
+   return arr;
+},
+
 getNoSubKeyHtml: function(url) {
// url 
http://www.proxmox.com/products/proxmox-ve/subscription-service-plans
return Ext.String.format('You do not have a valid subscription for this 
server. Please visit www.proxmox.com to get a 
list of available options.', url || 'https://www.proxmox.com');
diff --git a/form/BondModeSelector.js b/form/BondModeSelector.js
index 8781ebc..26c2747 100644
--- a/form/BondModeSelector.js
+++ b/form/BondModeSelector.js
@@ -8,22 +8,22 @@ Ext.define('Proxmox.form.BondModeSelector', {
var me = this;
 
if (me.openvswitch) {
-   me.comboItems = [
-  ['active-backup', 'active-backup'],
-  ['balance-slb', 'balance-slb'],
-  ['lacp-balance-slb', 'LACP (balance-slb)'],
-  ['lacp-balance-tcp', 'LACP (balance-tcp)']
-  ];
+   me.comboItems = Proxmox.Utils.bond_mode_array([
+  'active-backup',
+  'balance-slb',
+  'lacp-balance-slb',
+  'lacp-balance-tcp',
+   ]);
} else {
-me.comboItems = [
-   ['balance-rr', 'balance-rr'],
-   ['active-backup', 'active-backup'],
-   ['balance-xor', 'balance-xor'],
-   ['broadcast', 'broadcast'],
-   ['802.3ad', 'LACP (802.3ad)'],
-   ['balance-tlb', 'balance-tlb'],
-   ['balance-alb', 'balance-alb']
-   ];
+   me.comboItems = Proxmox.Utils.bond_mode_array([
+   'balance-rr',
+   'active-backup',
+   'balance-xor',
+   'broadcast',
+   '802.3ad',
+   'balance-tlb',
+   'balance-alb',
+   ]);
}
 
me.callParent();
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 docs 2/2] Update pvecm documentation for corosync 3

2019-07-10 Thread Thomas Lamprecht
On 7/9/19 12:25 PM, Stefan Reiter wrote:
> Parts about multicast and RRP have been removed entirely. Instead, a new
> section 'Corosync Redundancy' has been added explaining the concept of
> links and link priorities.
> 
> Signed-off-by: Stefan Reiter 
> ---
> 
> v1 -> v2:
>  * Spelling mistakes
>  * Rewording to improve clarity
>  * Fixed redundancy explanation and example
>  * Added note about multiple clusters in one network
> 
> Didn't want to add back the entire section for the last point, would have
> basically been a heading with a single sentence below. I think the note is
> enough.
> 

applied, but re-added a reworded "Multiple Clusters In Same Network", people
had often question about the possibility and implications of this, so a separate
section for visibility is still nice, IMO.

Anyway, much thanks for the improved docs, appreciated!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH v2 cluster] fix #2270: allow custom lxc options to be restored as root

2019-07-10 Thread Thomas Lamprecht
On 7/9/19 5:20 PM, Stefan Reiter wrote:
> Seems to be a regression introduced with
> f360d7f16b094fa258cf82d2557d06f3284435e4 (related to #2028).
> $conf->{'lxc'} would always be defined, hence we never replaced it with
> the restored options.
> 
> Co-developed-by: Oguz Bektas 
> Signed-off-by: Stefan Reiter 
> ---
> 
> Nevermind v1, perl arrays and hashes are confusing. This time it works with
> multiple custom options as well ;)

Oh, and FYI: your repo tag is wrong here.
And perl hashes and arrays itself aren't to bad, but yes, the mix of
both needs some syntactic bending, though.. ;-)

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 docs 1/2] Use correct xref: syntax and add pvecm prefix

2019-07-10 Thread Thomas Lamprecht
On 7/9/19 12:25 PM, Stefan Reiter wrote:
> Signed-off-by: Stefan Reiter 
> ---
> 
> No changes for v2. As mentioned, I did not find any references to the changed
> names.
> 
>  pvecm.adoc | 30 +++---
>  1 file changed, 15 insertions(+), 15 deletions(-)
> 

applied, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 cluster] fix #2270: allow custom lxc options to be restored as root

2019-07-10 Thread Thomas Lamprecht
On 7/9/19 5:20 PM, Stefan Reiter wrote:
> Seems to be a regression introduced with
> f360d7f16b094fa258cf82d2557d06f3284435e4 (related to #2028).
> $conf->{'lxc'} would always be defined, hence we never replaced it with
> the restored options.
> 
> Co-developed-by: Oguz Bektas 
> Signed-off-by: Stefan Reiter 
> ---
> 
> Nevermind v1, perl arrays and hashes are confusing. This time it works with
> multiple custom options as well ;)
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH manager] pveproxy: allow to request debug sources directly over GET parameter

2019-07-10 Thread Thomas Lamprecht
On 7/10/19 9:38 AM, Thomas Lamprecht wrote:
> Avoids the need to stop the current pveproxy and restart it in debug
> mode, if one wants to get the debug un-minified sources for ExtJS,
> for example.
> 
> For a local PVE instance the following URL would ensure you load the
> debug sources:
> https://localhost:8006/?debug#v1:0:18:4:25:19::8:20:28
> 
> For convenience allow to omit the parameters value, and default to
> true, iow, ?debug and ?debug=1 are the same, one could force it off -
> even if the server was started in debug mode - by using ?debug=0
> 
> Signed-off-by: Thomas Lamprecht 
> ---
> 

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager] pveproxy: allow to request debug sources directly over GET parameter

2019-07-10 Thread Thomas Lamprecht
Avoids the need to stop the current pveproxy and restart it in debug
mode, if one wants to get the debug un-minified sources for ExtJS,
for example.

For a local PVE instance the following URL would ensure you load the
debug sources:
https://localhost:8006/?debug#v1:0:18:4:25:19::8:20:28

For convenience allow to omit the parameters value, and default to
true, iow, ?debug and ?debug=1 are the same, one could force it off -
even if the server was started in debug mode - by using ?debug=0

Signed-off-by: Thomas Lamprecht 
---

note: PMG has similar functionallity already, just that the convenience for
"debug" without parameter misses. Also the server debug flag cannot be
overwritten there.

 PVE/Service/pveproxy.pm | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/PVE/Service/pveproxy.pm b/PVE/Service/pveproxy.pm
index 4a515707..a07330c4 100755
--- a/PVE/Service/pveproxy.pm
+++ b/PVE/Service/pveproxy.pm
@@ -220,6 +220,11 @@ sub get_index {
$wtversion = $1;
 };
 
+my $debug = $server->{debug};
+if (exists $args->{debug}) {
+   $debug = !defined($args->{debug}) || $args->{debug};
+}
+
 my $vars = {
lang => $lang,
langfile => $langfile,
@@ -227,7 +232,7 @@ sub get_index {
token => $token,
console => $args->{console},
nodename => $nodename,
-   debug => $server->{debug},
+   debug => $debug,
version => "$version",
wtversion => $wtversion,
 };
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel