Re: [pve-devel] training week : students features request

2018-03-14 Thread Alexandre DERUMIER
>>- ceph: add replication like zfs to a remote ceph cluster for disaster 
>>recovery

looking at current replication implementation, I don't think it can be used for 
disaster recovery (as it should be another proxmox cluster, and current 
replication feature is more for local replication with failover on a another 
local storage)
.
Maybe simply add a backup + restore in 1 step to remote host could be easy. 
This could work with proxmox backup too, and also with storage snapshot 
backup/restore.

I'm seeing that vzdump have already a plugin architecture, I'll try to look at 
it.



- Mail original -
De: "aderumier" 
À: "pve-devel" 
Envoyé: Jeudi 15 Mars 2018 03:29:46
Objet: [pve-devel] training week : students features request

Hi, 

we have almost finished our training week, 

student are very happy with the hard work done on proxmox5, 
bugfixes,documentation and stability improvements. 


They have some features requests/suggestion 


- vm balancing/redispatching based on cpu 
- auto-snapshot feature (cron like) 
- global/central backup scheduler (be able to do backup sequentially of all 
vms, but not on all hosts at the same time) 
- incremental backup 
- ceph: backup implementation with rbd export-diff 
(I personally think that it could be great to implement alternative storage 
snasphot/export backup in storage plugins as an option , it's a lot faster on 
ceph for example, and support diff backup out of the box) 
- ceph: add replication like zfs to a remote ceph cluster for disaster recovery 
- fault-tolerance implementation (COLO) : (lot of work, seem that it'll be 
possible ready for qemu 2.12) 


Alexandre 
___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] training week : students features request

2018-03-14 Thread Alexandre DERUMIER
Hi,

we have almost finished our training week,

student are very happy with the hard work done on proxmox5, 
bugfixes,documentation and stability improvements.


They have some features requests/suggestion


- vm balancing/redispatching based on cpu
- auto-snapshot feature (cron like)
- global/central backup scheduler (be able to do backup sequentially of all 
vms, but not on all hosts at the same time)
- incremental backup 
- ceph: backup implementation with rbd export-diff 
   (I personally think that it could be great to implement alternative storage 
snasphot/export backup in storage plugins as an option , it's a lot faster on 
ceph for example, and support diff backup out of the box)
- ceph: add replication like zfs to a remote ceph cluster for disaster recovery
- fault-tolerance implementation (COLO) : (lot of work, seem that it'll be 
possible ready for qemu 2.12)


Alexandre
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH storage 07/11] Add CIFS dependencies in package management.

2018-03-14 Thread Fabian Grünbichler
On Wed, Mar 14, 2018 at 02:39:25PM +0100, Wolfgang Bumiller wrote:
> On Tue, Mar 13, 2018 at 03:11:07PM +0100, Wolfgang Link wrote:
> > ---
> >  debian/control | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/debian/control b/debian/control
> > index 04812bb..ffaefff 100644
> > --- a/debian/control
> > +++ b/debian/control
> > @@ -24,6 +24,8 @@ Depends: cstream,
> >   smartmontools,
> >   thin-provisioning-tools,
> >   udev,
> > +cifs-utils,
> > +smbclient,
> 
> Wrong indentation (tabs in the middle of a spaces-only list)

also wrong order ;)

wrap-and-sort (from 'devscripts') does both automatically

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH storage 08/11] Register CIFSPlugin in the storage plugin system.

2018-03-14 Thread Wolfgang Bumiller
On Tue, Mar 13, 2018 at 03:11:08PM +0100, Wolfgang Link wrote:
> diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm
> index 6f72cee..64d4e2f 100644
> --- a/PVE/Storage/Plugin.pm
> +++ b/PVE/Storage/Plugin.pm
> @@ -332,7 +332,7 @@ sub parse_config {
>   $d->{content} = $def->{content}->[1] if !$d->{content};
>   }
>  
> - if ($type eq 'iscsi' || $type eq 'nfs' || $type eq 'rbd' || $type eq 
> 'sheepdog' || $type eq 'iscsidirect' || $type eq 'glusterfs' || $type eq 
> 'zfs' || $type eq 'drbd') {
> + if ($type eq 'iscsi' || $type eq 'nfs' || $type eq 'cifs' || $type eq 
> 'rbd' || $type eq 'sheepdog' || $type eq 'iscsidirect' || $type eq 
> 'glusterfs' || $type eq 'zfs' || $type eq 'drbd') {

Can we please either split this so we have 1 storage per line?
if ($type eq 'iscsi' ||
$type eq 'nfs' ||
$type eq 'cifs' ||
(...)

Or have a global array/hash somewhere we grep/query at this point
instead?

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH storage 07/11] Add CIFS dependencies in package management.

2018-03-14 Thread Wolfgang Bumiller
On Tue, Mar 13, 2018 at 03:11:07PM +0100, Wolfgang Link wrote:
> ---
>  debian/control | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/debian/control b/debian/control
> index 04812bb..ffaefff 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -24,6 +24,8 @@ Depends: cstream,
>   smartmontools,
>   thin-provisioning-tools,
>   udev,
> +  cifs-utils,
> +  smbclient,

Wrong indentation (tabs in the middle of a spaces-only list)

>   ${perl:Depends},
>  Description: Proxmox VE storage management library
>   This package contains the storage management library used by Proxmox VE.
> -- 
> 2.11.0

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH storage 03/11] Add helper function for extract CIFS credentials.

2018-03-14 Thread Wolfgang Bumiller
On Tue, Mar 13, 2018 at 03:23:10PM +0100, Thomas Lamprecht wrote:
> On 03/13/2018 03:11 PM, Wolfgang Link wrote:
> > ---
> >  PVE/API2/Storage/Config.pm | 16 
> >  1 file changed, 16 insertions(+)
> > 
> > diff --git a/PVE/API2/Storage/Config.pm b/PVE/API2/Storage/Config.pm
> > index aa8c931..fa8d4de 100755
> > --- a/PVE/API2/Storage/Config.pm
> > +++ b/PVE/API2/Storage/Config.pm
> > @@ -12,6 +12,7 @@ use HTTP::Status qw(:constants);
> >  use Storable qw(dclone);
> >  use PVE::JSONSchema qw(get_standard_option);
> >  use PVE::RPCEnvironment;
> > +use Term::ReadLine;
> >  
> please do not use Term::ReadLine anymore, rather use Wolfgang B.'s
> nice PTY module, i.e.:

Also this should be CLI-only, and therefore handled in in CLI/pvesm.pm
via the new parameter mapping functionality instead of having this in
API2/Storage/Config.pm

In the API, just don't set a password if it hasn't been provided.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH qemu] build: use 3.0 source format

2018-03-14 Thread Wolfgang Bumiller
applied

On Tue, Mar 13, 2018 at 02:40:02PM +0100, Fabian Grünbichler wrote
> instead of manually including and calling quilt. resulting binary debs
> are identical.
> 
> Signed-off-by: Fabian Grünbichler 
> ---
> note: the rules file could probably benefit from further cleanup, but this was
> trivial and easily verifiable ;)
> 
>  debian/rules | 8 ++--
>  debian/source/format | 1 +
>  2 files changed, 3 insertions(+), 6 deletions(-)
>  create mode 100644 debian/source/format
> 
> diff --git a/debian/rules b/debian/rules
> index 4209404..02fcce8 100755
> --- a/debian/rules
> +++ b/debian/rules
> @@ -20,10 +20,6 @@ ARCH ?= $(shell dpkg-architecture -qDEB_HOST_GNU_CPU)
>  PACKAGE=pve-qemu-kvm
>  destdir := $(CURDIR)/debian/$(PACKAGE)
>  
> -ifneq "$(wildcard /usr/share/quilt/quilt.make)" ""
> -include /usr/share/quilt/quilt.make
> -endif
> -
>  CFLAGS = -Wall
>  
>  ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS)))
> @@ -48,7 +44,7 @@ config.status: configure
>   --enable-virtfs --disable-libnfs \
>   --disable-guest-agent --disable-guest-agent-msi
>  
> -build: patch build-stamp
> +build: build-stamp
>  
>  build-stamp:  config.status
>   dh_testdir
> @@ -60,7 +56,7 @@ build-stamp:  config.status
>  
>   touch $@
>  
> -clean: unpatch
> +clean:
>   dh_testdir
>   dh_testroot
>   rm -f build-stamp
> diff --git a/debian/source/format b/debian/source/format
> new file mode 100644
> index 000..163aaf8
> --- /dev/null
> +++ b/debian/source/format
> @@ -0,0 +1 @@
> +3.0 (quilt)
> -- 
> 2.14.2

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH container v5] Fix pct skiplock

2018-03-14 Thread Dietmar Maurer
applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH container v5] Fix pct skiplock

2018-03-14 Thread Alwin Antreich
The method vm_start sets an environment variable that is not picked up
anymore by systemd. This patch removes the environment variable and
introduces a skiplock file that is picked up by the
lxc-pve-prestart-hook.

Signed-off-by: Alwin Antreich 
---
note: made changes by Dietmar's comments

 src/PVE/LXC.pm| 13 +++--
 src/lxc-pve-prestart-hook |  5 -
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 7adbcd1..2a3950c 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1545,11 +1545,20 @@ sub vm_start {

 update_lxc_config($vmid, $conf);

-local $ENV{PVE_SKIPLOCK}=1 if $skiplock;
+my $skiplock_flag_fn = "/run/lxc/skiplock-$vmid";
+
+if ($skiplock) {
+   open(my $fh, '>', $skiplock_flag_fn) || die "failed to open 
$skiplock_flag_fn for writing: $!\n";
+   close($fh);
+}

 my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];

-PVE::Tools::run_command($cmd);
+eval { PVE::Tools::run_command($cmd); };
+if (my $err = $@) {
+   unlink $skiplock_flag_fn;
+   die $err if $err;
+}

 return;
 }
diff --git a/src/lxc-pve-prestart-hook b/src/lxc-pve-prestart-hook
index fd29423..61a8ef3 100755
--- a/src/lxc-pve-prestart-hook
+++ b/src/lxc-pve-prestart-hook
@@ -57,13 +57,16 @@ __PACKAGE__->register_method ({
return undef if $param->{name} !~ m/^\d+$/;

my $vmid = $param->{name};
+   my $skiplock_flag_fn = "/run/lxc/skiplock-$vmid";
+   my $skiplock = 1 if -e $skiplock_flag_fn;
+   unlink $skiplock_flag_fn if $skiplock;

PVE::Cluster::check_cfs_quorum(); # only start if we have quorum

return undef if ! -f PVE::LXC::Config->config_file($vmid);

my $conf = PVE::LXC::Config->load_config($vmid);
-   if (!$ENV{PVE_SKIPLOCK} && !PVE::LXC::Config->has_lock($conf, 
'mounted')) {
+   if (!$skiplock && !PVE::LXC::Config->has_lock($conf, 'mounted')) {
PVE::LXC::Config->check_lock($conf);
}

--
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH container v4] Fix pct skiplock

2018-03-14 Thread Dietmar Maurer
another comment inline
> diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
> index 7adbcd1..4398cfd 100644
> --- a/src/PVE/LXC.pm
> +++ b/src/PVE/LXC.pm
> @@ -1545,11 +1545,20 @@ sub vm_start {
> 
>  update_lxc_config($vmid, $conf);
> 
> -local $ENV{PVE_SKIPLOCK}=1 if $skiplock;
> +my $skiplock_flag_fn = "/run/lxc/skiplock-$vmid";
> +
> +if ($skiplock) {
> + open(my $fh, '>', $skiplock_flag_fn) || die "failed to open
> $skiplock_flag_fn for writing: $!\n";
> + close($fh);
> +}
> 
>  my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];
> 
> -PVE::Tools::run_command($cmd);
> +eval { PVE::Tools::run_command($cmd); };
> +if (my $err = $@) {
> + unlink $skiplock_flag_fn if -e $skiplock_flag_fn;

there is no need to test with -e, simply use

unlink $skiplock_flag_fn;


> + die $err if $err;
> +}
> 
>  return;

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH container v4] Fix pct skiplock

2018-03-14 Thread Dietmar Maurer
> diff --git a/src/lxc-pve-prestart-hook b/src/lxc-pve-prestart-hook
> index fd29423..79297da 100755
> --- a/src/lxc-pve-prestart-hook
> +++ b/src/lxc-pve-prestart-hook
> @@ -57,13 +57,16 @@ __PACKAGE__->register_method ({
>   return undef if $param->{name} !~ m/^\d+$/;
> 
>   my $vmid = $param->{name};
> + my $skiplock_flag_fn = "/run/lxc/skiplock-$vmid";
> + my $skiplock = 1 if -e $skiplock_flag_fn;
> + unlink $skiplock_flag_fn if -e $skiplock_flag_fn;

Why not:

> + unlink $skiplock_flag_fn if $skiplock;

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH container v4] Fix pct skiplock

2018-03-14 Thread Alwin Antreich
The method vm_start sets an environment variable that is not picked up
anymore by systemd. This patch removes the environment variable and
introduces a skiplock file that is picked up by the
lxc-pve-prestart-hook.

Signed-off-by: Alwin Antreich 
---
note: this version changes the path of the skiplock file to /run/lxc/.

 src/PVE/LXC.pm| 13 +++--
 src/lxc-pve-prestart-hook |  5 -
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 7adbcd1..4398cfd 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1545,11 +1545,20 @@ sub vm_start {

 update_lxc_config($vmid, $conf);

-local $ENV{PVE_SKIPLOCK}=1 if $skiplock;
+my $skiplock_flag_fn = "/run/lxc/skiplock-$vmid";
+
+if ($skiplock) {
+   open(my $fh, '>', $skiplock_flag_fn) || die "failed to open 
$skiplock_flag_fn for writing: $!\n";
+   close($fh);
+}

 my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];

-PVE::Tools::run_command($cmd);
+eval { PVE::Tools::run_command($cmd); };
+if (my $err = $@) {
+   unlink $skiplock_flag_fn if -e $skiplock_flag_fn;
+   die $err if $err;
+}

 return;
 }
diff --git a/src/lxc-pve-prestart-hook b/src/lxc-pve-prestart-hook
index fd29423..79297da 100755
--- a/src/lxc-pve-prestart-hook
+++ b/src/lxc-pve-prestart-hook
@@ -57,13 +57,16 @@ __PACKAGE__->register_method ({
return undef if $param->{name} !~ m/^\d+$/;

my $vmid = $param->{name};
+   my $skiplock_flag_fn = "/run/lxc/skiplock-$vmid";
+   my $skiplock = 1 if -e $skiplock_flag_fn;
+   unlink $skiplock_flag_fn if -e $skiplock_flag_fn;

PVE::Cluster::check_cfs_quorum(); # only start if we have quorum

return undef if ! -f PVE::LXC::Config->config_file($vmid);

my $conf = PVE::LXC::Config->load_config($vmid);
-   if (!$ENV{PVE_SKIPLOCK} && !PVE::LXC::Config->has_lock($conf, 
'mounted')) {
+   if (!$skiplock && !PVE::LXC::Config->has_lock($conf, 'mounted')) {
PVE::LXC::Config->check_lock($conf);
}

--
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [RFC edk2-firmware] Initial commit

2018-03-14 Thread Thomas Lamprecht
On 03/14/2018 12:32 PM, Fabian Grünbichler wrote:
> looks good all in all, some minor nits below
> 
> On Tue, Mar 13, 2018 at 04:29:24PM +0100, Thomas Lamprecht wrote:
>> Currently builds OVMF in a simmilar fashion the Debian package does,
>> we just focus on a x86_64 build for now, disable secure boot (was
>> disabled in our binary images from pve-qemu, it needs a static build
>> of an specific version of OpenSSL, bit of an headache) and has our
>> Logo for the boot splash.
>>
>> Signed-off-by: Thomas Lamprecht 
>> ---
>>  .gitignore   |   4 +++
>>  .gitmodules  |   3 +++
>>  Makefile |  47 
>>  debian/Logo.bmp  | Bin 0 -> 49078 bytes
>>  debian/changelog |   6 +
>>  debian/clean |   4 +++
>>  debian/compat|   1 +
>>  debian/control   |  21 +++
>>  debian/copyright |  29 
>>  debian/pve-edk2-firmware.install |   3 +++
>>  debian/pve-edk2-firmware.links   |   1 +
>>  debian/rules |  56 
>> +++
>>  edk2 |   1 +
>>  13 files changed, 176 insertions(+)
>>  create mode 100644 .gitignore
>>  create mode 100644 .gitmodules
>>  create mode 100644 Makefile
>>  create mode 100644 debian/Logo.bmp
>>  create mode 100644 debian/changelog
>>  create mode 100644 debian/clean
>>  create mode 100644 debian/compat
>>  create mode 100644 debian/control
>>  create mode 100644 debian/copyright
>>  create mode 100644 debian/pve-edk2-firmware.install
>>  create mode 100644 debian/pve-edk2-firmware.links
>>  create mode 100755 debian/rules
>>  create mode 16 edk2
>>
>> diff --git a/.gitignore b/.gitignore
>> new file mode 100644
>> index 000..a4c231b
>> --- /dev/null
>> +++ b/.gitignore
>> @@ -0,0 +1,4 @@
>> +*.deb
>> +*.buildinfo
>> +*.changes
>> +edk2.build
>> diff --git a/.gitmodules b/.gitmodules
>> new file mode 100644
>> index 000..e24b385
>> --- /dev/null
>> +++ b/.gitmodules
>> @@ -0,0 +1,3 @@
>> +[submodule "edk2"]
>> +path = edk2
>> +url = ../mirror_edk2
>> diff --git a/Makefile b/Makefile
>> new file mode 100644
>> index 000..831dcff
>> --- /dev/null
>> +++ b/Makefile
>> @@ -0,0 +1,47 @@
>> +PACKAGE=pve-edk2-firmware
>> +VER=1.$(shell date -d @$$(dpkg-parsechangelog -S timestamp) '+%Y%m%d')
> 
> why not 'dpkg-parsechangelog -S version'? ;)
> 

because it seems I did not read the whole man page and was
content with the result at the times ;)
fixing for v2

>> +PKGREL=1
>> +
>> +SRCDIR=edk2
>> +BUILDDIR=${SRCDIR}.build
>> +
>> +GITVERSION:=$(shell git rev-parse HEAD)
>> +
>> +DEB=${PACKAGE}_${VER}-${PKGREL}_all.deb
>> +
>> +all: ${DEB}
>> +@echo ${DEB}
>> +
>> +.PHONY: deb
>> +deb: ${DEB}
>> +${DEB}: | submodule
>> +rm -rf ${BUILDDIR}
>> +cp -rpa ${SRCDIR} ${BUILDDIR}
>> +cp -a debian ${BUILDDIR}
>> +echo "git clone git://git.proxmox.com/git/pve-guest-fw-edk2.git\\ngit 
>> checkout ${GITVERSION}" > ${BUILDDIR}/debian/SOURCE
> 
> name of repo and package are different - intentional or leftover from
> previous iteration?
> 

yes, there where many iterations and names

>> +cd ${BUILDDIR}; dpkg-buildpackage -rfakeroot -b -uc -us
> 
> '-rfakeroot' is redundant, it's the default if you are not root, and if
> you are root it will cause a warning to be printed.
> 

OK

>> +lintian ${DEB}
>> +@echo ${DEB}
>> +
>> +.PHONY: submodule
>> +submodule:
>> +test -f "${SRCDIR}/Readme.md" || git submodule update --init
>> +
>> +.PHONY: update_modules
>> +update_modules: submodule
>> +git submodule foreach 'git pull --ff-only origin master'
>> +
>> +.PHONY: upload
>> +upload: ${DEB}
>> +tar cf - ${DEB}|ssh -X repo...@repo.proxmox.com -- upload --product 
>> pmg,pve --dist stretch
> 
> do we need it in pmg? our qemu is only available in PVE
> 

nope, not needed, copy-is-my-hobby leftover

>> +
>> +.PHONY: distclean
>> +distclean: clean
>> +
>> +.PHONY: clean
>> +clean:
>> +rm -rf *~ debian/*~ *.deb ${BUILDDIR} *.changes *.dsc *.buildinfo
>> +
>> +.PHONY: dinstall
>> +dinstall: ${DEB}
>> +dpkg -i ${DEB}
>> diff --git a/debian/Logo.bmp b/debian/Logo.bmp
>> new file mode 100644
>> index 
>> ..4c46dd84772b32fa5c70e31d2483d66850495dc3
>> GIT binary patch
>> literal 49078
>> zcmeHP33wGnwmzxmKD=> zuqh!50%1NgKAJHN5@mF7VG#rYVH5;GHYX?$Ai2xyIn~{@^}SgJ;`jXCRPp6@b#3R=
>> z|JSL#xwCivMsVCMRREf!?+-};Q}CGtS2*F*HvrVE?*3sQjd1_q-v9_rQWXO9nG^)5
>> zk_1(;oFoiLGO9r}Okz~24ul2> zRJ$?+@b9$&Lf1jfl> zftpui88w>%q&A1EufZ}>Tf#MUt^rcJ1sGSi0;t^vu1%{0b<(bbq-$>msM8M8>eYq1
>> z^{xlF?iPIS0N2;A2lX1%2P3T$G-%WS;QBkDVdF;7s7Yf;s{bDV^?N{*W+> z89?Li&@8<> z=H{CqspSJeZWs*EY6#qXOIv7nE9%htA%Hf+;nw!I0ckS|$juJ}+%yJm>)0Nk-AI78
>> zk3z@WJ3-Pd6X5oY&H(Kv0^IgEWOPB@I{p@*!!$_h_#|}c-W5ov-$A##x&w5c4sgc|
>> z=#hCBK*lrZp9MX8WkTlNnE-b_2jtE> zy?ZSJ$jk=l{TjgCIWRD55I~> z^%;~0

Re: [pve-devel] [RFC edk2-firmware] Initial commit

2018-03-14 Thread Fabian Grünbichler
looks good all in all, some minor nits below

On Tue, Mar 13, 2018 at 04:29:24PM +0100, Thomas Lamprecht wrote:
> Currently builds OVMF in a simmilar fashion the Debian package does,
> we just focus on a x86_64 build for now, disable secure boot (was
> disabled in our binary images from pve-qemu, it needs a static build
> of an specific version of OpenSSL, bit of an headache) and has our
> Logo for the boot splash.
> 
> Signed-off-by: Thomas Lamprecht 
> ---
>  .gitignore   |   4 +++
>  .gitmodules  |   3 +++
>  Makefile |  47 
>  debian/Logo.bmp  | Bin 0 -> 49078 bytes
>  debian/changelog |   6 +
>  debian/clean |   4 +++
>  debian/compat|   1 +
>  debian/control   |  21 +++
>  debian/copyright |  29 
>  debian/pve-edk2-firmware.install |   3 +++
>  debian/pve-edk2-firmware.links   |   1 +
>  debian/rules |  56 
> +++
>  edk2 |   1 +
>  13 files changed, 176 insertions(+)
>  create mode 100644 .gitignore
>  create mode 100644 .gitmodules
>  create mode 100644 Makefile
>  create mode 100644 debian/Logo.bmp
>  create mode 100644 debian/changelog
>  create mode 100644 debian/clean
>  create mode 100644 debian/compat
>  create mode 100644 debian/control
>  create mode 100644 debian/copyright
>  create mode 100644 debian/pve-edk2-firmware.install
>  create mode 100644 debian/pve-edk2-firmware.links
>  create mode 100755 debian/rules
>  create mode 16 edk2
> 
> diff --git a/.gitignore b/.gitignore
> new file mode 100644
> index 000..a4c231b
> --- /dev/null
> +++ b/.gitignore
> @@ -0,0 +1,4 @@
> +*.deb
> +*.buildinfo
> +*.changes
> +edk2.build
> diff --git a/.gitmodules b/.gitmodules
> new file mode 100644
> index 000..e24b385
> --- /dev/null
> +++ b/.gitmodules
> @@ -0,0 +1,3 @@
> +[submodule "edk2"]
> + path = edk2
> + url = ../mirror_edk2
> diff --git a/Makefile b/Makefile
> new file mode 100644
> index 000..831dcff
> --- /dev/null
> +++ b/Makefile
> @@ -0,0 +1,47 @@
> +PACKAGE=pve-edk2-firmware
> +VER=1.$(shell date -d @$$(dpkg-parsechangelog -S timestamp) '+%Y%m%d')

why not 'dpkg-parsechangelog -S version'? ;)

> +PKGREL=1
> +
> +SRCDIR=edk2
> +BUILDDIR=${SRCDIR}.build
> +
> +GITVERSION:=$(shell git rev-parse HEAD)
> +
> +DEB=${PACKAGE}_${VER}-${PKGREL}_all.deb
> +
> +all: ${DEB}
> + @echo ${DEB}
> +
> +.PHONY: deb
> +deb: ${DEB}
> +${DEB}: | submodule
> + rm -rf ${BUILDDIR}
> + cp -rpa ${SRCDIR} ${BUILDDIR}
> + cp -a debian ${BUILDDIR}
> + echo "git clone git://git.proxmox.com/git/pve-guest-fw-edk2.git\\ngit 
> checkout ${GITVERSION}" > ${BUILDDIR}/debian/SOURCE

name of repo and package are different - intentional or leftover from
previous iteration?

> + cd ${BUILDDIR}; dpkg-buildpackage -rfakeroot -b -uc -us

'-rfakeroot' is redundant, it's the default if you are not root, and if
you are root it will cause a warning to be printed.

> + lintian ${DEB}
> + @echo ${DEB}
> +
> +.PHONY: submodule
> +submodule:
> + test -f "${SRCDIR}/Readme.md" || git submodule update --init
> +
> +.PHONY: update_modules
> +update_modules: submodule
> + git submodule foreach 'git pull --ff-only origin master'
> +
> +.PHONY: upload
> +upload: ${DEB}
> + tar cf - ${DEB}|ssh -X repo...@repo.proxmox.com -- upload --product 
> pmg,pve --dist stretch

do we need it in pmg? our qemu is only available in PVE

> +
> +.PHONY: distclean
> +distclean: clean
> +
> +.PHONY: clean
> +clean:
> + rm -rf *~ debian/*~ *.deb ${BUILDDIR} *.changes *.dsc *.buildinfo
> +
> +.PHONY: dinstall
> +dinstall: ${DEB}
> + dpkg -i ${DEB}
> diff --git a/debian/Logo.bmp b/debian/Logo.bmp
> new file mode 100644
> index 
> ..4c46dd84772b32fa5c70e31d2483d66850495dc3
> GIT binary patch
> literal 49078
> zcmeHP33wGnwmzxmKD= zuqh!50%1NgKAJHN5@mF7VG#rYVH5;GHYX?$Ai2xyIn~{@^}SgJ;`jXCRPp6@b#3R=
> z|JSL#xwCivMsVCMRREf!?+-};Q}CGtS2*F*HvrVE?*3sQjd1_q-v9_rQWXO9nG^)5
> zk_1(;oFoiLGO9r}Okz~24ul2 zRJ$?+@b9$&Lf1jfl zftpui88w>%q&A1EufZ}>Tf#MUt^rcJ1sGSi0;t^vu1%{0b<(bbq-$>msM8M8>eYq1
> z^{xlF?iPIS0N2;A2lX1%2P3T$G-%WS;QBkDVdF;7s7Yf;s{bDV^?N{*W+ z89?Li&@8< z=H{CqspSJeZWs*EY6#qXOIv7nE9%htA%Hf+;nw!I0ckS|$juJ}+%yJm>)0Nk-AI78
> zk3z@WJ3-Pd6X5oY&H(Kv0^IgEWOPB@I{p@*!!$_h_#|}c-W5ov-$A##x&w5c4sgc|
> z=#hCBK*lrZp9MX8WkTlNnE-b_2jtE zy?ZSJ$jk=l{TjgCIWRD55I~ z^%;~0WS|MdM?4Hw2E7ZAl@B9EJpypwdi=u%_`rKGYV2r$!3AJE@DYr8bS%Jwn*fIX
> z1^+&SM4w(4cNdP1Ez@*8)0T}re!07!j
> zc`CNzF@J-kvERVdC#He%Uk3ml{T2d`9t0S72%dQANr3T((SICZ!guhy8BgK+Nr1 z!i;C0hG%9z3(w7(3BR8`3ueuE9%j#-124Qd7v{b6BK+Z%m*D03ufY683t-`Ei(t{~
> zufgK%*WvZ0OJGS(He|oC6qdf318=^y43@8a3s&T=gjIjch1GBW5&pF19e8K$8d&pZ
> z6aH+y3;F9Uu-;n->o%;1^#$+2h7Ss$VB`Dn{)ZpH#=m|DAAS5+*!1bg@bPD#z-L=G
> z!{%*UV9WNcux-cZuzlwa*s

Re: [pve-devel] [PATCH container v2] Fix pct skiplock

2018-03-14 Thread Dietmar Maurer
Please send those white-space cleanups as separate patch

> diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
> index 7adbcd1..acb5cfd 100644
> --- a/src/PVE/LXC.pm
> +++ b/src/PVE/LXC.pm
> @@ -393,7 +393,7 @@ sub update_lxc_config {
> 
>  # some init scripts expect a linux terminal (turnkey).
>  $raw .= "lxc.environment = TERM=linux\n";
> -
> +

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server] use pve-edk2-firmware to load ovmf

2018-03-14 Thread Fabian Grünbichler
On Wed, Mar 14, 2018 at 11:14:57AM +0100, Thomas Lamprecht wrote:
> On 03/14/2018 10:56 AM, Fabian Grünbichler wrote:
> > small nit below, I'll take a look at the edk2 package itself later
> > 
> > On Tue, Mar 13, 2018 at 04:29:25PM +0100, Thomas Lamprecht wrote:
> >> We're able to just change it's path as we use the FD_SIZE_2MB option
> >> to build OVMF in the new pve-edk2-firmware package, which was the
> >> earlier implicit size of our current used OVMF BLOB.
> >>
> >> Incoming live migrations have their OVMF_CODE image in a RAMBlock[1],
> >> and only load the new image on a warm reboot, or, naturally, an full
> >> stop-start cycle.
> >>
> >> Signed-off-by: Thomas Lamprecht 
> >> ---
> >>
> >> If this would get applied already we need an version bump so that the
> >> pve-qemu 'Break: qemu-server (<= 5.0-23)' works.
> >>
> >> This can be seen as a RFC for now...
> >>
> >>  PVE/QemuServer.pm | 4 ++--
> >>  debian/control| 1 +
> >>  2 files changed, 3 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> >> index a26b2ab..2d27b74 100644
> >> --- a/PVE/QemuServer.pm
> >> +++ b/PVE/QemuServer.pm
> >> @@ -38,8 +38,8 @@ use Time::HiRes qw(gettimeofday);
> >>  use File::Copy qw(copy);
> >>  use URI::Escape;
> >>  
> >> -my $OVMF_CODE = '/usr/share/kvm/OVMF_CODE-pure-efi.fd';
> >> -my $OVMF_VARS = '/usr/share/kvm/OVMF_VARS-pure-efi.fd';
> >> +my $OVMF_CODE = '/usr/share/OVMF/OVMF_CODE.fd';
> >> +my $OVMF_VARS = '/usr/share/OVMF/OVMF_VARS.fd';
> > 
> > /usr/share/pve-edk2-firmware please - the package is not called "OVMF"
> > after all ;)
> > 
> 
> No, I conflict/provide OVMF and I used its exact paths,
> mirroring the Debian upstream packaging behaviour.
> 
> If you dislike providing OVMF and act like a complete separate
> package, then yes, the path would need to be changed. 

seems like I forgot about this ;)

the only non-qemu reverse-depends on "ovmf" is "mkosi" which only has a
recommends, and which (at least at first glance) does not use the ovmf
images directly at all.

I think we could just drop the provides and conflicts in this case and
move the files accordingly - if Debian ever gets other packages
depending on OVMF we don't break them accidentally (especially since the
version scheme is different, potentially breaking versioned
dependencies), and the one that matters to us is provided by us anyway,
so the paths are fully under our control there as well.

but this is not a hard requirement at all, I just noticed it in passing

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH container v3] Fix pct skiplock

2018-03-14 Thread Alwin Antreich
The method vm_start sets an environment variable that is not picked up
anymore by systemd. This patch removes the environment variable and
introduces a skiplock file that is picked up by the
lxc-pve-prestart-hook.

Signed-off-by: Alwin Antreich 
---
note: after discussion with Fabian, I removed the ENV variable. But I left the
path of the skiplock file because we use it on other places in the code too.
Though, I added a cleanup step if the container start fails.

 src/PVE/LXC.pm| 13 +++--
 src/lxc-pve-prestart-hook |  5 -
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 7adbcd1..acb5cfd 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1545,11 +1545,20 @@ sub vm_start {

 update_lxc_config($vmid, $conf);

-local $ENV{PVE_SKIPLOCK}=1 if $skiplock;
+my $skiplock_flag_fn = "/var/lib/lxc/$vmid/skiplock";
+
+if ($skiplock) {
+   open(my $fh, '>', $skiplock_flag_fn) || die "failed to open 
$skiplock_flag_fn for writing: $!\n";
+   close($fh);
+}

 my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];

-PVE::Tools::run_command($cmd);
+eval { PVE::Tools::run_command($cmd); };
+if (my $err = $@) {
+   unlink $skiplock_flag_fn if -e $skiplock_flag_fn;
+   die $err if $err;
+}

 return;
 }
diff --git a/src/lxc-pve-prestart-hook b/src/lxc-pve-prestart-hook
index fd29423..eba23ee 100755
--- a/src/lxc-pve-prestart-hook
+++ b/src/lxc-pve-prestart-hook
@@ -57,13 +57,16 @@ __PACKAGE__->register_method ({
return undef if $param->{name} !~ m/^\d+$/;

my $vmid = $param->{name};
+   my $skiplock_flag_fn = "/var/lib/lxc/$vmid/skiplock";
+   my $skiplock = 1 if -e $skiplock_flag_fn;
+   unlink $skiplock_flag_fn if -e $skiplock_flag_fn;

PVE::Cluster::check_cfs_quorum(); # only start if we have quorum

return undef if ! -f PVE::LXC::Config->config_file($vmid);

my $conf = PVE::LXC::Config->load_config($vmid);
-   if (!$ENV{PVE_SKIPLOCK} && !PVE::LXC::Config->has_lock($conf, 
'mounted')) {
+   if (!$skiplock && !PVE::LXC::Config->has_lock($conf, 'mounted')) {
PVE::LXC::Config->check_lock($conf);
}

--
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH container v2] Fix pct skiplock

2018-03-14 Thread Alwin Antreich
The method vm_start sets an environment variable that is not picked up
anymore by systemd. This patch removes the environment variable and
introduces a skiplock file that is picked up by the
lxc-pve-prestart-hook.

Signed-off-by: Alwin Antreich 
---
note: after discussion with Fabian, I removed the ENV variable. But I left the
path of the skiplock file because we use it on other places in the code too.
Though, I added a cleanup step if the container start fails.

 src/PVE/LXC.pm| 27 ++-
 src/lxc-pve-prestart-hook |  5 -
 2 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 7adbcd1..acb5cfd 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -393,7 +393,7 @@ sub update_lxc_config {

 # some init scripts expect a linux terminal (turnkey).
 $raw .= "lxc.environment = TERM=linux\n";
-
+
 my $utsname = $conf->{hostname} || "CT$vmid";
 $raw .= "lxc.uts.name = $utsname\n";

@@ -454,7 +454,7 @@ sub update_lxc_config {
}
$raw .= "lxc.cgroup.cpuset.cpus = ".$cpuset->short_string()."\n";
 }
-
+
 File::Path::mkpath("$dir/rootfs");

 PVE::Tools::file_set_contents("$dir/config", $raw);
@@ -566,7 +566,7 @@ sub destroy_lxc_container {

 sub vm_stop_cleanup {
 my ($storage_cfg, $vmid, $conf, $keepActive) = @_;
-
+
 eval {
if (!$keepActive) {

@@ -1172,19 +1172,19 @@ sub mountpoint_mount {
 my $type = $mountpoint->{type};
 my $quota = !$snapname && !$mountpoint->{ro} && $mountpoint->{quota};
 my $mounted_dev;
-
+
 return if !$volid || !$mount;

 $mount =~ s!/+!/!g;

 my $mount_path;
 my ($mpfd, $parentfd, $last_dir);
-
+
 if (defined($rootdir)) {
($rootdir, $mount_path, $mpfd, $parentfd, $last_dir) =
__mount_prepare_rootdir($rootdir, $mount);
 }
-
+
 my ($storage, $volname) = PVE::Storage::parse_volume_id($volid, 1);

 die "unknown snapshot path for '$volid'" if !$storage && 
defined($snapname);
@@ -1288,7 +1288,7 @@ sub mountpoint_mount {
warn "cannot enable quota control for bind mounts\n" if $quota;
return wantarray ? ($volid, 0, undef) : $volid;
 }
-
+
 die "unsupported storage";
 }

@@ -1545,11 +1545,20 @@ sub vm_start {

 update_lxc_config($vmid, $conf);

-local $ENV{PVE_SKIPLOCK}=1 if $skiplock;
+my $skiplock_flag_fn = "/var/lib/lxc/$vmid/skiplock";
+
+if ($skiplock) {
+   open(my $fh, '>', $skiplock_flag_fn) || die "failed to open 
$skiplock_flag_fn for writing: $!\n";
+   close($fh);
+}

 my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];

-PVE::Tools::run_command($cmd);
+eval { PVE::Tools::run_command($cmd); };
+if (my $err = $@) {
+   unlink $skiplock_flag_fn if -e $skiplock_flag_fn;
+   die $err if $err;
+}

 return;
 }
diff --git a/src/lxc-pve-prestart-hook b/src/lxc-pve-prestart-hook
index fd29423..eba23ee 100755
--- a/src/lxc-pve-prestart-hook
+++ b/src/lxc-pve-prestart-hook
@@ -57,13 +57,16 @@ __PACKAGE__->register_method ({
return undef if $param->{name} !~ m/^\d+$/;

my $vmid = $param->{name};
+   my $skiplock_flag_fn = "/var/lib/lxc/$vmid/skiplock";
+   my $skiplock = 1 if -e $skiplock_flag_fn;
+   unlink $skiplock_flag_fn if -e $skiplock_flag_fn;

PVE::Cluster::check_cfs_quorum(); # only start if we have quorum

return undef if ! -f PVE::LXC::Config->config_file($vmid);

my $conf = PVE::LXC::Config->load_config($vmid);
-   if (!$ENV{PVE_SKIPLOCK} && !PVE::LXC::Config->has_lock($conf, 
'mounted')) {
+   if (!$skiplock && !PVE::LXC::Config->has_lock($conf, 'mounted')) {
PVE::LXC::Config->check_lock($conf);
}

--
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH storage 01/11] Remove pool with -f parameter.

2018-03-14 Thread Thomas Lamprecht
applied

would maybe also make sense to trap SIG{INT,TERM,QUIT} and
call the cleanup function there too, just as a general side note.

On 03/13/2018 03:11 PM, Wolfgang Link wrote:
> The test pool will not removed if we do not force it.
> ---
>  test/run_test_zfspoolplugin.pl | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/test/run_test_zfspoolplugin.pl b/test/run_test_zfspoolplugin.pl
> index bda8348..f6218b1 100755
> --- a/test/run_test_zfspoolplugin.pl
> +++ b/test/run_test_zfspoolplugin.pl
> @@ -2643,7 +2643,7 @@ sub setup_zpool {
>  sub clean_up_zpool {
>  
>  eval {
> - run_command("zpool destroy $subvol");
> + run_command("zpool destroy -f $subvol");
>  };
>  if ($@) {
>   warn $@;}
> 


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH qemu-server] remove legacy vm_monitor_command

2018-03-14 Thread Wolfgang Bumiller
applied

On Tue, Mar 13, 2018 at 08:21:59AM +0100, Thomas Lamprecht wrote:
> We introduced our QMP socket with commit
> c971c4f2213524f27125f558978a428b53628f34 (29.05.2012)
> 
> Already tried to remove this with commit
> 7b7c6d1b5dcee25e1194d4b8a0219bd5c31a5639 (13.07.2012)
> 
> But reverted that to allow migration of VMs still using the old
> montior to ones which already switched over to the new QMP one,
> in commit dab36e1ee924be0efab3f85937c23910b456f4b9 (17.08.2012)
> see bug #242 for reference
> 
> This was all done  and released in PVE 2.2, as no migration through
> nodes differing more than one major version is possible we can
> finally remove this code for good.
> 
> Signed-off-by: Thomas Lamprecht 
> ---
>  PVE/QemuServer.pm | 79 
> ---
>  1 file changed, 79 deletions(-)
> 
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index a26b2ab..5462d07 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -4169,81 +4169,6 @@ sub __read_avail {
>  return $res;
>  }
>  
> -# old code, only used to shutdown old VM after update
> -sub vm_monitor_command {
> -my ($vmid, $cmdstr, $nocheck) = @_;
> -
> -my $res;
> -
> -eval {
> - die "VM $vmid not running\n" if !check_running($vmid, $nocheck);
> -
> - my $sname = "${var_run_tmpdir}/$vmid.mon";
> -
> - my $sock = IO::Socket::UNIX->new( Peer => $sname ) ||
> - die "unable to connect to VM $vmid socket - $!\n";
> -
> - my $timeout = 3;
> -
> - # hack: migrate sometime blocks the monitor (when migrate_downtime
> - # is set)
> - if ($cmdstr =~ m/^(info\s+migrate|migrate\s)/) {
> - $timeout = 60*60; # 1 hour
> - }
> -
> - # read banner;
> - my $data = __read_avail($sock, $timeout);
> -
> - if ($data !~ m/^QEMU\s+(\S+)\s+monitor\s/) {
> - die "got unexpected qemu monitor banner\n";
> - }
> -
> - my $sel = new IO::Select;
> - $sel->add($sock);
> -
> - if (!scalar(my @ready = $sel->can_write($timeout))) {
> - die "monitor write error - timeout";
> - }
> -
> - my $fullcmd = "$cmdstr\r";
> -
> - # syslog('info', "VM $vmid monitor command: $cmdstr");
> -
> - my $b;
> - if (!($b = $sock->syswrite($fullcmd)) || ($b != length($fullcmd))) {
> - die "monitor write error - $!";
> - }
> -
> - return if ($cmdstr eq 'q') || ($cmdstr eq 'quit');
> -
> - $timeout = 20;
> -
> - if ($cmdstr =~ m/^(info\s+migrate|migrate\s)/) {
> - $timeout = 60*60; # 1 hour
> - } elsif ($cmdstr =~ m/^(eject|change)/) {
> - $timeout = 60; # note: cdrom mount command is slow
> - }
> - if ($res = __read_avail($sock, $timeout)) {
> -
> - my @lines = split("\r?\n", $res);
> -
> - shift @lines if $lines[0] !~ m/^unknown command/; # skip echo
> -
> - $res = join("\n", @lines);
> - $res .= "\n";
> - }
> -};
> -
> -my $err = $@;
> -
> -if ($err) {
> - syslog("err", "VM $vmid monitor command failed - $err");
> - die $err;
> -}
> -
> -return $res;
> -}
> -
>  sub qemu_block_resize {
>  my ($vmid, $deviceid, $storecfg, $volid, $size) = @_;
>  
> @@ -5049,10 +4974,6 @@ sub vm_qmp_command {
>   my $qmpclient = PVE::QMPClient->new();
>  
>   $res = $qmpclient->cmd($vmid, $cmd, $timeout);
> - } elsif (-e "${var_run_tmpdir}/$vmid.mon") {
> - die "can't execute complex command on old monitor - stop/start your 
> vm to fix the problem\n"
> - if scalar(%{$cmd->{arguments}});
> - vm_monitor_command($vmid, $cmd->{execute}, $nocheck);
>   } else {
>   die "unable to open monitor socket\n";
>   }
> -- 
> 2.14.2

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH qemu-server 1/3] use default hostname when none is set

2018-03-14 Thread Wolfgang Bumiller
applied series

On Mon, Mar 12, 2018 at 04:25:04PM +0100, Dominik Csapak wrote:
> use "VM$vmid" like we do in a container
> 
> Signed-off-by: Dominik Csapak 
> ---
>  PVE/QemuServer/Cloudinit.pm | 12 ++--
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/PVE/QemuServer/Cloudinit.pm b/PVE/QemuServer/Cloudinit.pm
> index dc55d2d..31f8462 100644
> --- a/PVE/QemuServer/Cloudinit.pm
> +++ b/PVE/QemuServer/Cloudinit.pm
> @@ -63,8 +63,8 @@ sub get_cloudinit_format {
>  }
>  
>  sub get_hostname_fqdn {
> -my ($conf) = @_;
> -my $hostname = $conf->{name};
> +my ($conf, $vmid) = @_;
> +my $hostname = $conf->{name} // "VM$vmid";
>  my $fqdn;
>  if ($hostname =~ /\./) {
>   $fqdn = $hostname;
> @@ -96,9 +96,9 @@ sub get_dns_conf {
>  }
>  
>  sub cloudinit_userdata {
> -my ($conf) = @_;
> +my ($conf, $vmid) = @_;
>  
> -my ($hostname, $fqdn) = get_hostname_fqdn($conf);
> +my ($hostname, $fqdn) = get_hostname_fqdn($conf, $vmid);
>  
>  my $content = "#cloud-config\n";
>  
> @@ -198,7 +198,7 @@ EOF
>  sub generate_configdrive2 {
>  my ($conf, $vmid, $drive, $volname, $storeid) = @_;
>  
> -my $user_data = cloudinit_userdata($conf);
> +my $user_data = cloudinit_userdata($conf, $vmid);
>  my $network_data = configdrive2_network($conf);
>  
>  my $digest_data = $user_data . $network_data;
> @@ -363,7 +363,7 @@ sub nocloud_metadata {
>  sub generate_nocloud {
>  my ($conf, $vmid, $drive, $volname, $storeid) = @_;
>  
> -my $user_data = cloudinit_userdata($conf);
> +my $user_data = cloudinit_userdata($conf, $vmid);
>  my $network_data = nocloud_network($conf);
>  
>  my $digest_data = $user_data . $network_data;
> -- 
> 2.11.0

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server] use pve-edk2-firmware to load ovmf

2018-03-14 Thread Thomas Lamprecht
On 03/14/2018 10:56 AM, Fabian Grünbichler wrote:
> small nit below, I'll take a look at the edk2 package itself later
> 
> On Tue, Mar 13, 2018 at 04:29:25PM +0100, Thomas Lamprecht wrote:
>> We're able to just change it's path as we use the FD_SIZE_2MB option
>> to build OVMF in the new pve-edk2-firmware package, which was the
>> earlier implicit size of our current used OVMF BLOB.
>>
>> Incoming live migrations have their OVMF_CODE image in a RAMBlock[1],
>> and only load the new image on a warm reboot, or, naturally, an full
>> stop-start cycle.
>>
>> Signed-off-by: Thomas Lamprecht 
>> ---
>>
>> If this would get applied already we need an version bump so that the
>> pve-qemu 'Break: qemu-server (<= 5.0-23)' works.
>>
>> This can be seen as a RFC for now...
>>
>>  PVE/QemuServer.pm | 4 ++--
>>  debian/control| 1 +
>>  2 files changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
>> index a26b2ab..2d27b74 100644
>> --- a/PVE/QemuServer.pm
>> +++ b/PVE/QemuServer.pm
>> @@ -38,8 +38,8 @@ use Time::HiRes qw(gettimeofday);
>>  use File::Copy qw(copy);
>>  use URI::Escape;
>>  
>> -my $OVMF_CODE = '/usr/share/kvm/OVMF_CODE-pure-efi.fd';
>> -my $OVMF_VARS = '/usr/share/kvm/OVMF_VARS-pure-efi.fd';
>> +my $OVMF_CODE = '/usr/share/OVMF/OVMF_CODE.fd';
>> +my $OVMF_VARS = '/usr/share/OVMF/OVMF_VARS.fd';
> 
> /usr/share/pve-edk2-firmware please - the package is not called "OVMF"
> after all ;)
> 

No, I conflict/provide OVMF and I used its exact paths,
mirroring the Debian upstream packaging behaviour.

If you dislike providing OVMF and act like a complete separate
package, then yes, the path would need to be changed. 


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server] use pve-edk2-firmware to load ovmf

2018-03-14 Thread Fabian Grünbichler
small nit below, I'll take a look at the edk2 package itself later

On Tue, Mar 13, 2018 at 04:29:25PM +0100, Thomas Lamprecht wrote:
> We're able to just change it's path as we use the FD_SIZE_2MB option
> to build OVMF in the new pve-edk2-firmware package, which was the
> earlier implicit size of our current used OVMF BLOB.
> 
> Incoming live migrations have their OVMF_CODE image in a RAMBlock[1],
> and only load the new image on a warm reboot, or, naturally, an full
> stop-start cycle.
> 
> Signed-off-by: Thomas Lamprecht 
> ---
> 
> If this would get applied already we need an version bump so that the
> pve-qemu 'Break: qemu-server (<= 5.0-23)' works.
> 
> This can be seen as a RFC for now...
> 
>  PVE/QemuServer.pm | 4 ++--
>  debian/control| 1 +
>  2 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index a26b2ab..2d27b74 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -38,8 +38,8 @@ use Time::HiRes qw(gettimeofday);
>  use File::Copy qw(copy);
>  use URI::Escape;
>  
> -my $OVMF_CODE = '/usr/share/kvm/OVMF_CODE-pure-efi.fd';
> -my $OVMF_VARS = '/usr/share/kvm/OVMF_VARS-pure-efi.fd';
> +my $OVMF_CODE = '/usr/share/OVMF/OVMF_CODE.fd';
> +my $OVMF_VARS = '/usr/share/OVMF/OVMF_VARS.fd';

/usr/share/pve-edk2-firmware please - the package is not called "OVMF"
after all ;)

>  
>  my $qemu_snap_storage = {rbd => 1, sheepdog => 1};
>  
> diff --git a/debian/control b/debian/control
> index b8aa4ed..cba1016 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -32,6 +32,7 @@ Depends: dbus,
>   pve-ha-manager,
>   pve-qemu-kvm (>= 2.9.1-9),
>   socat,
> + pve-edk2-firmware,
>   ${perl:Depends},
>   ${shlibs:Depends},
>  Description: Qemu Server Tools
> -- 
> 2.14.2
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH] increase zfs default timeout to 30sec

2018-03-14 Thread Fabian Grünbichler
On Tue, Mar 13, 2018 at 04:28:17PM +0200, Lauri Tirkkonen wrote:
> On Tue, Mar 13 2018 16:12:27 +0200, Lauri Tirkkonen wrote:
> > On Tue, Mar 13 2018 16:06:12 +0200, Lauri Tirkkonen wrote:
> > > Hi,
> > > 
> > > On Tue, Mar 13 2018 09:45:18 +0100, Fabian Grünbichler wrote:
> > > > if you haven't already, please send a signed CLA[1] to
> > > 
> > > looks like you forgot to define [1] :)

I did indeed, sorry :) it was supposed to point to

https://pve.proxmox.com/wiki/Developer_Documentation

> > 
> > Well, I found the CLAs anyway, but I really don't think it's worth
> > bothering our legal dept
> 
> Well, they seemed to be aching for things to do so I did that anyway -
> so maybe we'll get that thing signed. Still, I find it a bit ridiculous
> to do all this for a oneliner (:

while just changing one number to another is probably trivial enough to
not be copyrightable and thus not require a signed CLA, we'd rather not
have to think about where to draw the line and start requiring one (and
of course also hope that you might want to contribute other fixes and/or
features in the future :)). it's easier and simpler to just require it
in general, and it is thankfully only a one-time thing.

with that out of the way, just to summarize the situation:

bumping the timeout a bit (not to 30, but something in the 10-20 range)
might already solve your concrete issue, and is something that could get
merged (we'd have to look at all the code paths that call this piece of
code from the API without forking a worker, because these "run thing
with timeout" tend to accumulate in unfortunate ways, and like Thomas
said we have a 30s limit for the whole API request).

the real fix is to expose more storage operations (and things that imply
storage operations) over the API as workers, so that we can safely bump
the timeouts a lot. this would then require changes on the GUI side as
well to make use of the "workerized" API paths. the biggest hurdle there
is most likely the storage content listing, because that is called often
by the GUI, and can take quite a while on a busy big storage, so we'd
need some kind of client side caching and invalidation/force reload
mechanism.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel