Re: [pve-devel] VDI solution...

[Alexandre DERUMIER]

Hi,

my 2 cents:


For me, vdi are more spice solution.

Something like you have a customer portal, user click on create a new vdi,

It's spawing a new vm (linked clone from a template).

Spice is launching (maybe better with a spice html5 implementation)

and when user close the session, the vm is destroyed.

I don't think that spice is able currently to only display 1 app in 1vm (I 
think vmware is able to do it).



Not sure this kind of feature should be directly integrated in proxmox, 
but I could be build easily in a external webapp calling proxmox apis.




- Mail original -
De: "dietmar" 
À: "Gilberto Nunes" , "pve-devel" 

Envoyé: Vendredi 20 Avril 2018 06:41:32
Objet: Re: [pve-devel] VDI solution...

> You guys could integrate Apache Guacamole to Proxmox VE, perhaps. 

You can install an run that inside any VM. So I don't really understand what 
we should integrate? 

___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] VDI solution...

[Dietmar Maurer]

> You guys could integrate Apache Guacamole to Proxmox VE, perhaps.

You can install an run that inside any VM. So I don't really understand what
we should integrate?

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] Add a shortcut into Web Interface

[Gilberto Nunes]

Hi there


How can I add a shortcut to an external link, into web interface?
Someone can point me the direction to do that??

Thanks a lot
---
Gilberto Nunes Ferreira

(47) 3025-5907
(47) 99676-7530 - Whatsapp / Telegram

Skype: gilberto.nunes36
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] VDI solution...

[Gilberto Nunes]

You guys could integrate Apache Guacamole to Proxmox VE, perhaps.

---
Gilberto Nunes Ferreira

(47) 3025-5907
(47) 99676-7530 - Whatsapp / Telegram

Skype: gilberto.nunes36




2018-04-19 9:33 GMT-03:00 Gilberto Nunes :

> Hi there...
>
> Is there any plan to add some VDI solution, like Ulteo OVD, VMWare
> Horizons, something like that.
> I thing this would bring a big differential to Proxmox.
>
> Thanks
> ---
> Gilberto Nunes Ferreira
>
> (47) 3025-5907
> (47) 99676-7530 - Whatsapp / Telegram
>
> Skype: gilberto.nunes36
>
>
>
>
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH manager] fix #1732: actually post values in IScsiEdit

[Fabian Grünbichler]

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v2 common 2/4] acme: add ACME library

[Fabian Grünbichler]

this implements those parts of draft-ietf-acme-acme-09 which are needed
to use Let's Encrypt's v2 API.

(based on an internal implement for the Let's Encrypt v1 API)
Co-Authored-By: Wolfgang Bumiller 
Signed-off-by: Fabian Grünbichler 
---
 src/PVE/ACME.pm | 531 
 1 file changed, 531 insertions(+)
 create mode 100644 src/PVE/ACME.pm

diff --git a/src/PVE/ACME.pm b/src/PVE/ACME.pm
new file mode 100644
index 000..0c7aa23
--- /dev/null
+++ b/src/PVE/ACME.pm
@@ -0,0 +1,531 @@
+package PVE::ACME;
+
+use strict;
+use warnings;
+
+use POSIX;
+
+use Data::Dumper;
+use Date::Parse;
+use MIME::Base64 qw(encode_base64url);
+use File::Path qw(make_path);
+use JSON;
+use Digest::SHA qw(sha256 sha256_hex);
+
+use HTTP::Request;
+use LWP::UserAgent;
+
+use Crypt::OpenSSL::RSA;
+
+use PVE::Certificate;
+use PVE::Tools qw(
+file_set_contents
+file_get_contents
+);
+
+Crypt::OpenSSL::RSA->import_random_seed();
+
+my $LETSENCRYPT_STAGING = 
'https://acme-staging-v02.api.letsencrypt.org/directory';
+
+### ACME library (compatible with Let's Encrypt v2 API)
+#
+# sample usage:
+#
+# 1) my $acme = PVE::ACME->new('path/to/account.json', 'API directory URL');
+# 2) $acme->init(4096); # generate account key
+# 4) my $tos_url = $acme->get_meta()->{termsOfService}; # optional, display if 
applicable
+# 5) $acme->new_account($tos_url, contact => ['mailto:exam...@example.com']);
+#
+# 1) my $acme = PVE::ACME->new('path/to/account.json', 'API directory URL');
+# 2) $acme->load();
+# 3) my ($order_url, $order) = $acme->new_order(['foo.example.com', 
'bar.example.com']);
+# 4) # repeat a-f for each $auth_url in $order->{authorizations}
+# a) my $authorization = $acme->get_authorization($auth_url);
+# b) # pick $challenge from $authorization->{challenges} according to desired 
type
+# c) my $key_auth = $acme->key_authorization($challenge->{token});
+# d) # setup challenge validation according to specification
+# e) $acme->request_challenge_validation($challenge->{url}, $key_auth);
+# f) # poll $acme->get_authorization($auth_url) until status is 'valid'
+# 5) # generate CSR in PEM format
+# 6) $acme->finalize_order($order, $csr);
+# 7) # poll $acme->get_order($order_url) until status is 'valid'
+# 8) my $cert = $acme->get_certificate($order);
+# 9) # $key is path to key file, $cert contains PEM-encoded certificate chain
+#
+# 1) my $acme = PVE::ACME->new('path/to/account.json', 'API directory URL');
+# 2) $acme->load();
+# 3) $acme->revoke_certificate($cert);
+
+# Tools
+sub encode($) { # acme requires 'base64url' encoding
+return encode_base64url($_[0]);
+}
+
+sub tojs($;%) { # shortcut for to_json with utf8=>1
+my ($data, %data) = @_;
+return to_json($data, { utf8 => 1, %data });
+}
+
+sub fromjs($) {
+return from_json($_[0]);
+}
+
+sub fatal($$;$$) {
+my ($self, $msg, $dump, $noerr) = @_;
+
+warn Dumper($dump), "\n" if $self->{debug} && $dump;
+if ($noerr) {
+   warn "$msg\n";
+} else {
+   die "$msg\n";
+}
+}
+
+# Implementation
+
+# $path: account JSON file
+# $directory: the ACME directory URL used to find method URLs
+sub new($$$) {
+my ($class, $path, $directory) = @_;
+
+$directory //= $LETSENCRYPT_STAGING;
+
+my $ua = LWP::UserAgent->new();
+$ua->env_proxy();
+$ua->agent('pve-acme/0.1');
+$ua->protocols_allowed(['https']);
+
+my $self = {
+   ua => $ua,
+   path => $path,
+   directory => $directory,
+   nonce => undef,
+   key => undef,
+   location => undef,
+   account => undef,
+   tos => undef,
+};
+
+return bless $self, $class;
+}
+
+# RS256: PKCS#1 padding, no OAEP, SHA256
+my $configure_key = sub {
+my ($key) = @_;
+$key->use_pkcs1_padding();
+$key->use_sha256_hash();
+};
+
+# Create account key with $keybits bits
+# use instead of load, overwrites existing account JSON file!
+sub init {
+my ($self, $keybits) = @_;
+die "Already have a key\n" if defined($self->{key});
+$keybits //= 4096;
+my $key = Crypt::OpenSSL::RSA->generate_key($keybits);
+$configure_key->($key);
+$self->{key} = $key;
+$self->save();
+}
+
+my @SAVED_VALUES = qw(location account tos debug directory);
+# Serialize persistent parts of $self to $self->{path} as JSON
+sub save {
+my ($self) = @_;
+my $o = {};
+my $keystr;
+if (my $key = $self->{key}) {
+   $keystr = $key->get_private_key_string();
+   $o->{key} = $keystr;
+}
+for my $k (@SAVED_VALUES) {
+   my $v = $self->{$k} // next;
+   $o->{$k} = $v;
+}
+# pretty => 1 for readability
+# canonical => 1 to reduce churn
+file_set_contents($self->{path}, tojs($o, pretty => 1, canonical => 1));
+}
+
+# Load serialized account JSON file into $self
+sub load {
+my ($self) = @_;
+return if $self->{loaded};
+$self->{loaded} = 1;
+my $data = 

[pve-devel] [PATCH v2 manager 3/5] add ACME account API endpoints

[Fabian Grünbichler]

for registering, updating, refreshing and deactiving a PVE-managed ACME
account, as well as for retrieving the (optional, but required if
available) terms of service of the ACME API provider / CA.

Signed-off-by: Fabian Grünbichler 
---
 PVE/API2/Makefile   |   1 +
 PVE/API2/ACMEAccount.pm | 347 
 PVE/API2/Cluster.pm |   7 +
 3 files changed, 355 insertions(+)
 create mode 100644 PVE/API2/ACMEAccount.pm

diff --git a/PVE/API2/Makefile b/PVE/API2/Makefile
index 51b8b30a..d72ddd9b 100644
--- a/PVE/API2/Makefile
+++ b/PVE/API2/Makefile
@@ -14,6 +14,7 @@ PERLSOURCE =  \
Pool.pm \
Tasks.pm\
Network.pm  \
+   ACMEAccount.pm  \
NodeConfig.pm   \
Services.pm
 
diff --git a/PVE/API2/ACMEAccount.pm b/PVE/API2/ACMEAccount.pm
new file mode 100644
index ..fe7619d8
--- /dev/null
+++ b/PVE/API2/ACMEAccount.pm
@@ -0,0 +1,347 @@
+package PVE::API2::ACMEAccount;
+
+use strict;
+use warnings;
+
+use PVE::ACME;
+use PVE::CertHelpers;
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::RPCEnvironment;
+use PVE::Tools qw(extract_param);
+
+use base qw(PVE::RESTHandler);
+
+my $acme_directories = [
+{
+   name => 'Let\'s Encrypt V2',
+   url => 'https://acme-v02.api.letsencrypt.org/directory',
+},
+{
+   name => 'Let\'s Encrypt V2 Staging',
+   url => 'https://acme-staging-v02.api.letsencrypt.org/directory',
+},
+];
+
+my $acme_default_directory_url = $acme_directories->[0]->{url};
+
+my $account_contact_from_param = sub {
+my ($param) = @_;
+return [ map { "mailto:$_; } PVE::Tools::split_list(extract_param($param, 
'contact')) ];
+};
+
+my $acme_account_dir = PVE::CertHelpers::acme_account_dir();
+
+__PACKAGE__->register_method ({
+name => 'index',
+path => '',
+method => 'GET',
+permissions => { user => 'all' },
+description => "ACMEAccount index.",
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   },
+},
+returns => {
+   type => 'array',
+   items => {
+   type => "object",
+   properties => {},
+   },
+   links => [ { rel => 'child', href => "{name}" } ],
+},
+code => sub {
+   my ($param) = @_;
+
+   return [
+   { name => 'account' },
+   { name => 'tos' },
+   { name => 'directories' },
+   ];
+}});
+
+__PACKAGE__->register_method ({
+name => 'account_index',
+path => 'account',
+method => 'GET',
+permissions => { user => 'all' },
+description => "ACMEAccount index.",
+protected => 1,
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   },
+},
+returns => {
+   type => 'array',
+   items => {
+   type => "object",
+   properties => {},
+   },
+   links => [ { rel => 'child', href => "{name}" } ],
+},
+code => sub {
+   my ($param) = @_;
+
+   my $accounts = PVE::CertHelpers::list_acme_accounts();
+   return [ map { { name => $_ }  } @$accounts ];
+}});
+
+__PACKAGE__->register_method ({
+name => 'register_account',
+path => 'account',
+method => 'POST',
+description => "Register a new ACME account with CA.",
+protected => 1,
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   name => get_standard_option('pve-acme-account-name'),
+   contact => get_standard_option('pve-acme-account-contact'),
+   tos_url => {
+   type => 'string',
+   description => 'URL of CA TermsOfService - setting this 
indicates agreement.',
+   optional => 1,
+   },
+   directory => get_standard_option('pve-acme-directory-url', {
+   default => $acme_default_directory_url,
+   optional => 1,
+   }),
+   },
+},
+returns => {
+   type => 'string',
+},
+code => sub {
+   my ($param) = @_;
+
+   my $account_name = extract_param($param, 'name') // 'default';
+   my $account_file = "${acme_account_dir}/${account_name}";
+
+   mkdir $acme_account_dir;
+
+   raise_param_exc({'name' => "ACME account config file '${account_name}' 
already exists."})
+   if -e $account_file;
+
+   my $directory = extract_param($param, 'directory') // 
$acme_default_directory_url;
+   my $contact = $account_contact_from_param->($param);
+
+   my $rpcenv = PVE::RPCEnvironment::get();
+
+   my $authuser = $rpcenv->get_user();
+
+   my $realcmd = sub {
+   PVE::Cluster::cfs_lock_acme($account_name, 10, sub {
+   die "ACME account config file '${account_name}' already 
exists.\n"
+   if -e $account_file;
+
+   my $acme = PVE::ACME->new($account_file, 

[pve-devel] VDI solution...

[Gilberto Nunes]

Hi there...

Is there any plan to add some VDI solution, like Ulteo OVD, VMWare
Horizons, something like that.
I thing this would bring a big differential to Proxmox.

Thanks
---
Gilberto Nunes Ferreira

(47) 3025-5907
(47) 99676-7530 - Whatsapp / Telegram

Skype: gilberto.nunes36
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v2 manager 5/5] add certificates API endpoints

[Fabian Grünbichler]

to allow retrieval of certificate information, and uploading or removing
of custom certificate files.

Signed-off-by: Fabian Grünbichler 
---
 PVE/API2/Makefile|   1 +
 PVE/API2/Certificates.pm | 205 +++
 PVE/API2/Nodes.pm|   8 ++
 3 files changed, 214 insertions(+)
 create mode 100644 PVE/API2/Certificates.pm

diff --git a/PVE/API2/Makefile b/PVE/API2/Makefile
index 44b9cf7c..9862e498 100644
--- a/PVE/API2/Makefile
+++ b/PVE/API2/Makefile
@@ -14,6 +14,7 @@ PERLSOURCE =  \
Pool.pm \
Tasks.pm\
Network.pm  \
+   Certificates.pm \
ACME.pm \
ACMEAccount.pm  \
NodeConfig.pm   \
diff --git a/PVE/API2/Certificates.pm b/PVE/API2/Certificates.pm
new file mode 100644
index ..cc3b2477
--- /dev/null
+++ b/PVE/API2/Certificates.pm
@@ -0,0 +1,205 @@
+package PVE::API2::Certificates;
+
+use strict;
+use warnings;
+
+use PVE::API2::ACME;
+use PVE::Certificate;
+use PVE::CertHelpers;;
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param file_get_contents file_set_contents);
+
+use base qw(PVE::RESTHandler);
+
+
+__PACKAGE__->register_method ({
+subclass => "PVE::API2::ACME",
+path => 'acme',
+});
+
+__PACKAGE__->register_method ({
+name => 'index',
+path => '',
+method => 'GET',
+permissions => { user => 'all' },
+description => "Node index.",
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   node => get_standard_option('pve-node'),
+   },
+},
+returns => {
+   type => 'array',
+   items => {
+   type => "object",
+   properties => {},
+   },
+   links => [ { rel => 'child', href => "{name}" } ],
+},
+code => sub {
+   my ($param) = @_;
+
+   return [
+   { name => 'acme' },
+   { name => 'custom' },
+   { name => 'info' },
+   ];
+},
+});
+
+__PACKAGE__->register_method ({
+name => 'info',
+path => 'info',
+method => 'GET',
+permissions => { user => 'all' },
+proxyto => 'node',
+description => "Get information about node's certificates.",
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   node => get_standard_option('pve-node'),
+   },
+},
+returns => {
+   type => 'array',
+   items => get_standard_option('pve-certificate-info'),
+},
+code => sub {
+   my ($param) = @_;
+
+   my $node_path = "/etc/pve/nodes/$param->{node}";
+
+   my $res = [];
+   my $cert_paths = [
+   '/etc/pve/pve-root-ca.pem',
+   "$node_path/pve-ssl.pem",
+   "$node_path/pveproxy-ssl.pem",
+   ];
+   for my $path (@$cert_paths) {
+   eval {
+   my $info = PVE::Certificate::get_certificate_info($path);
+   push @$res, $info if $info;
+   };
+   }
+   return $res;
+},
+});
+
+__PACKAGE__->register_method ({
+name => 'upload_custom_cert',
+path => 'custom',
+method => 'POST',
+description => 'Upload or update custom certificate chain and key.',
+protected => 1,
+proxyto => 'node',
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   node => get_standard_option('pve-node'),
+   certificates => {
+   type => 'string',
+   format => 'pem-certificate-chain',
+   description => 'PEM encoded certificate (chain).',
+   },
+   key => {
+   type => 'string',
+   description => 'PEM encoded private key.',
+   format => 'pem-string',
+   optional => 1,
+   },
+   force => {
+   type => 'boolean',
+   description => 'Overwrite existing custom or ACME certificate 
files.',
+   optional => 1,
+   default => 0,
+   },
+   restart => {
+   type => 'boolean',
+   description => 'Restart pveproxy.',
+   optional => 1,
+   default => 0,
+   },
+   },
+},
+returns => get_standard_option('pve-certificate-info'),
+code => sub {
+   my ($param) = @_;
+
+   my $node = extract_param($param, 'node');
+   my $cert_prefix = PVE::CertHelpers::cert_path_prefix($node);
+
+   my $certs = extract_param($param, 'certificates');
+   $certs = PVE::Certificate::strip_leading_text($certs);
+
+   my $key = extract_param($param, 'key');
+   if ($key) {
+   $key = PVE::Certificate::strip_leading_text($key);
+   } else {
+   raise_param_exc({'key' => "Attempted to upload custom certificate 
without (existing) key."})
+   if ! -e "${cert_prefix}.key";
+   }

[pve-devel] [PATCH v2 cluster] cluster: add cfs_lock_acme

[Fabian Grünbichler]

to lock an ACME account config file

Signed-off-by: Fabian Grünbichler 
---
 data/PVE/Cluster.pm | 8 
 1 file changed, 8 insertions(+)

diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm
index ef7a541..d12c244 100644
--- a/data/PVE/Cluster.pm
+++ b/data/PVE/Cluster.pm
@@ -960,6 +960,14 @@ sub cfs_lock_domain {
 &$cfs_lock($lockid, $timeout, $code, @param);
 }
 
+sub cfs_lock_acme {
+my ($account, $timeout, $code, @param) = @_;
+
+my $lockid = "acme-$account";
+
+&$cfs_lock($lockid, $timeout, $code, @param);
+}
+
 my $log_levels = {
 "emerg" => 0,
 "alert" => 1,
-- 
2.14.2


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v2 common 1/4] add Certificate helper

[Fabian Grünbichler]

general purpose certificate related helper functions

Signed-off-by: Fabian Grünbichler 
---
 src/Makefile   |   1 +
 src/PVE/Certificate.pm | 396 +
 2 files changed, 397 insertions(+)
 create mode 100644 src/PVE/Certificate.pm

diff --git a/src/Makefile b/src/Makefile
index e544882..c81991d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -8,6 +8,7 @@ PERLDIR=${PREFIX}/share/perl5
 
 LIB_SOURCES = \
AtomicFile.pm \
+   Certificate.pm \
CLIHandler.pm \
CalendarEvent.pm \
CpuSet.pm \
diff --git a/src/PVE/Certificate.pm b/src/PVE/Certificate.pm
new file mode 100644
index 000..b0b9bbf
--- /dev/null
+++ b/src/PVE/Certificate.pm
@@ -0,0 +1,396 @@
+package PVE::Certificate;
+
+use strict;
+use warnings;
+
+use Date::Parse;
+use Encode qw(decode encode);
+use MIME::Base64 qw(decode_base64 encode_base64);
+use Net::SSLeay;
+
+use PVE::JSONSchema qw(get_standard_option);
+
+Net::SSLeay::load_error_strings();
+Net::SSLeay::randomize();
+
+PVE::JSONSchema::register_format('pem-certificate', sub {
+my ($content, $noerr) = @_;
+
+return check_pem($content, noerr => $noerr);
+});
+
+PVE::JSONSchema::register_format('pem-certificate-chain', sub {
+my ($content, $noerr) = @_;
+
+return check_pem($content, noerr => $noerr, multiple => 1);
+});
+
+PVE::JSONSchema::register_format('pem-string', sub {
+my ($content, $noerr) = @_;
+
+return check_pem($content, noerr => $noerr, label => qr/.*?/);
+});
+
+PVE::JSONSchema::register_standard_option('pve-certificate-info', {
+type => 'object',
+properties => {
+   filename => {
+   type => 'string',
+   optional => 1,
+   },
+   fingerprint => get_standard_option('fingerprint-sha256', {
+   optional => 1,
+   }),
+   subject => {
+   type => 'string',
+   description => 'Certificate subject name.',
+   optional => 1,
+   },
+   issuer => {
+   type => 'string',
+   description => 'Certificate issuer name.',
+   optional => 1,
+   },
+   notbefore => {
+   type => 'integer',
+   description => 'Certificate\'s notBefore timestamp (UNIX epoch).',
+   optional => 1,
+   },
+   notafter => {
+   type => 'integer',
+   description => 'Certificate\'s notAfter timestamp (UNIX epoch).',
+   optional => 1,
+   },
+   san => {
+   type => 'array',
+   description => 'List of Certificate\'s SubjectAlternativeName 
entries.',
+   optional => 1,
+   items => {
+   type => 'string',
+   },
+   },
+   pem => {
+   type => 'string',
+   description => 'Certificate in PEM format',
+   format => 'pem-certificate',
+   optional => 1,
+   },
+},
+});
+
+# see RFC 7468
+my $b64_char_re = qr![0-9A-Za-z\+/]!;
+my $header_re = sub {
+my ($label) = @_;
+return qr!-BEGIN\ $label-(?:\s|\n)*!;
+};
+my $footer_re = sub {
+my ($label) = @_;
+return qr!-END\ $label-(?:\s|\n)*!;
+};
+my $pem_re = sub {
+my ($label) = @_;
+
+my $header = $header_re->($label);
+my $footer = $footer_re->($label);
+
+return qr{
+   $header
+   (?:(?:$b64_char_re)+\s*\n)*
+   (?:$b64_char_re)*(?:=\s*\n=|={0,2})?\s*\n
+   $footer
+}x;
+};
+
+sub strip_leading_text {
+my ($content) = @_;
+
+my $header = $header_re->(qr/.*?/);
+$content =~ s/^.*?(?=$header)//s;
+return $content;
+};
+
+sub split_pem {
+my ($content, %opts) = @_;
+my $label = $opts{label} // 'CERTIFICATE';
+
+my $header = $header_re->($label);
+return split(/(?=$header)/,$content);
+}
+
+sub check_pem {
+my ($content, %opts) = @_;
+
+my $label = $opts{label} // 'CERTIFICATE';
+my $multiple = $opts{multiple};
+my $noerr = $opts{noerr};
+
+$content = strip_leading_text($content);
+
+my $re = $pem_re->($label);
+
+$re = qr/($re\n+)*$re/ if $multiple;
+
+if ($content =~ /^$re$/) {
+   return $content;
+} else {
+   return undef if $noerr;
+   die "not a valid PEM-formatted string.\n";
+}
+}
+
+sub pem_to_der {
+my ($content) = @_;
+
+my $header = $header_re->(qr/.*?/);
+my $footer = $footer_re->(qr/.*?/);
+
+$content = strip_leading_text($content);
+
+# only take first PEM entry
+$content =~ s/^$header$//mg;
+$content =~ s/$footer.*//sg;
+
+$content = decode_base64($content);
+
+return $content;
+}
+
+sub der_to_pem {
+my ($content, %opts) = @_;
+
+my $label = $opts{label} // 'CERTIFICATE';
+
+my $b64 = encode_base64($content, '');
+$b64 = join("\n", ($b64 =~ /.{1,64}/sg));
+return "-BEGIN $label-\n$b64\n-END $label-\n";
+}
+
+my $ssl_die = sub {
+my ($msg) = @_;
+Net::SSLeay::die_now($msg);
+};
+
+my $ssl_warn = sub {
+my 

[pve-devel] [PATCH cluster/common/manager v2 0/10] ACME library and certificate management

[Fabian Grünbichler]

this patch series is implementing the following features:

- certificate utility module (pve-common)
- ACME v9 / Let's Encrypt v2 API client library (pve-common)
- PVE specific Certificate helper utility (pve-manager)
- new per-node configuration file in /etc/pve/nodes/FOO/config and API for 
managing (pve-manager)
- API for managing ACME accounts (pve-manager)
- API for managing custom and ACME certificate (pve-manager)

high-level changes since v1/RFC:
- pve-common: patches are re-ordered (Certificate helper before ACME)
- pve-common: CSR generation now via Net::SSLeay
- pve-manager: more API annotation / formats / registered options
- more error handling
- more helper methods
- pve-manager/acme: new expiry check for renewal (30 days)
- pve-manager/acme: new 'force' parameter to skip expiry check for renewal
- pve-manager/acme-account: get now returns local info, empty put refreshes 
from CA
- pve-manager/acme-account: new directories endpoint to return known directories
- API return values have changed for some endpoints

thanks for the feedback Dominik and Wolfgang!

TODOs as follow-ups (in order of importance ;)):
- move de-activated account files to allow re-usal of account/file name
- CLI including service/timer for renewal
- GUI
- documentation
- refactor config file handling together with LXC and Qemu Configs

potential improvements:
- make key type configurable
- ACME challenge plugin infrastructure
- support wildcard certificates (need DNS challenge with Let's Encrypt)
- manage self-signed certificates with same helpers?
- deprecate pvecm updatecerts in favour of new code in pve-manager?

pve-common diff-stat:

Fabian Grünbichler (4):
  add Certificate helper
  acme: add ACME library
  acme: add challenge plugins
  build: install ACME files

 src/Makefile   |   5 +
 src/PVE/ACME.pm| 531 +
 src/PVE/ACME/Challenge.pm  |  22 ++
 src/PVE/ACME/StandAlone.pm |  74 +++
 src/PVE/Certificate.pm | 396 +
 5 files changed, 1028 insertions(+)
 create mode 100644 src/PVE/ACME.pm
 create mode 100644 src/PVE/ACME/Challenge.pm
 create mode 100644 src/PVE/ACME/StandAlone.pm
 create mode 100644 src/PVE/Certificate.pm

pve-cluster diff-stat:

Fabian Grünbichler (1):
  cluster: add cfs_lock_acme

 data/PVE/Cluster.pm | 8 
 1 file changed, 8 insertions(+)

pve-manager diff-stat:

Fabian Grünbichler (5):
  add CertHelpers utility
  add node configuration file and API
  add ACME account API endpoints
  add ACME certificate API endpoints
  add certificates API endpoints

 PVE/API2/Makefile|   4 +
 PVE/Makefile |   2 +
 PVE/API2/ACME.pm | 325 
 PVE/API2/ACMEAccount.pm  | 347 +++
 PVE/API2/Certificates.pm | 205 
 PVE/API2/Cluster.pm  |   7 +
 PVE/API2/NodeConfig.pm   | 101 ++
 PVE/API2/Nodes.pm|  15 ++
 PVE/CertHelpers.pm   | 107 +++
 PVE/NodeConfig.pm| 205 
 10 files changed, 1318 insertions(+)
 create mode 100644 PVE/API2/ACME.pm
 create mode 100644 PVE/API2/ACMEAccount.pm
 create mode 100644 PVE/API2/Certificates.pm
 create mode 100644 PVE/API2/NodeConfig.pm
 create mode 100644 PVE/CertHelpers.pm
 create mode 100644 PVE/NodeConfig.pm

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v2 manager 4/5] add ACME certificate API endpoints

[Fabian Grünbichler]

for creating/ordering a new certificate and renewing respectively
revoking an existing one.

Signed-off-by: Fabian Grünbichler 
---
 PVE/API2/Makefile |   1 +
 PVE/API2/ACME.pm  | 325 ++
 2 files changed, 326 insertions(+)
 create mode 100644 PVE/API2/ACME.pm

diff --git a/PVE/API2/Makefile b/PVE/API2/Makefile
index d72ddd9b..44b9cf7c 100644
--- a/PVE/API2/Makefile
+++ b/PVE/API2/Makefile
@@ -14,6 +14,7 @@ PERLSOURCE =  \
Pool.pm \
Tasks.pm\
Network.pm  \
+   ACME.pm \
ACMEAccount.pm  \
NodeConfig.pm   \
Services.pm
diff --git a/PVE/API2/ACME.pm b/PVE/API2/ACME.pm
new file mode 100644
index ..185f7b1b
--- /dev/null
+++ b/PVE/API2/ACME.pm
@@ -0,0 +1,325 @@
+package PVE::API2::ACME;
+
+use strict;
+use warnings;
+
+use PVE::ACME;
+use PVE::ACME::StandAlone;
+use PVE::CertHelpers;
+use PVE::Certificate;
+use PVE::Exception qw(raise raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::NodeConfig;
+use PVE::Tools qw(extract_param);
+
+use IO::Handle;
+
+use base qw(PVE::RESTHandler);
+
+my $acme_account_dir = PVE::CertHelpers::acme_account_dir();
+
+__PACKAGE__->register_method ({
+name => 'index',
+path => '',
+method => 'GET',
+permissions => { user => 'all' },
+description => "ACME index.",
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   node => get_standard_option('pve-node'),
+   },
+},
+returns => {
+   type => 'array',
+   items => {
+   type => "object",
+   properties => {},
+   },
+   links => [ { rel => 'child', href => "{name}" } ],
+},
+code => sub {
+   my ($param) = @_;
+
+   return [
+   { name => 'certificate' },
+   ];
+}});
+
+my $order_certificate = sub {
+my ($acme, $domains) = @_;
+print "Placing ACME order\n";
+my ($order_url, $order) = $acme->new_order($domains);
+print "Order URL: $order_url\n";
+for my $auth_url (@{$order->{authorizations}}) {
+   print "\nGetting authorization details from '$auth_url'\n";
+   my $auth = $acme->get_authorization($auth_url);
+   if ($auth->{status} eq 'valid') {
+   print "... already validated!\n";
+   } else {
+   print "... pending!\n";
+   print "Setting up webserver\n";
+   my $validation = eval { PVE::ACME::StandAlone->setup($acme, $auth) 
};
+   die "failed setting up webserver - $@\n" if $@;
+
+   print "Triggering validation\n";
+   eval {
+   $acme->request_challenge_validation($validation->{url}, 
$validation->{key_auth});
+   while (1) {
+   $auth = $acme->get_authorization($auth_url);
+   if ($auth->{status} eq 'pending') {
+   print "still pending, trying again in 30 seconds\n";
+   sleep 30;
+   next;
+   } elsif ($auth->{status} eq 'valid') {
+   last;
+   }
+   die "validating challenge '$auth' failed\n";
+   }
+   };
+   my $err = $@;
+   eval { $validation->teardown() };
+   warn "$@\n" if $@;
+   die $err if $err;
+   }
+}
+print "\nAll domains validated!\n";
+print "\nCreating CSR\n";
+my ($csr, $key) = PVE::Certificate::generate_csr(identifiers => 
$order->{identifiers});
+
+print "Finalizing order\n";
+$acme->finalize_order($order, PVE::Certificate::pem_to_der($csr));
+
+print "Checking order status\n";
+while (1) {
+   $order = $acme->get_order($order_url);
+   if ($order->{status} eq 'pending') {
+   print "still pending, trying again in 30 seconds\n";
+   sleep 30;
+   next;
+   } elsif ($order->{status} eq 'valid') {
+   print "valid!\n";
+   last;
+   }
+   die "order status: $order->{status}\n";
+}
+
+print "\nDownloading certificate\n";
+my $cert = $acme->get_certificate($order);
+
+return ($cert, $key);
+};
+
+__PACKAGE__->register_method ({
+name => 'new_certificate',
+path => 'certificate',
+method => 'POST',
+description => "Order a new certificate from ACME-compatible CA.",
+protected => 1,
+proxyto => 'node',
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   node => get_standard_option('pve-node'),
+   force => {
+   type => 'boolean',
+   description => 'Overwrite existing custom certificate.',
+   optional => 1,
+   default => 0,
+   },
+   },
+},
+returns => {
+   type => 'string',
+},
+code => sub {
+   my ($param) = @_;
+
+   my $node = 

[pve-devel] [PATCH v2 common 3/4] acme: add challenge plugins

[Fabian Grünbichler]

Signed-off-by: Fabian Grünbichler 
---
 src/PVE/ACME/Challenge.pm  | 22 ++
 src/PVE/ACME/StandAlone.pm | 74 ++
 2 files changed, 96 insertions(+)
 create mode 100644 src/PVE/ACME/Challenge.pm
 create mode 100644 src/PVE/ACME/StandAlone.pm

diff --git a/src/PVE/ACME/Challenge.pm b/src/PVE/ACME/Challenge.pm
new file mode 100644
index 000..40d32b6
--- /dev/null
+++ b/src/PVE/ACME/Challenge.pm
@@ -0,0 +1,22 @@
+package PVE::ACME::Challenge;
+
+use strict;
+use warnings;
+
+sub supported_challenge_types {
+return {};
+}
+
+sub setup {
+my ($class, $acme, $authorization) = @_;
+
+die "implement me\n";
+}
+
+sub teardown {
+my ($self) = @_;
+
+die "implement me\n";
+}
+
+1;
diff --git a/src/PVE/ACME/StandAlone.pm b/src/PVE/ACME/StandAlone.pm
new file mode 100644
index 000..0d82213
--- /dev/null
+++ b/src/PVE/ACME/StandAlone.pm
@@ -0,0 +1,74 @@
+package PVE::ACME::StandAlone;
+
+use strict;
+use warnings;
+
+use base qw(PVE::ACME::Challenge);
+
+sub supported_challenge_types {
+return { 'http-01' => 1 };
+}
+
+sub setup {
+my ($class, $acme, $authorization) = @_;
+
+my $challenges = $authorization->{challenges};
+die "no challenges defined in authorization\n" if !$challenges;
+
+my $http_challenges = [ grep {$_->{type} eq 'http-01'} @$challenges ];
+die "no http-01 challenge defined in authorization\n"
+   if ! scalar $http_challenges;
+
+my $http_challenge = $http_challenges->[0];
+
+die "no token found in http-01 challenge\n" if !$http_challenge->{token};
+
+my $key_authorization = $acme->key_authorization($http_challenge->{token});
+
+my $server = PVE::ACME::StandAlone::Server->new(80);
+$server->{key_auth} = $key_authorization;
+my $pid = $server->background();
+
+my $self = {
+   server => $server,
+   pid => $pid,
+   authorization => $authorization,
+   key_auth => $key_authorization,
+   url => $http_challenge->{url},
+};
+
+return bless $self, $class;
+}
+
+sub teardown {
+my ($self) = @_;
+
+kill 'KILL', $self->{pid};
+}
+
+1;
+
+package PVE::ACME::StandAlone::Server;
+
+use HTTP::Server::Simple::CGI;
+use base qw(HTTP::Server::Simple::CGI);
+
+sub handle_request {
+my $self = shift;
+my $cgi  = shift;
+
+my $key_auth = $self->{key_auth};
+$key_auth =~ /^(.*)\..*$/;
+my $token = $1;
+
+my $path = $cgi->path_info();
+if ($path eq "/.well-known/acme-challenge/${token}") {
+   print "HTTP/1.0 200 OK\r\n";
+   print $cgi->header, $key_auth;
+} else {
+   print "HTTP/1.0 404 Not found\r\n";
+   print $cgi->header;
+}
+}
+
+1;
-- 
2.14.2


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v2 manager 1/5] add CertHelpers utility

[Fabian Grünbichler]

PVE-specific certificate helper functions

Signed-off-by: Fabian Grünbichler 
---
 PVE/Makefile   |   1 +
 PVE/CertHelpers.pm | 107 +
 2 files changed, 108 insertions(+)
 create mode 100644 PVE/CertHelpers.pm

diff --git a/PVE/Makefile b/PVE/Makefile
index 5cc19904..395faf8a 100644
--- a/PVE/Makefile
+++ b/PVE/Makefile
@@ -3,6 +3,7 @@ include ../defines.mk
 SUBDIRS=API2 Status CLI Service
 
 PERLSOURCE =   \
+   CertHelpers.pm  \
API2.pm \
API2Tools.pm\
HTTPServer.pm   \
diff --git a/PVE/CertHelpers.pm b/PVE/CertHelpers.pm
new file mode 100644
index ..52316aa0
--- /dev/null
+++ b/PVE/CertHelpers.pm
@@ -0,0 +1,107 @@
+package PVE::CertHelpers;
+
+use strict;
+use warnings;
+
+use PVE::Certificate;
+use PVE::JSONSchema;
+use PVE::Tools;
+
+my $account_prefix = '/etc/pve/priv/acme';
+
+PVE::JSONSchema::register_standard_option('pve-acme-account-name', {
+description => 'ACME account config file name.',
+type => 'string',
+format => 'pve-configid',
+format_description => 'name',
+optional => 1,
+default => 'default',
+});
+
+PVE::JSONSchema::register_standard_option('pve-acme-account-contact', {
+type => 'string',
+format => 'email-list',
+description => 'Contact email addresses.',
+});
+
+PVE::JSONSchema::register_standard_option('pve-acme-directory-url', {
+type => 'string',
+description => 'URL of ACME CA directory endpoint.',
+pattern => '^https?://.*',
+});
+
+my $local_cert_lock = '/var/lock/pve-certs.lock';
+
+sub cert_path_prefix {
+my ($node) = @_;
+
+return "/etc/pve/nodes/${node}/pveproxy-ssl";
+}
+
+sub cert_lock {
+my ($timeout, $code, @param) = @_;
+
+return PVE::Tools::lock_file($local_cert_lock, $timeout, $code, @param);
+}
+
+sub set_cert_files {
+my ($cert, $key, $path_prefix, $force) = @_;
+
+my ($old_cert, $old_key, $info);
+
+my $cert_path = "${path_prefix}.pem";
+my $cert_path_tmp = "${path_prefix}.pem.old";
+my $key_path = "${path_prefix}.key";
+my $key_path_tmp = "${path_prefix}.key.old";
+
+die "Custom certificate file exists but force flag is not set.\n"
+   if !$force && -e $cert_path;
+die "Custom certificate key file exists but force flag is not set.\n"
+   if !$force && -e $key_path;
+
+PVE::Tools::file_copy($cert_path, $cert_path_tmp) if -e $cert_path;
+PVE::Tools::file_copy($key_path, $key_path_tmp) if -e $key_path;
+
+eval {
+   PVE::Tools::file_set_contents($cert_path, $cert);
+   PVE::Tools::file_set_contents($key_path, $key) if $key;
+   $info = PVE::Certificate::get_certificate_info($cert_path);
+};
+my $err = $@;
+
+if ($err) {
+   if (-e $cert_path_tmp && -e $key_path_tmp) {
+   eval {
+   warn "Attempting to restore old certificate files..\n";
+   PVE::Tools::file_copy($cert_path_tmp, $cert_path);
+   PVE::Tools::file_copy($key_path_tmp, $key_path);
+   };
+   warn "$@\n" if $@;
+   }
+   die "Setting certificate files failed - $err\n"
+}
+
+unlink $cert_path_tmp;
+unlink $key_path_tmp;
+
+return $info;
+}
+
+sub acme_account_dir {
+return $account_prefix;
+}
+
+sub list_acme_accounts {
+my $accounts = [];
+
+return $accounts if ! -d $account_prefix;
+
+PVE::Tools::dir_glob_foreach($account_prefix, qr/[^.]+.*/, sub {
+   my ($name) = @_;
+
+   push @$accounts, $name
+   if PVE::JSONSchema::pve_verify_configid($name, 1);
+});
+
+return $accounts;
+}
-- 
2.14.2


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH librados2-perl v2] Convert to dpkg-buildpackage so we get a .changes file

[René Jochum]

Also fixes a minor spelling error in changelog.Debian and extends the
.gitignore to ignore build files.

Signed-off-by: René Jochum 
---
 .gitignore   |  7 +++
 Makefile | 29 ++--
 changelog.Debian => debian/changelog |  2 +-
 debian/compat|  1 +
 control.in => debian/control.in  |  9 +
 copyright => debian/copyright|  0
 debian/rules | 11 +++
 7 files changed, 39 insertions(+), 20 deletions(-)
 create mode 100644 .gitignore
 rename changelog.Debian => debian/changelog (96%)
 create mode 100644 debian/compat
 rename control.in => debian/control.in (84%)
 rename copyright => debian/copyright (100%)
 create mode 100755 debian/rules

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 000..70af628
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,7 @@
+RADOS.c
+RADOS.so
+build/
+
+*.buildinfo
+*.changes
+*.deb
diff --git a/Makefile b/Makefile
index 2c1451f..c2473a7 100644
--- a/Makefile
+++ b/Makefile
@@ -14,6 +14,8 @@ DOCDIR=${PREFIX}/share/doc/${PACKAGE}
 MAN1DIR=${MANDIR}/man1/
 PERLDIR=${PREFIX}/share/perl5

+BUILDDIR=build
+
 PERL_ARCHLIB := `perl -MConfig -e 'print $$Config{archlib};'`
 PERL_INSTALLVENDORARCH := `perl -MConfig -e 'print 
$$Config{installvendorarch};'`
 PERL_APIVER := `perl -MConfig -e 'print 
$$Config{debian_abi}//$$Config{version};'`
@@ -26,7 +28,7 @@ CFLAGS= -shared -fPIC -O2 -Werror -Wtype-limits -Wall 
-Wl,-z,relro \
 PERLSODIR=$(PERL_INSTALLVENDORARCH)/auto

 ARCH:=$(shell dpkg-architecture -qDEB_BUILD_ARCH)
-GITVERSION:=$(shell cat .git/refs/heads/master)
+GITVERSION:=$(shell git rev-parse HEAD)

 DEB=${PACKAGE}_${VERSION}-${PKGREL}_${ARCH}.deb

@@ -52,23 +54,20 @@ install: PVE/RADOS.pm RADOS.so
 .PHONY: deb ${DEB}
 deb: ${DEB}
 ${DEB}:
-   rm -rf debian
-   mkdir debian
-   make DESTDIR=${CURDIR}/debian install
-   install -d -m 0755 debian/DEBIAN
-   sed -e s/@@VERSION@@/${VERSION}/ -e s/@@PKGRELEASE@@/${PKGREL}/ -e 
s/@@ARCH@@/${ARCH}/ -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" debian/DEBIAN/control
-   install -D -m 0644 copyright debian/${DOCDIR}/copyright
-   install -m 0644 changelog.Debian debian/${DOCDIR}/
-   gzip -9 -n debian/${DOCDIR}/changelog.Debian
-   echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
checkout ${GITVERSION}" > debian/${DOCDIR}/SOURCE
-   fakeroot dpkg-deb --build debian
-   mv debian.deb ${DEB}
-   rm -rf debian
+   rm -rf ${BUILDDIR}
+   rsync -ra * ${BUILDDIR}
+
+   sed -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" ${BUILDDIR}/debian/control
+
+   echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
checkout ${GITVERSION}" > ${BUILDDIR}/debian/SOURCE
+
+   cd ${BUILDDIR}; dpkg-buildpackage -b -uc -us
+   rm -rf build
lintian ${DEB}

 .PHONY: clean
-clean:
-   rm -rf debian *.deb ${PACKAGE}-*.tar.gz dist *.1.pod *.1.gz RADOS.so 
RADOS.c
+clean:
+   rm -rf ${BUILDDIR} *.deb ${PACKAGE}-*.tar.gz dist *.1.pod *.1.gz 
RADOS.so RADOS.c
find . -name '*~' -exec rm {} ';'

 .PHONY: distclean
diff --git a/changelog.Debian b/debian/changelog
similarity index 96%
rename from changelog.Debian
rename to debian/changelog
index 7f4b898..38084e2 100644
--- a/changelog.Debian
+++ b/debian/changelog
@@ -1,6 +1,6 @@
 librados2-perl (1.0-5) unstable; urgency=medium

-  * allow to specify the userid with rados_create
+  * allow one to specify the userid with rados_create

   * split method pve_rados_connect

diff --git a/debian/compat b/debian/compat
new file mode 100644
index 000..9a03714
--- /dev/null
+++ b/debian/compat
@@ -0,0 +1 @@
+10
\ No newline at end of file
diff --git a/control.in b/debian/control.in
similarity index 84%
rename from control.in
rename to debian/control.in
index c2dc77e..5e32cab 100644
--- a/control.in
+++ b/debian/control.in
@@ -1,9 +1,10 @@
-Package: librados2-perl
-Version: @@VERSION@@-@@PKGRELEASE@@
+Source: librados2-perl
+Maintainer: Proxmox Support Team 
 Section: perl
 Priority: optional
-Architecture: @@ARCH@@
+
+Package: librados2-perl
+Architecture: any
 Depends: libc6 (>= 2.2.5), perl (>= 5.20.1-5), @PERLAPI@, librados2 (>= 
0.67.5), libpve-access-control
-Maintainer: Proxmox Support Team 
 Description: Perl bindings for librados
  This package contains librados perl binding used by Proxmox VE.
diff --git a/copyright b/debian/copyright
similarity index 100%
rename from copyright
rename to debian/copyright
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 000..a4652c2
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,11 @@
+#!/usr/bin/make -f
+# See debhelper(7) (uncomment to enable)
+# output every command that modifies files on the build system.
+DH_VERBOSE = 1
+
+
+%:
+   dh $@
+
+override_dh_auto_build:
+   make RADOS.so
--
2.17.0

Re: [pve-devel] [PATCH librados2-perl] Convert to dpkg-buildpackage

[Fabian Grünbichler]

On Thu, Apr 19, 2018 at 11:31:08AM +0200, Rene Jochum wrote:
> Thanks again, comments inline.
> 
> On 2018-04-19 10:16, Fabian Grünbichler wrote:
> >> @@ -52,23 +54,27 @@ install: PVE/RADOS.pm RADOS.so
> >>  .PHONY: deb ${DEB}
> >>  deb: ${DEB}
> >>  ${DEB}:
> >> -  rm -rf debian
> >> -  mkdir debian
> >> -  make DESTDIR=${CURDIR}/debian install
> >> -  install -d -m 0755 debian/DEBIAN
> >> -  sed -e s/@@VERSION@@/${VERSION}/ -e s/@@PKGRELEASE@@/${PKGREL}/ -e 
> >> s/@@ARCH@@/${ARCH}/ -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g"  >> >debian/DEBIAN/control
> >> -  install -D -m 0644 copyright debian/${DOCDIR}/copyright
> >> -  install -m 0644 changelog.Debian debian/${DOCDIR}/
> >> -  gzip -9 -n debian/${DOCDIR}/changelog.Debian
> >> -  echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
> >> checkout ${GITVERSION}" > debian/${DOCDIR}/SOURCE
> >> -  fakeroot dpkg-deb --build debian
> >> -  mv debian.deb ${DEB}
> >> -  rm -rf debian
> >> +  rm -rf ${BUILDDIR}
> >> +  rsync -a * ${BUILDDIR}
> >> +
> >> +  cp -fpr debian ${BUILDDIR}
> > why rsync + cp? one should be enough ;)
> 
> Hihi, ofc.
> 
> > 
> >> +  cp changelog.Debian ${BUILDDIR}/debian/changelog
> >> +  cp copyright ${BUILDDIR}/debian/copyright
> > see above, those two could go into the checked-in debian/
> > 
> >> +  sed -e s/@@VERSION@@/${VERSION}/ -e s/@@PKGRELEASE@@/${PKGREL}/ -e 
> >> s/@@ARCH@@/${ARCH}/ -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" 
> >> ${BUILDDIR}/debian/control
> > except for the PERLAPI one, all of these should be dropped / replaced:
> > VERSION and PKGRELEASE come from debian/changelog
> > ARCH comes via control and dpkg-architecture
> > 
> > whether the PERLAPI is really needed should probably be investigated.
> 
> It is needed, as "RADOS.so" only works with that perl Version - on of
> the main reasons I have to recompile on Buster.

I meant whether we really need to auto-generate it via a variable.

modifying debian/control once per major release is probably not an
issue, and we can skip generating debian/control then (it also serves as
a reminder to change for ani API-related breakage ;)).

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH librados2-perl] Convert to dpkg-buildpackage

[Fabian Grünbichler]

On Thu, Apr 19, 2018 at 11:02:44AM +0200, Rene Jochum wrote:
> On 2018-04-19 10:16, Fabian Grünbichler wrote:
> > some comments inline (your patch also only applies when ignoring
> > whitespace changes!)
> 
> 
> Hi Fabian,
> 
> thank you for looking into it! Can you give me a tip on making patches
> that apply without ignoring white space changes?

I think your editor or MUA modifies lines only containing whitespace.

e.g., if I diff the patch as you sent it, and the patch exported again
after applying it with '--ignore-whitespace', I see lots of lines where
your original patch has /^$/ (a completely empty line), and the
re-exported one has /^ $/ (a line containing a single space).

the former is not valid patch syntax, because the first character
indicates added (+), removed (-) or context ( ), the latter means an
otherwise empty context line.

> Will implement all the given recommendations now.

great :)

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH librados2-perl] Convert to dpkg-buildpackage

[Rene Jochum]

Thanks again, comments inline.

On 2018-04-19 10:16, Fabian Grünbichler wrote:
>> @@ -52,23 +54,27 @@ install: PVE/RADOS.pm RADOS.so
>>  .PHONY: deb ${DEB}
>>  deb: ${DEB}
>>  ${DEB}:
>> -rm -rf debian
>> -mkdir debian
>> -make DESTDIR=${CURDIR}/debian install
>> -install -d -m 0755 debian/DEBIAN
>> -sed -e s/@@VERSION@@/${VERSION}/ -e s/@@PKGRELEASE@@/${PKGREL}/ -e 
>> s/@@ARCH@@/${ARCH}/ -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" > >debian/DEBIAN/control
>> -install -D -m 0644 copyright debian/${DOCDIR}/copyright
>> -install -m 0644 changelog.Debian debian/${DOCDIR}/
>> -gzip -9 -n debian/${DOCDIR}/changelog.Debian
>> -echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
>> checkout ${GITVERSION}" > debian/${DOCDIR}/SOURCE
>> -fakeroot dpkg-deb --build debian
>> -mv debian.deb ${DEB}
>> -rm -rf debian
>> +rm -rf ${BUILDDIR}
>> +rsync -a * ${BUILDDIR}
>> +
>> +cp -fpr debian ${BUILDDIR}
> why rsync + cp? one should be enough ;)

Hihi, ofc.

> 
>> +cp changelog.Debian ${BUILDDIR}/debian/changelog
>> +cp copyright ${BUILDDIR}/debian/copyright
> see above, those two could go into the checked-in debian/
> 
>> +sed -e s/@@VERSION@@/${VERSION}/ -e s/@@PKGRELEASE@@/${PKGREL}/ -e 
>> s/@@ARCH@@/${ARCH}/ -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" 
>> ${BUILDDIR}/debian/control
> except for the PERLAPI one, all of these should be dropped / replaced:
> VERSION and PKGRELEASE come from debian/changelog
> ARCH comes via control and dpkg-architecture
> 
> whether the PERLAPI is really needed should probably be investigated.

It is needed, as "RADOS.so" only works with that perl Version - on of
the main reasons I have to recompile on Buster.

> 
>> +make DESTDIR=${BUILDDIR} install
>> +install -D -m 0644 copyright ${BUILDDIR}/${DOCDIR}/copyright
>> +install -m 0644 changelog.Debian ${BUILDDIR}/${DOCDIR}/
>> +gzip -9 -n ${BUILDDIR}/${DOCDIR}/changelog.Debian
> these four should not be needed (dpkg-buildpackage will call the
> Makefile to compile/build anyway, and sets DESTDIR correctly)
> 
> copyright and changelog are handled by the appropriate debhelpers
> 
>> +echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
>> checkout ${GITVERSION}" > ${BUILDDIR}/${DOCDIR}/SOURCE
> this does not work as intended (debdiff says the SOURCE file is not
> contained after applying the patch)

Fixed, in V2




-- 
René Jochum
Mail: r...@jochums.at
Tel: +43 664 750 77 653
Web: https://rene.jochums.at
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH librados2-perl] Convert to dpkg-buildpackage

[Rene Jochum]

On 2018-04-19 10:16, Fabian Grünbichler wrote:
> some comments inline (your patch also only applies when ignoring
> whitespace changes!)


Hi Fabian,

thank you for looking into it! Can you give me a tip on making patches
that apply without ignoring white space changes?

Will implement all the given recommendations now.

Thanks,
René


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager] fix #1732: actually post values in IScsiEdit

[Dominik Csapak]

this prevented creating/editing an iscsi storage

Signed-off-by: Dominik Csapak 
---
 www/manager6/storage/IScsiEdit.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/www/manager6/storage/IScsiEdit.js 
b/www/manager6/storage/IScsiEdit.js
index c7e8ecaf..33a198dd 100644
--- a/www/manager6/storage/IScsiEdit.js
+++ b/www/manager6/storage/IScsiEdit.js
@@ -65,7 +65,7 @@ Ext.define('PVE.storage.IScsiInputPanel', {
values.content = values.luns ? 'images' : 'none';
delete values.luns;
 
-   me.callParent([values]);
+   return me.callParent([values]);
 },
 
 setValues: function(values) {
-- 
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH librados2-perl] Convert to dpkg-buildpackage

[Fabian Grünbichler]

thanks for this! one of the leftover packages which we haven't converted
so far (for lack of recent changes). now that Alwin did some changes and
is planning to do some more, it's probably a good idea to convert it.

some comments inline (your patch also only applies when ignoring
whitespace changes!)

I already tested some of my recommendations to make sure they work, so
maybe you want to use the following on top of your patch as the base of
a v2:

---8<---
From afc0bb4ebf13d90d8bee5a0da4633c5df59dfc3c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fabian=20Gr=C3=BCnbichler?= 
Date: Thu, 19 Apr 2018 10:04:48 +0200
Subject: [PATCH] WIP
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Fabian Grünbichler 
---
 Makefile | 17 +
 debian/control.in|  2 +-
 changelog.Debian => debian/changelog |  0
 copyright => debian/copyright|  0
 debian/librados2-perl.docs   |  1 +
 debian/rules |  3 ++-
 6 files changed, 9 insertions(+), 14 deletions(-)
 rename changelog.Debian => debian/changelog (100%)
 rename copyright => debian/copyright (100%)
 create mode 100644 debian/librados2-perl.docs

diff --git a/Makefile b/Makefile
index a99627d..13add44 100644
--- a/Makefile
+++ b/Makefile
@@ -55,18 +55,11 @@ install: PVE/RADOS.pm RADOS.so
 deb: ${DEB}
 ${DEB}:
rm -rf ${BUILDDIR}
-   rsync -a * ${BUILDDIR}
-
-   cp -fpr debian ${BUILDDIR}
-   cp changelog.Debian ${BUILDDIR}/debian/changelog
-   cp copyright ${BUILDDIR}/debian/copyright
-   sed -e s/@@VERSION@@/${VERSION}/ -e s/@@PKGRELEASE@@/${PKGREL}/ -e 
s/@@ARCH@@/${ARCH}/ -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" 
${BUILDDIR}/debian/control
-
-   make DESTDIR=${BUILDDIR} install
-   install -D -m 0644 copyright ${BUILDDIR}/${DOCDIR}/copyright
-   install -m 0644 changelog.Debian ${BUILDDIR}/${DOCDIR}/
-   gzip -9 -n ${BUILDDIR}/${DOCDIR}/changelog.Debian
-   echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
checkout ${GITVERSION}" > ${BUILDDIR}/${DOCDIR}/SOURCE
+   rsync -ra * ${BUILDDIR}
+
+   sed -e "s|@PERLAPI@|perlapi-$(PERL_APIVER)|g" ${BUILDDIR}/debian/control
+
+   echo "git clone git://git.proxmox.com/git/librados2-perl.git\\ngit 
checkout ${GITVERSION}" > ${BUILDDIR}/SOURCE
 
cd ${BUILDDIR}; dpkg-buildpackage -b -uc -us
rm -rf build
diff --git a/debian/control.in b/debian/control.in
index cb29f38..f5fdc8e 100644
--- a/debian/control.in
+++ b/debian/control.in
@@ -5,7 +5,7 @@ Priority: optional
 Standards-Version: @@VERSION@@-@@PKGRELEASE@@
 
 Package: librados2-perl
-Architecture: @@ARCH@@
+Architecture: any
 Depends: libc6 (>= 2.2.5), perl (>= 5.20.1-5), @PERLAPI@, librados2 (>= 
0.67.5), libpve-access-control
 Description: Perl bindings for librados
  This package contains librados perl binding used by Proxmox VE.
diff --git a/changelog.Debian b/debian/changelog
similarity index 100%
rename from changelog.Debian
rename to debian/changelog
diff --git a/copyright b/debian/copyright
similarity index 100%
rename from copyright
rename to debian/copyright
diff --git a/debian/librados2-perl.docs b/debian/librados2-perl.docs
new file mode 100644
index 000..11b531f
--- /dev/null
+++ b/debian/librados2-perl.docs
@@ -0,0 +1 @@
+SOURCE
diff --git a/debian/rules b/debian/rules
index 0983a2d..a4652c2 100755
--- a/debian/rules
+++ b/debian/rules
@@ -7,4 +7,5 @@ DH_VERBOSE = 1
 %:
dh $@
 
-override_dh_auto_build:
\ No newline at end of file
+override_dh_auto_build:
+   make RADOS.so
-- 
2.14.2

--->8---

On Wed, Apr 18, 2018 at 10:27:00PM +0200, René Jochum wrote:
> Also fixes a minor spelling error in changelog.Debian and extends the
> .gitignore to ignore build files.
> 
> I've made this so i get a .changes file which i need for my dput usage.
> ---
>  .gitignore  |  7 +++
>  Makefile| 36 +++--
>  changelog.Debian|  2 +-

I'd move this into debian as well ;)

>  debian/compat   |  1 +
>  control.in => debian/control.in |  8 +---
>  debian/rules| 10 +
>  6 files changed, 45 insertions(+), 19 deletions(-)
>  create mode 100644 .gitignore
>  create mode 100644 debian/compat
>  rename control.in => debian/control.in (83%)
>  create mode 100755 debian/rules
> 
> diff --git a/.gitignore b/.gitignore
> new file mode 100644
> index 000..70af628
> --- /dev/null
> +++ b/.gitignore
> @@ -0,0 +1,7 @@
> +RADOS.c
> +RADOS.so

these should only be generated in build/ , and thus not need to be
ignored separately? doesn't really hurt though.

> +build/
> +
> +*.buildinfo
> +*.changes
> +*.deb
> diff --git a/Makefile b/Makefile
> index 2c1451f..a99627d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -14,6 +14,8 @@ 

[pve-devel] [PatchV2 storage 2/2] fix #1691: increase timeout in worker

[Wolfgang Link]

A ZFS storage under heavy load can take more time.
---
 PVE/Storage/ZFSPoolPlugin.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[Patch V2]
make timout depending on whether it is in a worker or not.

diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm
index 9680a94..7abf3c6 100644
--- a/PVE/Storage/ZFSPoolPlugin.pm
+++ b/PVE/Storage/ZFSPoolPlugin.pm
@@ -211,7 +211,7 @@ sub alloc_image {
$class->zfs_create_zvol($scfg, $volname, $size);
my $devname = "/dev/zvol/$scfg->{pool}/$volname";
 
-   my $timeout = 10;
+   my $timeout = PVE::RPCEnvironment->is_worker() ? 60*5 : 10;
for (my $i = 1; $i <= $timeout; $i++) {
last if -b $devname;
die "Timeout: no zvol after $timeout sec found.\n"
-- 
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [Patch V2 manager] fix #1691: use asynchronous API call for HD create

[Wolfgang Link]

It should use an asynchronous API call to create an image,
because some storages may need longer to alloc an image.

If a storage needs more than 30 sec
the proxy will timeout and we have an orphan image.

Also, it is possible to increase the timeout for some
storages if an asynchronous call is used
---
 www/manager6/qemu/HDEdit.js | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/www/manager6/qemu/HDEdit.js b/www/manager6/qemu/HDEdit.js
index 237128e5..4b6039a7 100644
--- a/www/manager6/qemu/HDEdit.js
+++ b/www/manager6/qemu/HDEdit.js
@@ -364,6 +364,8 @@ Ext.define('PVE.qemu.HDEdit', {
 
 isAdd: true,
 
+backgroundDelay: 5,
+
 initComponent : function() {
var me = this;
 
-- 
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [Patch V2] fix #1961

[Wolfgang Link]

Use the async API call 'update_vm' to create a HD.
This will encourage a worker and we can increase the time-out for ZFS at high 
load.

Patch V1 -> V2
[storage]
Rebased 32c0a1d59bfea6a21212cb5386a9d06e27089f10 -> 
bce13b7f505ea63bc92a12d3beabe0ebb9a802fe
make timout depending on whether it is in a worker or not



___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PatchV2 storage 1/2] fix #1691: replace udev check

[Wolfgang Link]

`zfs create` add the creation job in a worker queue,
which should normally execute instantly. But there are circumstances
where the job will take a while to get processed.
If this is the case udev settle will see no dev in the queue and the program
will continue without an allocated dev.

The busy waiting is not best practice but the only way to be sure,
that the block device exists.
---
 PVE/Storage/ZFSPoolPlugin.pm | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm
index e864a58..9680a94 100644
--- a/PVE/Storage/ZFSPoolPlugin.pm
+++ b/PVE/Storage/ZFSPoolPlugin.pm
@@ -211,9 +211,14 @@ sub alloc_image {
$class->zfs_create_zvol($scfg, $volname, $size);
my $devname = "/dev/zvol/$scfg->{pool}/$volname";
 
-   run_command("udevadm trigger --subsystem-match block");
-   system('udevadm', 'settle', '--timeout', '10', 
"--exit-if-exists=${devname}");
+   my $timeout = 10;
+   for (my $i = 1; $i <= $timeout; $i++) {
+   last if -b $devname;
+   die "Timeout: no zvol after $timeout sec found.\n"
+   if $i == $timeout;
 
+   sleep(1);
+   }
 } elsif ( $fmt eq 'subvol') {
 
die "illegal name '$volname' - sould be 'subvol-$vmid-*'\n"
-- 
2.11.0


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel