[pve-devel] applied-series: Re: [PATCH qemu 1/2] Add some qemu_vfree statements to prevent memory leaks

2020-06-24 Thread Thomas Lamprecht
On 6/22/20 2:54 PM, Stefan Reiter wrote:
> Suggested-by: Lars Ellenberg 
> Signed-off-by: Stefan Reiter 
> ---
>  vma-writer.c | 2 ++
>  vma.c| 2 ++
>  2 files changed, 4 insertions(+)
> 
>

applied series, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager v2 2/2] ui: fw: Show warning only if some rule is enabled

2020-06-24 Thread Thomas Lamprecht
Am 6/24/20 um 11:32 AM schrieb Dominic Jäger:
> Fixup for #2815: The existence of a rule alone should NOT yet trigger the
> warning. Only if it is enabled but the whole firewall for that level is not.
> 
> Signed-off-by: Dominic Jäger 
> ---
> Didn't exist in v1. Not sure if we want this. If yes you can just apply both 
> or
> squash. Of course, I can also send it as v3.
> 

Makes sense, IMO. If 1/2 works as expected we can just commit it as is, if 
there'd
be a v3 you could indeed squash it.

>  www/manager6/grid/FirewallRules.js | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/www/manager6/grid/FirewallRules.js 
> b/www/manager6/grid/FirewallRules.js
> index 99f85fb6..9bf723e1 100644
> --- a/www/manager6/grid/FirewallRules.js
> +++ b/www/manager6/grid/FirewallRules.js
> @@ -491,7 +491,8 @@ Ext.define('PVE.FirewallRules', {
>   Ext.Msg.alert(gettext('Error'), response.htmlStatus);
>   },
>   success: function (response) {
> - let warningRequired = me.store.getCount() != 0 && 
> !response.result.data.enable;
> + let warningRequired = !response.result.data.enable
> + && me.store.findExact('enable', true) >= 0;
>   
> me.down('displayfield[name=fw-warning]').setVisible(warningRequired)
>   },
>   });
> 


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [RFC qemu-server] close #2741: check for VM.Config.Cloudinit permission

2020-06-24 Thread Mira Limbeck

On 6/24/20 11:51 AM, Fabian Grünbichler wrote:

On June 3, 2020 3:58 pm, Mira Limbeck wrote:

This allows setting ciuser, cipassword and all other cloudinit settings that
are not part of the network without VM.Config.Network permissions.

Signed-off-by: Mira Limbeck 
---
  PVE/API2/Qemu.pm | 5 -
  1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 974ee3b..23a569e 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -357,8 +357,11 @@ my $check_vm_modify_config_perm = sub {
$rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.PowerMgmt']);
} elsif ($diskoptions->{$opt}) {
$rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.Config.Disk']);
-   } elsif ($cloudinitoptions->{$opt} || ($opt =~ 
m/^(?:net|ipconfig)\d+$/)) {
+   } elsif ($opt =~ m/^(?:net|ipconfig)\d+$/) {
$rpcenv->check_vm_perm($authuser, $vmid, $pool, 
['VM.Config.Network']);
+   } elsif ($cloudinitoptions->{$opt}) {
+   print "checking VM.Config.Cloudinit\n";

print probably left from debugging?

Yes, somehow I missed that.



+   $rpcenv->check_vm_perm($authuser, $vmid, $pool, 
['VM.Config.Cloudinit']);

shouldn't this also still work with VM.Config.Network to not break
existing setups, at least for the duration of 6.x?


Yes, otherwise this breaks for custom roles.


} elsif ($opt eq 'vmstate') {
# the user needs Disk and PowerMgmt privileges to change the vmstate
# also needs privileges on the storage, that will be checked later
--
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [RFC qemu-server] close #2741: check for VM.Config.Cloudinit permission

2020-06-24 Thread Fabian Grünbichler
On June 3, 2020 3:58 pm, Mira Limbeck wrote:
> This allows setting ciuser, cipassword and all other cloudinit settings that
> are not part of the network without VM.Config.Network permissions.
> 
> Signed-off-by: Mira Limbeck 
> ---
>  PVE/API2/Qemu.pm | 5 -
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index 974ee3b..23a569e 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -357,8 +357,11 @@ my $check_vm_modify_config_perm = sub {
>   $rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.PowerMgmt']);
>   } elsif ($diskoptions->{$opt}) {
>   $rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.Config.Disk']);
> - } elsif ($cloudinitoptions->{$opt} || ($opt =~ 
> m/^(?:net|ipconfig)\d+$/)) {
> + } elsif ($opt =~ m/^(?:net|ipconfig)\d+$/) {
>   $rpcenv->check_vm_perm($authuser, $vmid, $pool, 
> ['VM.Config.Network']);
> + } elsif ($cloudinitoptions->{$opt}) {
> + print "checking VM.Config.Cloudinit\n";

print probably left from debugging?

> + $rpcenv->check_vm_perm($authuser, $vmid, $pool, 
> ['VM.Config.Cloudinit']);

shouldn't this also still work with VM.Config.Network to not break 
existing setups, at least for the duration of 6.x?

>   } elsif ($opt eq 'vmstate') {
>   # the user needs Disk and PowerMgmt privileges to change the vmstate
>   # also needs privileges on the storage, that will be checked later
> -- 
> 2.20.1
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH v2 common 1/2] JSONSchema: add format validator support and cleanup check_format

2020-06-24 Thread Fabian Grünbichler
On June 24, 2020 10:54 am, Stefan Reiter wrote:
> On 6/23/20 3:39 PM, Fabian Grünbichler wrote:
>> LGTM, what do you think about the following follow-up:
>> 
>> --8<-
>> 
>> diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm
>> index b2ba9f7..d28143d 100644
>> --- a/src/PVE/JSONSchema.pm
>> +++ b/src/PVE/JSONSchema.pm
>> @@ -1878,9 +1878,12 @@ sub generate_typetext {
>>   sub print_property_string {
>>   my ($data, $format, $skip, $path) = @_;
>>   
>> +my $validator;
>>   if (ref($format) ne 'HASH') {
>>  my $schema = get_format($format);
>>  die "not a valid format: $format\n" if !$schema;
>> +# named formats can have validators attached
>> +$validator = $format_validators->{$format};
>>  $format = $schema;
>>   }
>>   
>> @@ -1890,6 +1893,8 @@ sub print_property_string {
>>  raise "format error", errors => $errors;
>>   }
>>   
>> +$res = $validator->($res) if $validator;
> 
> This would have to be $data, I suppose?

yes, sorry!

> 
>> +
>>   my ($default_key, $keyAliasProps) = &$find_schema_default_key($format);
>>   
>>   my $res = '';
>> 
>> ->8--
>> 
>> to ensure our code calling 'print_property_string' also adheres to the
>> format the validator enforces?
>> 
> 
> Fine by me, though I'm not sure how necessary. It implies that 
> validators always validate positively on values they have already 
> accepted (and potentially modified) once before though. Also I'm not 
> sure I would use the validators result for print_property_string, since 
> that means that the value the user passes in might not be the one 
> printed (if a validator modifies it) - so maybe call the validator but 
> throw away it's result?

well, it's not a given at all that $data is something that was 
previously/originally returned by parse_property_string. but yes, IMHO 
every object that we transform into a property string should pass the 
checks that that property string parsed back into an object passes (what 
a sentence -.-). or in other words, a validator should not transform a 
valid value into an invalid one ;)

I was not sure whether to include modifications done by the validator or 
not, but I'd tend to include them just to make the interface simpler. 
otherwise we'd have to dclone $data, because even if we throwaway the 
return value $data might have been modified. also, the question is 
whether to run the validator before or after check_object. for almost 
all cases it probably does not make a difference, so we might also just 
re-visit that if we ever find the need to switch the order around.

the main use case for a validator that modifies is probably to enforce a 
single/newer variant where the format itself would accept multiple, and 
that is something that does not hurt in the opposite direction either.

in any case the whole mechanism should probably be documented next to 
register_format's declaration.

> 
>> it might also be nice to mark formats with a validator (at least for the
>> API dump) to make it obvious that the displayed format might be further
>> restricted.
>> 
> 
> Sounds good to me, I'll look into it.
> 
>> I went through the code paths (again) and still think it might be
>> worthwhile to extend named formats to check_object as well, instead of
>> just scalar property string values. but that is a bigger follow-up, for
>> now limiting the scope of these validators to just property strings
>> seems sensible.
>> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager v2 1/2] ui: fw: Close #2815: Add warning if fw is disabled

2020-06-24 Thread Dominic Jäger
Currently people add firewall rules but forget to activate the firewall on
guest level. This commit adds a warning to the top bar of the firewall panel to
make them aware of this if necessary.

It avoids one O(n) lookup and might be sufficient to check only if some rule
exists and not for every rule if it is really enabled.

Signed-off-by: Dominic Jäger 
---
v1->v2: Thank you for the feedback, Dominik!
 - The 10 top margin was intentional, but using 'auto' is better I think.
   It has in my tests avoided that the field randomly sticks to the top without
   breaking the rest of the spacing.
 - Message now depends on level: datacenter, node, guest. Using pveSelNode was
   the first solution that came to my mind and it didn't seem clearly inferior
   to adding an new "level" member (requirement to keep in sync with 
pveSelNode?)
   to the class.
 - Drop leftover 'dock' property

Not really convinced by the getLevel function but had no better idea
 www/manager6/grid/FirewallRules.js | 43 --
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/www/manager6/grid/FirewallRules.js 
b/www/manager6/grid/FirewallRules.js
index ec2d1c84..99f85fb6 100644
--- a/www/manager6/grid/FirewallRules.js
+++ b/www/manager6/grid/FirewallRules.js
@@ -483,8 +483,26 @@ Ext.define('PVE.FirewallRules', {
throw "no list_refs_url specified";
}
 
+   let checkWarning = function () {
+   Proxmox.Utils.API2Request({
+   url: me.base_url.replace('rules', 'options'),
+   method: 'GET',
+   failure: function (response) {
+   Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+   },
+   success: function (response) {
+   let warningRequired = me.store.getCount() != 0 && 
!response.result.data.enable;
+   
me.down('displayfield[name=fw-warning]').setVisible(warningRequired)
+   },
+   });
+
+   };
+
var store = Ext.create('Ext.data.Store',{
-   model: 'pve-fw-rule'
+   model: 'pve-fw-rule',
+   listeners: {
+   'load': checkWarning,
+   },
});
 
var reload = function() {
@@ -606,12 +624,33 @@ Ext.define('PVE.FirewallRules', {
}
});
 
+   let getLevel = (id) => {
+   let invalid = 'this';
+   let level = /root/.test(id) ? 'datacenter'
+   : /node/.test(id) ? 'node'
+   : /qemu/.test(id) ? 'VM'
+   : /lxc/.test(id) ? 'container'
+   : invalid;
+   if (level === invalid) { console.warn(`Finding level failed for 
${id}`)};
+   return level;
+   };
+   me.warningField = Ext.create('Ext.form.field.Display',{
+   xtype: 'displayfield',
+   userCls: 'pmx-hint',
+   name: 'fw-warning',
+   margin: 'auto 0 0 0', // Avoid field randomly sticking at top
+   value: gettext(`Warning: Firewall still disabled at `
+   + `${getLevel(me.pveSelNode.id)} level! `
+   + `This can be changed in Firewall->Options.`),
+   hidden: true,
+   });
+
var tbar = me.tbar_prefix ? [ me.tbar_prefix ] : [];
tbar.push(me.addBtn, me.copyBtn);
if (me.groupBtn) {
tbar.push(me.groupBtn);
}
-   tbar.push(me.removeBtn, me.editBtn);
+   tbar.push(me.removeBtn, me.editBtn, me.warningField);
 
var render_errors = function(name, value, metaData, record) {
var errors = record.data.errors;
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager v2 2/2] ui: fw: Show warning only if some rule is enabled

2020-06-24 Thread Dominic Jäger
Fixup for #2815: The existence of a rule alone should NOT yet trigger the
warning. Only if it is enabled but the whole firewall for that level is not.

Signed-off-by: Dominic Jäger 
---
Didn't exist in v1. Not sure if we want this. If yes you can just apply both or
squash. Of course, I can also send it as v3.

 www/manager6/grid/FirewallRules.js | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/www/manager6/grid/FirewallRules.js 
b/www/manager6/grid/FirewallRules.js
index 99f85fb6..9bf723e1 100644
--- a/www/manager6/grid/FirewallRules.js
+++ b/www/manager6/grid/FirewallRules.js
@@ -491,7 +491,8 @@ Ext.define('PVE.FirewallRules', {
Ext.Msg.alert(gettext('Error'), response.htmlStatus);
},
success: function (response) {
-   let warningRequired = me.store.getCount() != 0 && 
!response.result.data.enable;
+   let warningRequired = !response.result.data.enable
+   && me.store.findExact('enable', true) >= 0;

me.down('displayfield[name=fw-warning]').setVisible(warningRequired)
},
});
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH pve-zsync 1/1] pve-zsync: Flip Source and Dest in functions to so jobs can share Dest

2020-06-24 Thread Thomas Lamprecht
Am 6/16/20 um 8:53 PM schrieb Bruce Wainer:
> Signed-off-by: Bruce Wainer 
> ---
>  pve-zsync | 42 +-
>  1 file changed, 25 insertions(+), 17 deletions(-)
> 

With Wolfgangs T-b/R-b tags: applied, thanks to both!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied-series: [PATCH v7 0/4] add needed changes for backup detail view

2020-06-24 Thread Thomas Lamprecht
Am 6/22/20 um 4:34 PM schrieb Aaron Lauterer:
> The first part of this series which touched pve-manager has been applied
> with v6 [0][1].
> 
> Missing are the qemu-server and pve-container patches. With the last
> suggestions [2] incorporated it is also necessary to update the
> AbstractConfig.pm to keep the description of get_backup_volumes() in
> sync with the actual implementations.
> 
> [0] https://pve.proxmox.com/pipermail/pve-devel/2020-June/044000.html
> [1] https://pve.proxmox.com/pipermail/pve-devel/2020-June/044001.html
> [2] https://pve.proxmox.com/pipermail/pve-devel/2020-June/044009.html
> 
> qemu-server: Aaron Lauterer (1):
>   vzdump: move include logic for volumes to method
> 
>  PVE/QemuConfig.pm| 32 
>  PVE/VZDump/QemuServer.pm | 35 ---
>  2 files changed, 48 insertions(+), 19 deletions(-)
> 
> 
> container: Aaron Lauterer (2):
>   vzdump: add reason for mountpoint backup inclusion
>   vzdump: move include logic for mountpoints to method
> 
>  src/PVE/LXC/Config.pm | 48 ---
>  src/PVE/VZDump/LXC.pm | 37 +++--
>  2 files changed, 63 insertions(+), 22 deletions(-)
> 
> 
> guest-common: Aaron Lauterer (1):
>   Adapt description of get_backup_volumes
> 
>  PVE/AbstractConfig.pm | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 

applied series, thanks. Did some minor whitespace cleanups, also included the 
reason for
exclusion in the respective CTs loginfo call.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH pve-network] allow [ ,;] for ip lists

2020-06-24 Thread Alexandre DERUMIER
>>why not use PVE::Tools::split_list ? it's our standard helper for these 
>>kind of things, and also correctly trims whitespace and has support for 
>>\0-separated lists ;

I have take it from ceph code ;)


/usr/share/perl5/PVE/CephConfig.pm:my $monhosts = [ split (/[ ,;]+/, 
$config->{global}->{mon_host} // "") ];
/usr/share/perl5/PVE/API2/Ceph/MON.pm:  $monhost =~ s/(^|[ 
,;]*)\[$vectorpart_re(?:,$vectorpart_re)*\](?:[ ,;]+|$)/$1/;
/usr/share/perl5/PVE/API2/Ceph/MON.pm:  $monhost =~ s/(^|[ 
,;]+)\Q$addr\E(?::\d+)?(?:[ ,;]+|$)/$1/;
/usr/share/perl5/PVE/API2/Ceph/MON.pm:  $monhost =~ s/(^|[ 
,;]+)\Q$addr\E(?:[ ,;]+|$)/$1/;
/usr/share/perl5/PVE/API2/Ceph/MON.pm:  $monhost =~ s/[ ,;]+$//;


I'll look to use the PVE::Tools::split_list.


- Mail original -
De: "Fabian Grünbichler" 
À: "pve-devel" 
Envoyé: Mercredi 24 Juin 2020 10:23:14
Objet: Re: [pve-devel] [PATCH pve-network] allow [ ,;] for ip lists

why not use PVE::Tools::split_list ? it's our standard helper for these 
kind of things, and also correctly trims whitespace and has support for 
\0-separated lists ;) 

On June 12, 2020 6:14 pm, Alexandre Derumier wrote: 
> Signed-off-by: Alexandre Derumier  
> --- 
> PVE/Network/SDN/Controllers/EvpnPlugin.pm | 4 ++-- 
> PVE/Network/SDN/Zones/EvpnPlugin.pm | 2 +- 
> PVE/Network/SDN/Zones/VxlanPlugin.pm | 2 +- 
> 3 files changed, 4 insertions(+), 4 deletions(-) 
> 
> diff --git a/PVE/Network/SDN/Controllers/EvpnPlugin.pm 
> b/PVE/Network/SDN/Controllers/EvpnPlugin.pm 
> index 79ecaeb..8db2bed 100644 
> --- a/PVE/Network/SDN/Controllers/EvpnPlugin.pm 
> +++ b/PVE/Network/SDN/Controllers/EvpnPlugin.pm 
> @@ -47,11 +47,11 @@ sub options { 
> sub generate_controller_config { 
> my ($class, $plugin_config, $controller, $id, $uplinks, $config) = @_; 
> 
> - my @peers = split(',', $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'}; 
> + my @peers = split(/[ ,;]+/, $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'}; 
> 
> my $asn = $plugin_config->{asn}; 
> my $gatewaynodes = $plugin_config->{'gateway-nodes'}; 
> - my @gatewaypeers = split(',', $plugin_config->{'gateway-external-peers'}) 
> if $plugin_config->{'gateway-external-peers'}; 
> + my @gatewaypeers = split(/[ ,;]+/, 
> $plugin_config->{'gateway-external-peers'}) if 
> $plugin_config->{'gateway-external-peers'}; 
> 
> return if !$asn; 
> 
> diff --git a/PVE/Network/SDN/Zones/EvpnPlugin.pm 
> b/PVE/Network/SDN/Zones/EvpnPlugin.pm 
> index 95fbb64..dba3ffc 100644 
> --- a/PVE/Network/SDN/Zones/EvpnPlugin.pm 
> +++ b/PVE/Network/SDN/Zones/EvpnPlugin.pm 
> @@ -52,7 +52,7 @@ sub generate_sdn_config { 
> die "missing vxlan tag" if !$tag; 
> warn "vlan-aware vnet can't be enabled with evpn plugin" if 
> $vnet->{vlanaware}; 
> 
> - my @peers = split(',', $controller->{'peers'}); 
> + my @peers = split(/[ ,;]+/, $controller->{'peers'}); 
> my ($ifaceip, $iface) = 
> PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers); 
> 
> my $mtu = 1450; 
> diff --git a/PVE/Network/SDN/Zones/VxlanPlugin.pm 
> b/PVE/Network/SDN/Zones/VxlanPlugin.pm 
> index bc585c6..f2c2eec 100644 
> --- a/PVE/Network/SDN/Zones/VxlanPlugin.pm 
> +++ b/PVE/Network/SDN/Zones/VxlanPlugin.pm 
> @@ -50,7 +50,7 @@ sub generate_sdn_config { 
> my $ipv6 = $vnet->{ipv6}; 
> my $mac = $vnet->{mac}; 
> my $multicastaddress = $plugin_config->{'multicast-address'}; 
> - my @peers = split(',', $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'}; 
> + my @peers = split(/[ ,;]+/, $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'}; 
> my $vxlan_iface = "vxlan_$vnetid"; 
> 
> die "missing vxlan tag" if !$tag; 
> -- 
> 2.20.1 
> 
> ___ 
> pve-devel mailing list 
> pve-devel@pve.proxmox.com 
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> 

___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH v2 common 1/2] JSONSchema: add format validator support and cleanup check_format

2020-06-24 Thread Stefan Reiter

On 6/23/20 3:39 PM, Fabian Grünbichler wrote:

LGTM, what do you think about the following follow-up:

--8<-

diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm
index b2ba9f7..d28143d 100644
--- a/src/PVE/JSONSchema.pm
+++ b/src/PVE/JSONSchema.pm
@@ -1878,9 +1878,12 @@ sub generate_typetext {
  sub print_property_string {
  my ($data, $format, $skip, $path) = @_;
  
+my $validator;

  if (ref($format) ne 'HASH') {
my $schema = get_format($format);
die "not a valid format: $format\n" if !$schema;
+   # named formats can have validators attached
+   $validator = $format_validators->{$format};
$format = $schema;
  }
  
@@ -1890,6 +1893,8 @@ sub print_property_string {

raise "format error", errors => $errors;
  }
  
+$res = $validator->($res) if $validator;


This would have to be $data, I suppose?


+
  my ($default_key, $keyAliasProps) = &$find_schema_default_key($format);
  
  my $res = '';


->8--

to ensure our code calling 'print_property_string' also adheres to the
format the validator enforces?



Fine by me, though I'm not sure how necessary. It implies that 
validators always validate positively on values they have already 
accepted (and potentially modified) once before though. Also I'm not 
sure I would use the validators result for print_property_string, since 
that means that the value the user passes in might not be the one 
printed (if a validator modifies it) - so maybe call the validator but 
throw away it's result?



it might also be nice to mark formats with a validator (at least for the
API dump) to make it obvious that the displayed format might be further
restricted.



Sounds good to me, I'll look into it.


I went through the code paths (again) and still think it might be
worthwhile to extend named formats to check_object as well, instead of
just scalar property string values. but that is a bigger follow-up, for
now limiting the scope of these validators to just property strings
seems sensible.



___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH installer] fix #2804: add a root shell on tty3

2020-06-24 Thread Thomas Lamprecht
Am 6/19/20 um 4:25 PM schrieb Stoiko Ivanov:
> Tested locally in a VM:
> The setsid was necessary to give the bash job-control (otherwise Ctrl+C would
> simply kill the shell).
> 
> Signed-off-by: Stoiko Ivanov 
> ---
>  unconfigured.sh | 3 +++
>  1 file changed, 3 insertions(+)
> 

had quite a similar line recently for some debugging stuff. Applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] Package for vma seperately

2020-06-24 Thread Fabian Grünbichler
On June 5, 2020 1:33 pm, Mark Schouten wrote:
> 
> Hi!
> 
> I'm building Linux Images in Gitlab to distribute over customer clusters 
> easily. We have an (daDup.eu) s3 bucket where we place the vma-files and 
> mount that bucket via s3fs (works pretty well!). But, to create the VMA, I 
> need to install  pve-qemu-kvm, just to install the binary VMA, which in part 
> depends on about 300 packages.
> 
> I would be grateful if there would be a separate package for VMA, if 
> possible. :)

I don't think you'd gain much, since that separate 'vma' package would 
need to depend on most of the dependencies of pve-qemu-kvm as well (look 
at the output of ldd /usr/bin/vma to get an idea ;)).

the vma binary is just a thin wrapper around qemu functions after all.

the format is easy enough[1], so I guess in theory you could write your own 
'dump .vma to raw img' binary if you really really want to..

1: https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=vma_spec.txt

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH pve-network] allow [ ,;] for ip lists

2020-06-24 Thread Fabian Grünbichler
why not use PVE::Tools::split_list ? it's our standard helper for these 
kind of things, and also correctly trims whitespace and has support for 
\0-separated lists ;)

On June 12, 2020 6:14 pm, Alexandre Derumier wrote:
> Signed-off-by: Alexandre Derumier 
> ---
>  PVE/Network/SDN/Controllers/EvpnPlugin.pm | 4 ++--
>  PVE/Network/SDN/Zones/EvpnPlugin.pm   | 2 +-
>  PVE/Network/SDN/Zones/VxlanPlugin.pm  | 2 +-
>  3 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/PVE/Network/SDN/Controllers/EvpnPlugin.pm 
> b/PVE/Network/SDN/Controllers/EvpnPlugin.pm
> index 79ecaeb..8db2bed 100644
> --- a/PVE/Network/SDN/Controllers/EvpnPlugin.pm
> +++ b/PVE/Network/SDN/Controllers/EvpnPlugin.pm
> @@ -47,11 +47,11 @@ sub options {
>  sub generate_controller_config {
>  my ($class, $plugin_config, $controller, $id, $uplinks, $config) = @_;
>  
> -my @peers = split(',', $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'};
> +my @peers = split(/[ ,;]+/, $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'};
>  
>  my $asn = $plugin_config->{asn};
>  my $gatewaynodes = $plugin_config->{'gateway-nodes'};
> -my @gatewaypeers = split(',', 
> $plugin_config->{'gateway-external-peers'}) if 
> $plugin_config->{'gateway-external-peers'};
> +my @gatewaypeers = split(/[ ,;]+/, 
> $plugin_config->{'gateway-external-peers'}) if 
> $plugin_config->{'gateway-external-peers'};
>  
>  return if !$asn;
>  
> diff --git a/PVE/Network/SDN/Zones/EvpnPlugin.pm 
> b/PVE/Network/SDN/Zones/EvpnPlugin.pm
> index 95fbb64..dba3ffc 100644
> --- a/PVE/Network/SDN/Zones/EvpnPlugin.pm
> +++ b/PVE/Network/SDN/Zones/EvpnPlugin.pm
> @@ -52,7 +52,7 @@ sub generate_sdn_config {
>  die "missing vxlan tag" if !$tag;
>  warn "vlan-aware vnet can't be enabled with evpn plugin" if 
> $vnet->{vlanaware};
>  
> -my @peers = split(',', $controller->{'peers'});
> +my @peers = split(/[ ,;]+/, $controller->{'peers'});
>  my ($ifaceip, $iface) = 
> PVE::Network::SDN::Zones::Plugin::find_local_ip_interface_peers(\@peers);
>  
>  my $mtu = 1450;
> diff --git a/PVE/Network/SDN/Zones/VxlanPlugin.pm 
> b/PVE/Network/SDN/Zones/VxlanPlugin.pm
> index bc585c6..f2c2eec 100644
> --- a/PVE/Network/SDN/Zones/VxlanPlugin.pm
> +++ b/PVE/Network/SDN/Zones/VxlanPlugin.pm
> @@ -50,7 +50,7 @@ sub generate_sdn_config {
>  my $ipv6 = $vnet->{ipv6};
>  my $mac = $vnet->{mac};
>  my $multicastaddress = $plugin_config->{'multicast-address'};
> -my @peers = split(',', $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'};
> +my @peers = split(/[ ,;]+/, $plugin_config->{'peers'}) if 
> $plugin_config->{'peers'};
>  my $vxlan_iface = "vxlan_$vnetid";
>  
>  die "missing vxlan tag" if !$tag;
> -- 
> 2.20.1
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server] api: add option to get pending config returned as object instead of an array

2020-06-24 Thread Fabian Grünbichler
a bit of a rationale would be nice ;) isn't this just a simple map 
transformation that can be done client-side?

my $hash = { map { my $key = delete $_->{key}; return ($key => $_); } @$array };

or in whatever language you need. filtering/sorting/limiting server-side 
makes sense for some calls that might return a lot of data otherwise, 
but I don't think adapting all API calls to represent data with 
different structures but identical content is a good idea.

(not that I am too happy that we have so many 'array of object with key 
property' return values to accomodate ExtJS, when our backend and API 
usually carries the key as key and not part of the object)

On May 20, 2020 2:23 pm, Tim Marx wrote:
> Signed-off-by: Tim Marx 
> ---
>  PVE/API2/Qemu.pm | 13 +
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index fd51bf3..0b31f53 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -940,14 +940,20 @@ __PACKAGE__->register_method({
>   check => ['perm', '/vms/{vmid}', [ 'VM.Audit' ]],
>  },
>  parameters => {
> - additionalProperties => 0,
>   properties => {
>   node => get_standard_option('pve-node'),
>   vmid => get_standard_option('pve-vmid', { completion => 
> \::QemuServer::complete_vmid }),
> + object => {
> + description => "In this case the root element is an object 
> instead of an array.".
> +"The key property of the items will be extracted 
> and used as the root object keys.",
> + optional => 1,
> + default => 0,
> + type => 'boolean',
> + },
>   },
>  },
>  returns => {
> - type => "array",
> + type => [ "array", "object"],
>   items => {
>   type => "object",
>   properties => {
> @@ -985,8 +991,7 @@ __PACKAGE__->register_method({
>  
>   $conf->{cipassword} = '**' if defined($conf->{cipassword});
>   $conf->{pending}->{cipassword} = '** ' if 
> defined($conf->{pending}->{cipassword});
> -
> - return PVE::GuestHelpers::config_with_pending_array($conf, 
> $pending_delete_hash);
> + return PVE::GuestHelpers::config_with_pending($conf, 
> $pending_delete_hash, $param->{object});
> }});
>  
>  # POST/PUT {vmid}/config implementation
> -- 
> 2.20.1
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH manager] fix #2810: reset state of mounts array in initComponent

2020-06-24 Thread Fabian Grünbichler
On June 24, 2020 9:32 am, Dominik Csapak wrote:
> so that each new instance has an empty mounts list
> 
> Signed-off-by: Dominik Csapak 
> ---
> @fabian @oguz, i remembered that i know this issue and had a fix already^^
>  www/manager6/lxc/FeaturesEdit.js | 8 +++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/www/manager6/lxc/FeaturesEdit.js 
> b/www/manager6/lxc/FeaturesEdit.js
> index 1275a2e0..e0b851de 100644
> --- a/www/manager6/lxc/FeaturesEdit.js
> +++ b/www/manager6/lxc/FeaturesEdit.js
> @@ -108,7 +108,13 @@ Ext.define('PVE.lxc.FeaturesInputPanel', {
>   }
>   this.callParent([res]);
>   }
> -}
> +},
> +
> +initComponent: function() {
> + let me = this;
> + me.mounts = []; // reset state
> + me.callParent();
> +},
>  });
>  
>  Ext.define('PVE.lxc.FeaturesEdit', {
> -- 
> 2.20.1
> 
> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server 3/3] fix #2794: allow legacy IGD passthrough

2020-06-24 Thread Thomas Lamprecht
Am 6/22/20 um 10:17 AM schrieb Stefan Reiter:
>>> @@ -89,7 +97,8 @@ sub get_pci_addr_map {
>>>   $pci_addr_map = {
>>>   piix3 => { bus => 0, addr => 1, conflict_ok => qw(ehci)  },
>>>   ehci => { bus => 0, addr => 1, conflict_ok => qw(piix3) }, # instead 
>>> of piix3 on arm
>>> -vga => { bus => 0, addr => 2 },
>>> +vga => { bus => 0, addr => 2, conflict_ok => qw(legacy-igd) },
>>
>> sorry, but how do they conflict? the address differs so the check can't hit.
>> Or is this just for documentation purpose?
>>
> 
> What do you mean the address differs? 'legacy-igd' and 'vga' both share bus 0 
> addr 2, so without the conflict_ok the 'run_pci_addr_checks.pl' test fails. 
> This is only save because legacy-igd requires vga=none anyway, as documented 
> by the comment below.
> 
>>> +'legacy-igd' => { bus => 0, addr => 2, conflict_ok => qw(vga) }, # 
>>> legacy-igd requires vga=none 

ignore me, confused with non legacy IGD, sorry. 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager] fix #2810: reset state of mounts array in initComponent

2020-06-24 Thread Dominik Csapak
so that each new instance has an empty mounts list

Signed-off-by: Dominik Csapak 
---
@fabian @oguz, i remembered that i know this issue and had a fix already^^
 www/manager6/lxc/FeaturesEdit.js | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/www/manager6/lxc/FeaturesEdit.js b/www/manager6/lxc/FeaturesEdit.js
index 1275a2e0..e0b851de 100644
--- a/www/manager6/lxc/FeaturesEdit.js
+++ b/www/manager6/lxc/FeaturesEdit.js
@@ -108,7 +108,13 @@ Ext.define('PVE.lxc.FeaturesInputPanel', {
}
this.callParent([res]);
}
-}
+},
+
+initComponent: function() {
+   let me = this;
+   me.mounts = []; // reset state
+   me.callParent();
+},
 });
 
 Ext.define('PVE.lxc.FeaturesEdit', {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] RFC: sdn: add ip management (IPAM -DHCP) ideas

2020-06-24 Thread dietmar
> >>Do you want to allocate IPs on VM creation time, or VM start time?
> 
> I think at vm creation time, or nic hotplug/unplug.
> the ipam api is called, return free ip address, and we write somewhere in vm 
> config the ip address.
> (for nic hotplug/unplug, we need to handle ipam removal on config revert)
> 
> Like this, user can generate cloudinit if needed with the ip address, can do 
> custom firewall rules, we could generate dhcp... before starting the vm.

yes, I would do it at VM creation time (and any time we creathe a new nic 
device). 
That way we can store the allocated IP in the VM nic config.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] RFC: sdn: add ip management (IPAM -DHCP) ideas

2020-06-24 Thread Alexandre DERUMIER
>>Do you want to allocate IPs on VM creation time, or VM start time?

I think at vm creation time, or nic hotplug/unplug.
the ipam api is called, return free ip address, and we write somewhere in vm 
config the ip address.
(for nic hotplug/unplug, we need to handle ipam removal on config revert)

Like this, user can generate cloudinit if needed with the ip address, can do 
custom firewall rules, we could generate dhcp... before starting the vm.




- Mail original -
De: "dietmar" 
À: "aderumier" 
Cc: "pve-devel" 
Envoyé: Mercredi 24 Juin 2020 07:42:07
Objet: Re: [pve-devel] RFC: sdn: add ip management (IPAM -DHCP) ideas

> >>You you also do not store the cidr there, and instead 
> >>store the some pool ID retured by IPAM? 
> 
> cidr should be the key/id of the subnet. Almost all ipam use the cidr as key. 

Ok, that makes sense now. 

Do you want to allocate IPs on VM creation time, or VM start time? 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel