[pve-devel] [PATCH pve-firewall] ebtables: add arp filtering

2019-03-08 Thread Alexandre Derumier
This implemented arp filtering if ipfilter is enable https://bugzilla.proxmox.com/show_bug.cgi?id=2125 They are another filters possible (ipv4,rarp), i don't known if we need them. Signed-off-by: Alexandre Derumier --- src/PVE/Firewall.pm | 42 +- 1 file

Re: [pve-devel] HA: vm shutdown/stop behaviour and other HA questions

2019-03-06 Thread Alexandre DERUMIER
quot;pve-devel" , "aderumier" Envoyé: Mercredi 6 Mars 2019 08:21:06 Objet: Re: [pve-devel] HA: vm shutdown/stop behaviour and other HA questions Hi! On 3/6/19 7:59 AM, Alexandre DERUMIER wrote: > Hi, > > I'm finally going use HA on my cluster when proxmox 6.0 will be

Re: [pve-devel] HA: vm shutdown/stop behaviour and other HA questions

2019-03-06 Thread Alexandre DERUMIER
days ago ;) https://bugzilla.proxmox.com/show_bug.cgi?id=2115 - Mail original - De: "datanom.net" À: "pve-devel" Envoyé: Mercredi 6 Mars 2019 08:06:34 Objet: Re: [pve-devel] HA: vm shutdown/stop behaviour and other HA questions On Wed, 6 Mar 2019 07:59:14

[pve-devel] HA: vm shutdown/stop behaviour and other HA questions

2019-03-05 Thread Alexandre DERUMIER
Hi, I'm finally going use HA on my cluster when proxmox 6.0 will be released (waiting for corosync 3.X). and, I have notice than shutdown or stop on vm, call both "HA stop" , which call "vm shutdown" then stop HA. Is is possible to implement a true "vm stop" without shutdown ? Also, I have

Re: [pve-devel] rfc : /etc/pve/networks.cfg implementation

2019-03-01 Thread Alexandre DERUMIER
Ok, I'll begin to code, and we'll see what's the best way ! - Mail original - De: "dietmar" À: "aderumier" Cc: "pve-devel" Envoyé: Vendredi 1 Mars 2019 09:39:33 Objet: Re: [pve-devel] rfc : /etc/pve/networks.cfg implementation > Maybe could we reuse pvestatd ? maybe > maybe we

Re: [pve-devel] rfc : /etc/pve/networks.cfg implementation

2019-02-28 Thread Alexandre DERUMIER
t: Re: [pve-devel] rfc : /etc/pve/networks.cfg implementation > On February 28, 2019 at 9:20 AM Alexandre DERUMIER > wrote: > > > >>Or just activate when needed (at VM start)? But yes, a separate config is > >>preferable. > > Another thing is if we want to

Re: [pve-devel] rfc : /etc/pve/networks.cfg implementation

2019-02-28 Thread Alexandre DERUMIER
>>Or just activate when needed (at VM start)? But yes, a separate config is >>preferable. Another thing is if we want to update config. (change multicast address, add a new unicast node,), when the vm are already running. - Mail original - De: "aderumier" À: "dietmar" Cc:

Re: [pve-devel] rfc : /etc/pve/networks.cfg implementation

2019-02-27 Thread Alexandre DERUMIER
>>Not sure if we need those extra switch settings? yes, indeed, I think something like vnet[0-4096] could be better, Can't we combine >>switch and transportzones? i.e. >> >>vnet1: vxlanfrr >>name: zone4 # not really required >>transportzone zone4 >>... >>l3vni:

[pve-devel] rfc : /etc/pve/networks.cfg implementation

2019-02-27 Thread Alexandre DERUMIER
Hi Dietmar, I'll work next week on /etc/pve/networks.cfg, I have take time to polish the configs file, I'll would to have some feedback before coding. 1) add transportzone in /etc/network/interface. only on physical interfaces (eth/bond), not tagged interfaces. This is only an hint,

Re: [pve-devel] [PATCH pve-firewall] add nf_conntrack_tcp_be_liberal option

2019-02-21 Thread Alexandre DERUMIER
mier" Cc: "pve-devel" Envoyé: Jeudi 21 Février 2019 09:13:36 Objet: Re: [pve-devel] [PATCH pve-firewall] add nf_conntrack_tcp_be_liberal option On Wed, Feb 20, 2019 at 11:05:50AM +0100, Alexandre Derumier wrote: > Allow to not flag as invalid, out of windows packets. > > Can

[pve-devel] [PATCH pve-firewall] add nf_conntrack_tcp_be_liberal option

2019-02-20 Thread Alexandre Derumier
Allow to not flag as invalid, out of windows packets. Can't persist in sysctl at boot, because if need to be set after conntrack loading. Signed-off-by: Alexandre Derumier --- src/PVE/Firewall.pm | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/src/PVE

Re: [pve-devel] [PATCH pve-firewall] service : cleanup + character at begin of line

2019-02-20 Thread Alexandre DERUMIER
et: Re: [pve-devel] [PATCH pve-firewall] service : cleanup + character at begin of line On 2/20/19 1:16 AM, Alexandre Derumier wrote: > --- > src/PVE/Service/pve_firewall.pm | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/PVE/Service/pve_firewall.pm

[pve-devel] [PATCH qemu-server] QemuMigrate : cleanup identation

2019-02-19 Thread Alexandre Derumier
--- PVE/QemuMigrate.pm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 07d57ed..d33b8ff 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -579,10 +579,10 @@ sub phase2 { $rport = int($1);

[pve-devel] [PATCH v6 qemu-server 2/3] add QemuMigrateExternal.pm

2019-02-19 Thread Alexandre Derumier
--- PVE/Makefile | 1 + PVE/QemuMigrateExternal.pm | 872 + 2 files changed, 873 insertions(+) create mode 100644 PVE/QemuMigrateExternal.pm diff --git a/PVE/Makefile b/PVE/Makefile index 2c800f6..0494cfb 100644 --- a/PVE/Makefile +++

[pve-devel] [PATCH v6 qemu-server 0/3] online vm migration to external cluster

2019-02-19 Thread Alexandre Derumier
: put code in patch3 + remove old no more used migration_type=external option Alexandre Derumier (3): api2 : add migrate_vm_external add QemuMigrateExternal.pm qemu : vm_start : add external_migration PVE/API2/Qemu.pm | 106 +- PVE/CLI/qm.pm | 2 + PVE/Makefile

[pve-devel] [PATCH v6 qemu-server 1/3] api2 : add migrate_vm_external

2019-02-19 Thread Alexandre Derumier
qm migrate_external [--targetstorage otherstorage] [--net[n] [,firewall=<1|0>] [,link_down=<1|0>][,rate=] [,tag=] [,trunks=]] and ssh private key must exist in /etc/pve/priv/external_migration/id_rsa_targetremotenode_fqdn_or_ip --- PVE/API2/Qemu.pm | 90

[pve-devel] [PATCH v6 qemu-server 3/3] qemu : vm_start : add external_migration

2019-02-19 Thread Alexandre Derumier
--- PVE/API2/Qemu.pm | 16 ++-- PVE/QemuServer.pm | 20 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 1f57643..60638d1 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1970,7 +1970,12 @@

[pve-devel] [PATCH pve-firewall] service : cleanup + character at begin of line

2019-02-19 Thread Alexandre Derumier
--- src/PVE/Service/pve_firewall.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Service/pve_firewall.pm b/src/PVE/Service/pve_firewall.pm index 5a0dd04..e70f6e5 100755 --- a/src/PVE/Service/pve_firewall.pm +++ b/src/PVE/Service/pve_firewall.pm @@ -170,7 +170,7 @@

Re: [pve-devel] [PATCH pve-firewall 1/3] global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP

2019-02-19 Thread Alexandre DERUMIER
y think we need some kind of protection to avoid that. - Mail original - De: "Thomas Lamprecht" À: "pve-devel" , "Alexandre Derumier" , "Wolfgang Bumiller" Envoyé: Mardi 19 Février 2019 11:07:42 Objet: Re: [pve-devel] [PATCH pve-firewall 1/3] global -m connt

Re: [pve-devel] [PATCH pve-firewall 1/3] global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP

2019-02-18 Thread Alexandre DERUMIER
rumier" Cc: "pve-devel" Envoyé: Lundi 18 Février 2019 10:56:42 Objet: Re: [pve-devel] [PATCH pve-firewall 1/3] global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP On Fri, Feb 15, 2019 at 10:48:01AM +0100, Alexandre Derumier wrote: > --- > src/PVE/Firew

Re: [pve-devel] [PATCH pve-firewall 1/3] global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP

2019-02-18 Thread Alexandre DERUMIER
ot;pve-devel" Envoyé: Lundi 18 Février 2019 10:56:42 Objet: Re: [pve-devel] [PATCH pve-firewall 1/3] global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP On Fri, Feb 15, 2019 at 10:48:01AM +0100, Alexandre Derumier wrote: > --- > src/PVE/Firewall.pm | 2 +- > 1 fil

[pve-devel] [PATCH pve-firewall] remove_pvefw_chains : flush conntrack

2019-02-15 Thread Alexandre Derumier
avoid invalid state if we reenable firewall later --- debian/control | 3 ++- src/PVE/Firewall.pm | 4 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/debian/control b/debian/control index a38250d..33fc666 100644 --- a/debian/control +++ b/debian/control @@ -15,7 +15,8 @@

[pve-devel] [PATCH pve-firewall 1/3] global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP

2019-02-15 Thread Alexandre Derumier
--- src/PVE/Firewall.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 2125d3b..61d5599 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -2137,7 +2137,7 @@ sub ruleset_chain_add_conn_filters { my ($ruleset,

[pve-devel] [PATCH pve-firewall 2/3] PVEFW-reject ipv4 : remove icmp drop rule

2019-02-15 Thread Alexandre Derumier
or we can't never match the next icmp reject rules --- src/PVE/Firewall.pm | 1 - 1 file changed, 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 61d5599..7890b51 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -568,7 +568,6 @@ $pve_std_chains_conf->{4} =

[pve-devel] [PATCH pve-firewall 3/3] remove -m conntrack --ctstate INVALID from PVEFW-Drop/PVEFW-Reject chains

2019-02-15 Thread Alexandre Derumier
We are check it on top of rules --- src/PVE/Firewall.pm | 8 1 file changed, 8 deletions(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 7890b51..8e057be 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -583,8 +583,6 @@ $pve_std_chains_conf->{4} = {

[pve-devel] [PATCH pve-firewall 0/3] improve conntrack INVALID handling

2019-02-15 Thread Alexandre Derumier
This fix connection breaking/hanging Alexandre Derumier (3): global -m conntrack --ctstate INVALID : PVEFW-reject instead DROP PVEFW-reject ipv4 : remove icmp drop rule remove -m conntrack --ctstate INVALID from PVEFW-Drop/PVEFW-Reject chains src/PVE/Firewall.pm | 11 +-- 1

Re: [pve-devel] pve-firewall : -m conntrack --ctstate INVALID -j DROP is dropping connections on firewall restart

2019-02-15 Thread Alexandre DERUMIER
INVALID -j DROP iptables -A FORWARD -m conntrack --ctstate INVALID -j INVALID That's way, my cephfs client get the RST, and reestablish connection. (same for ssh). - Mail original - De: "Alexandre Derumier" À: "pve-devel" Envoyé: Vendredi 15 Février 2019 08:22

Re: [pve-devel] pve-firewall : -m conntrack --ctstate INVALID -j DROP is dropping connections on firewall restart

2019-02-14 Thread Alexandre DERUMIER
than flushing conntrack after remove rules should be fine, if we reenable firewall, and vm have already an established connection, it should fill the conntrack again. (this is the same with vm live migration) - Mail original - De: "Alexandre Derumier" À: "pve-devel&q

Re: [pve-devel] pve-firewall : vm live migration: rules applied only after vm config file move

2019-02-14 Thread Alexandre DERUMIER
- De: "Paul Chambers" À: "Alexandre Derumier" Cc: "pve-devel" Envoyé: Jeudi 14 Février 2019 20:28:29 Objet: Re: [pve-devel] pve-firewall : vm live migration: rules applied only after vm config file move Perhaps using Network Namespaces would help? I'd like to see Proxmox

[pve-devel] pve-firewall : -m conntrack --ctstate INVALID -j DROP is dropping connections on firewall restart

2019-02-14 Thread Alexandre DERUMIER
Hi, I have notice a bug with current firewall implementation, with a simple test like: 1) host fw: iptables -F iptables -X iptables -A FORWARD -m conntrack --ctstate INVALID -j DROP iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT iptables-save > rules.dump

Re: [pve-devel] pve-firewall : vm live migration: rules applied only after vm config file move

2019-02-13 Thread Alexandre DERUMIER
>>(currently, if a CT or VM is booting fast, it's also possible to have some >>seconds with open firewall) sorry,that's wrong. The rules exist if the config file is present (vm started or stopped) - Mail original - De: "aderumier" À: "dietmar" Cc: "pve-devel" Envoyé: Mercredi 13

Re: [pve-devel] pve-firewall : vm live migration: rules applied only after vm config file move

2019-02-13 Thread Alexandre DERUMIER
>>Maybe live migration can tell firewall on target node to activate rules >>before we start migration. But I am not sure >>how to implement that. I think it should be done at vm/ct start, force firewall to activate rules before launching qemu or lxc. Like this we can be sure than rules are

Re: [pve-devel] Plan to add "Add Sound Card"

2019-02-07 Thread Alexandre DERUMIER
>>Is there any plan to put a resource in Add to add sound card? patches has been sent last month (not yet commited), to enable soundcard with spice https://pve.proxmox.com/pipermail/pve-devel/2019-January/035190.html (do you need it without spice ?) - Mail original - De: "Gilberto

Re: [pve-devel] roadmap for proxmox 6 ?

2019-02-05 Thread Alexandre DERUMIER
rive anymore, ... So this need a lot of tests. - Mail original - De: "Thomas Lamprecht" À: "pve-devel" , "Alexandre Derumier" Envoyé: Mardi 5 Février 2019 17:38:16 Objet: Re: [pve-devel] roadmap for proxmox 6 ? Hi! Am 2/5/19 um 2:47 PM schrieb Alexa

[pve-devel] roadmap for proxmox 6 ?

2019-02-05 Thread Alexandre DERUMIER
Hi, as debian buster is coming, do you have already a roadmap for proxmox6 ? Here some features I have in mind: - corosync 3 - qemu 3.x - ifupdown2 by default ? (need to test openvswitch) - I'll try to work on vxlan (+frr) /vlan, /etc/pve/networks.cfg. I'll try to send prelimaries patches

[pve-devel] [PATCH pve-firewall] log reject : add space after policy REJECT like drop

2019-02-05 Thread Alexandre Derumier
For log consistency and parsing, we already have a space after "policy DROP: " but not REJECT ex: DROP 135 6 tap135i1-IN 05/Feb/2019:10:59:55 +0100 policy DROP: IN=. REJECT 232 6 tap232i1-IN 05/Feb/2019:10:59:28 +0100 policy REJECT:IN= --- src/PVE/Firewall.pm | 2 +- 1 file changed, 1

Re: [pve-devel] [PATCH v5 qemu-server 0/3] online vm migration to external cluster

2019-02-04 Thread Alexandre DERUMIER
Hi, any comments for the v5 ? - Mail original - De: "Alexandre Derumier" À: "pve-devel" Cc: "Alexandre Derumier" Envoyé: Mardi 29 Janvier 2019 02:20:37 Objet: [PATCH v5 qemu-server 0/3] online vm migration to external cluster This add support to migra

Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

2019-02-04 Thread Alexandre DERUMIER
>>Did you test it without modifying the patch or only with your syslog change? I have only change my rsyslog config to drop pve-firewall log if $programname == 'pve-firewall' then stop - Mail original - De: "David Limbeck" À: "Alexandre Derumier" , &

Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

2019-01-30 Thread Alexandre DERUMIER
. Please build and install it again with that commit included and run it again. This might help narrow it down some more. On 1/14/19 11:42 AM, Alexandre DERUMIER wrote: > Hi, > > I have able to reproduce, after 1hour. > > I have enable debug to get it run in foreground. > &g

Re: [pve-devel] [PATCH v2 pve-firewall 0/5] improve update

2019-01-30 Thread Alexandre DERUMIER
you know, I'll go through your patches this week and probably adjust them, as Thomas suggested in his response, anyway I will keep you up to date. Thanks! > Alexandre Derumier hat am 14. Januar 2019 um 10:15 > geschrieben: > > > This patch improve firewall update. > We

[pve-devel] [PATCH v5 qemu-server 3/3] qemu : vm_start : add external_migration

2019-01-28 Thread Alexandre Derumier
--- PVE/API2/Qemu.pm | 16 ++-- PVE/QemuServer.pm | 20 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 97bae46..897aa0f 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1951,7 +1951,12 @@

[pve-devel] [PATCH v5 qemu-server 0/3] online vm migration to external cluster

2019-01-28 Thread Alexandre Derumier
al targetkey && targetvmid - migrate : add a new QemuMigrationExternal.pm with all migration code + remove some part not used, like sync_disk and zfs replication. - migrate : force ssh tunnel for migration - vm_start : put code in patch3 + remove old no more used migration_type=external option

[pve-devel] [PATCH v5 qemu-server 2/3] add QemuMigrateExternal.pm

2019-01-28 Thread Alexandre Derumier
--- PVE/Makefile | 1 + PVE/QemuMigrateExternal.pm | 872 + 2 files changed, 873 insertions(+) create mode 100644 PVE/QemuMigrateExternal.pm diff --git a/PVE/Makefile b/PVE/Makefile index 2c800f6..0494cfb 100644 --- a/PVE/Makefile +++

[pve-devel] [PATCH v5 qemu-server 1/3] api2 : add migrate_vm_external

2019-01-28 Thread Alexandre Derumier
qm migrate_external [--targetstorage otherstorage] [--targetvmid] [--targetkey] --net[n] [,bridge=] [,firewall=<1|0>] [,link_down=<1|0>] [,rate=] [,tag=] [,trunks=] - node is a ip or fqdn host from another cluster. OPTIONS: - targetstorage : allow to choose a different storage

Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

2019-01-25 Thread Alexandre DERUMIER
commit was pushed that enables building of debug symbols for pve-firewall. Please build and install it again with that commit included and run it again. This might help narrow it down some more. On 1/14/19 11:42 AM, Alexandre DERUMIER wrote: > Hi, > > I have able to reproduce, aft

Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

2019-01-14 Thread Alexandre DERUMIER
On Fri, Jan 11, 2019 at 06:05:36PM +0100, Alexandre DERUMIER wrote: > >>Do you have any additional information as to why it stopped? > > no sorry. > > >>Maybe we could increase the buffer size via nfnl_set_rcv_buffer_size by > >>default and continue to ignore ENOBU

[pve-devel] [PATCH v2 pve-firewall 2/5] remove_pvefw_chains_iptables : don't commit if rules are already removed.

2019-01-14 Thread Alexandre Derumier
currently, we commit at each update empty rules if they are already removed. --- src/PVE/Firewall.pm | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 94f1bd0..b2d055a 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm

[pve-devel] [PATCH v2 pve-firewall 5/5] update: read config twice at 1second interval

2019-01-14 Thread Alexandre Derumier
We want to be sure that all configs (cluster,host,vmconfig,vmdata) don't have change during an update, to have consistency. --- debian/control | 1 + src/PVE/Firewall.pm | 9 + 2 files changed, 10 insertions(+) diff --git a/debian/control b/debian/control index 2a92b16..c28353e

[pve-devel] [PATCH v2 pve-firewall 0/5] improve update

2019-01-14 Thread Alexandre Derumier
all configs twice at 1second interval, and compare values to be sure of consistency Alexandre Derumier (5): don't update if /etc/pve is not mounted remove_pvefw_chains_iptables : don't commit if rules are already removed. add syslog on pvefw chains removal split read_configuration

[pve-devel] [PATCH v2 pve-firewall 3/5] add syslog on pvefw chains removal

2019-01-14 Thread Alexandre Derumier
--- src/PVE/Firewall.pm | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index b2d055a..e092671 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -4157,8 +4157,10 @@ sub remove_pvefw_chains_iptables { $cmdlist .=

[pve-devel] [PATCH v2 pve-firewall 4/5] split read_configuration from compile sub

2019-01-14 Thread Alexandre Derumier
--- src/PVE/Firewall.pm | 15 ++- src/PVE/Service/pve_firewall.pm | 10 ++ test/fwtester.pl| 3 ++- 3 files changed, 18 insertions(+), 10 deletions(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index e092671..f738dba 100644 ---

[pve-devel] [PATCH v2 pve-firewall 1/5] don't update if /etc/pve is not mounted

2019-01-14 Thread Alexandre Derumier
--- src/PVE/Firewall.pm | 2 ++ src/pvefw-logger.c | 6 -- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 39f79d4..94f1bd0 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -4186,6 +4186,8 @@ sub init { sub update {

Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

2019-01-11 Thread Alexandre DERUMIER
itional information as to why it stopped? Maybe we could increase the buffer size via nfnl_set_rcv_buffer_size by default and continue to ignore ENOBUFS? On 1/10/19 4:32 PM, Alexandre DERUMIER wrote: > Just tested, no difference. (but I don't see ENOBUFS since I have increase > net.ipv4.

Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

2019-01-10 Thread Alexandre DERUMIER
Just tested, no difference. (but I don't see ENOBUFS since I have increase net.ipv4.tcp_rmem) But I have reproduce my new hang, and it seem that the pvefw-logger process was not running anymore. (seem to be a crash, I don't seen any out of memory). - Mail original - De: "Thomas

Re: [pve-devel] [PATCH pve-firewall 1/3] don't update if /etc/pve is not mounted

2019-01-10 Thread Alexandre DERUMIER
t;aderumier" Envoyé: Mercredi 9 Janvier 2019 15:15:23 Objet: Re: [pve-devel] [PATCH pve-firewall 1/3] don't update if /etc/pve is not mounted On 1/9/19 2:56 PM, Alexandre Derumier wrote: > --- > src/PVE/Firewall.pm | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git

Re: [pve-devel] [PATCH firewall] log and ignore ENOBUFS in nfct_catch

2019-01-09 Thread Alexandre DERUMIER
t;proxy, and proxy->servers) That's a lot of new connections by seconds. I have also increase my sysctl rmem, it's possible that ENOBUFS come from here. (and I never notice it before) Thanks you again for your great work ! - Mail original - De: "Thomas Lamprecht" À: &qu

Re: [pve-devel] [PATCH firewall] log and ignore ENOBUFS in nfct_catch

2019-01-09 Thread Alexandre DERUMIER
Ok, it's correctly continue to work after the error message now. But I still have hang after that (after some seconds, or minutes). Any error message in this case. (This is a really busy server, I have around 400MB log for 10minutes) cat /var/log/pve-firewall.log |grep -c NEW 1465965 # cat

Re: [pve-devel] [PATCH firewall] print error message in case of 'error catching nfct'

2019-01-09 Thread Alexandre DERUMIER
error catching nfct: No buffer space available - Mail original - De: "David Limbeck" À: "pve-devel" Envoyé: Mercredi 9 Janvier 2019 14:37:27 Objet: [pve-devel] [PATCH firewall] print error message in case of 'error catching nfct' Signed-off-by: David Limbeck --- I couldn't

[pve-devel] [PATCH pve-firewall 0/3] improve update

2019-01-09 Thread Alexandre Derumier
* check if /etc/pve is mounted to avoid rules removal when pve-cluster restart * don't commit pvefw chains removal at each update if already removed * add syslog for pvefw chain removal Alexandre Derumier (3): don't update if /etc/pve is not mounted remove_pvefw_chains_iptables : don't

[pve-devel] [PATCH pve-firewall 2/3] remove_pvefw_chains_iptables : don't commit if rules are already removed.

2019-01-09 Thread Alexandre Derumier
currently, we commit at each update empty rules if they are already removed. --- src/PVE/Firewall.pm | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 71327b0..1a9be89 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm

[pve-devel] [PATCH pve-firewall 1/3] don't update if /etc/pve is not mounted

2019-01-09 Thread Alexandre Derumier
--- src/PVE/Firewall.pm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 39f79d4..71327b0 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -4186,6 +4186,9 @@ sub init { sub update { my $code = sub { +eval {

[pve-devel] [PATCH pve-firewall 3/3] add syslog on pvefw chains removal

2019-01-09 Thread Alexandre Derumier
--- src/PVE/Firewall.pm | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index 1a9be89..dca1103 100644 --- a/src/PVE/Firewall.pm +++ b/src/PVE/Firewall.pm @@ -4157,8 +4157,10 @@ sub remove_pvefw_chains_iptables { $cmdlist .=

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-09 Thread Alexandre DERUMIER
break my firewalled connections in my vms - Mail original - De: "Thomas Lamprecht" À: "pve-devel" , "aderumier" Envoyé: Mercredi 9 Janvier 2019 09:49:44 Objet: Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rul

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-09 Thread Alexandre DERUMIER
19 09:49:44 Objet: Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ? On 1/9/19 9:17 AM, Thomas Lamprecht wrote: > On 1/9/19 8:36 AM, Alexandre DERUMIER wrote: >>>> Hmm, but if one wants to restore the defaults by simply deleting the fil

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-08 Thread Alexandre DERUMIER
e-devel" , "aderumier" , "Stefan Priebe, Profihost AG" Envoyé: Mercredi 9 Janvier 2019 08:16:46 Objet: Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ? On 1/8/19 10:19 PM, Alexandre DERUMIER wrote: >>> or those

Re: [pve-devel] [PATCH qemu-server v2 0/3] Fix #2041 and #413

2019-01-08 Thread Alexandre DERUMIER
Hi, they are also ich9-intel-hda as sound controller. (maybe when q35 is used) could be great to use intel hda for linux too, and not only modern windows. - Mail original - De: "Andreas Steinel" À: "pve-devel" Envoyé: Mardi 8 Janvier 2019 23:57:58 Objet: [pve-devel] [PATCH

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-08 Thread Alexandre DERUMIER
t;aderumier" , "Thomas Lamprecht" Envoyé: Mardi 8 Janvier 2019 21:59:44 Objet: Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ? Hi Alexandre, Am 08.01.19 um 21:55 schrieb Alexandre DERUMIER: >>> But, file_set_contents -

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-08 Thread Alexandre DERUMIER
l : possible bug/race when cluster.fw is replicated and rules are updated ? Hi, On 1/8/19 7:37 PM, Alexandre DERUMIER wrote: > I'm able to reproduce with: > --- > on 1 host: > > cluster.fw: > [OPTIONS] > > enable: 1 > policy_in: ACCEPT >

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-08 Thread Alexandre DERUMIER
I'm able to reproduce with: --- on 1 host: cluster.fw: [OPTIONS] enable: 1 policy_in: ACCEPT #!/usr/bin/perl use IO::File; use PVE::Firewall; use Data::Dumper; use Time::HiRes qw ( time alarm sleep usleep ); while(1){ $filename = "/etc/pve/firewall/cluster.fw";

[pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

2019-01-08 Thread Alexandre DERUMIER
Hi, I'm currently debugging a possible firewalling problem. I'm running some cephfs client in vm, firewalled by proxmox. cephfs client are really sensitive to network problem, and mainly with packets logss or dropped packets. I'm really not sure, but I have currently puppet updating my

[pve-devel] [PATCH v4 qemu-server 0/3] online vm migration to external cluster

2019-01-07 Thread Alexandre Derumier
e + remove some part not used, like sync_disk and zfs replication. - migrate : force ssh tunnel for migration - vm_start : put code in patch3 + remove old no more used migration_type=external option *** BLURB HERE *** Alexandre Derumier (3): api2 : add migrate_vm_external add QemuMigrateEx

[pve-devel] [PATCH v4 qemu-server 3/3] qemu : vm_start : add external_migration

2019-01-07 Thread Alexandre Derumier
--- PVE/API2/Qemu.pm | 16 ++-- PVE/QemuServer.pm | 20 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index b74f111..2f79a2b 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1951,7 +1951,12 @@

[pve-devel] [PATCH v4 qemu-server 2/3] add QemuMigrateExternal.pm

2019-01-07 Thread Alexandre Derumier
--- PVE/Makefile | 1 + PVE/QemuMigrateExternal.pm | 866 + 2 files changed, 867 insertions(+) create mode 100644 PVE/QemuMigrateExternal.pm diff --git a/PVE/Makefile b/PVE/Makefile index 2c800f6..0494cfb 100644 --- a/PVE/Makefile +++

Re: [pve-devel] applied: [PATCH frr 0/5] bump to 6.0.1

2019-01-04 Thread Alexandre DERUMIER
oy, Rhonda P.S.: I submitted the issue I found in the frr.postinst file as pull request directly to upstream: https://github.com/FRRouting/frr/pull/3565 On 12/31/18 10:29 AM, Alexandre Derumier wrote: > Update frr package to 6.0.1. > (This need to update frr_mirror to frr-6.0.1

Re: [pve-devel] applied: [PATCH v4 firewall 0/2] firewall conntrack logging

2019-01-04 Thread Alexandre DERUMIER
Hi David, I haved tested with real production traffic, and I have log hanf with "error catching nfct" example: [DESTROY] ipv4 2 tcp 6 src=78.130.61.209 dst=10.11.1.11 sport=62386 dport=443 packets=15 bytes=2283 src=10.11.1.11 dst=78.130.61.209 sport=443 dport=62386 packets=9

[pve-devel] [PATCH ifupdown2 2/3] changelog : dump to 1.2.2

2018-12-31 Thread Alexandre Derumier
--- debian/changelog | 6 ++ 1 file changed, 6 insertions(+) diff --git a/debian/changelog b/debian/changelog index 10fb621..ed147b5 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +ifupdown2 (1.2.2-1+pvetest1) unstable; urgency=medium + + * Update to 1.2.2 + + --

[pve-devel] [PATCH ifupdown2 1/3] Makefile : bump to 1.2.2

2018-12-31 Thread Alexandre Derumier
--- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 0955b62..91cb755 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ PACKAGE=ifupdown2 -VER=1.2.0 +VER=1.2.2 PKGREL=1+pvetest1 SRCDIR=ifupdown2 -- 2.11.0

[pve-devel] [PATCH ifupdown2 3/3] update config tuning patch

2018-12-31 Thread Alexandre Derumier
/0001-config-tuning.patch +++ b/debian/patches/pve/0001-config-tuning.patch @@ -1,43 +1,34 @@ -From Mon Sep 17 00:00:00 2001 +From 8b2e7b50a36dfada8d1e04412326c3d8e4ad9266 Mon Sep 17 00:00:00 2001 From: Alexandre Derumier -Date: Mon, 14 May 2018 16:36:58 +0200

[pve-devel] [PATCH ifupdown2 0/3] bump to 1.2.2

2018-12-31 Thread Alexandre Derumier
This need to update mirror_ifupdown2 to tag 1.2.2-1 Alexandre Derumier (3): Makefile : bump to 1.2.2 changelog : dump to 1.2.2 update config tuning patch Makefile| 2 +- debian/changelog| 6 + debian/patches/pve/0001

[pve-devel] [PATCH frr 5/5] changelog : bump to 6.0.1

2018-12-31 Thread Alexandre Derumier
Derumier Mon, 31 Dec 2018 09:00:00 +0200 + frr (6.0-0+pve) RELEASED; urgency=medium [ Alexandre Derumier ] -- 2.11.0 ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH frr 2/5] update debian directory

2018-12-31 Thread Alexandre Derumier
- frr 6.0.1 have fully reworked the debian packaging --- debian/.gitignore | 1 - debian/Makefile.am| 47 - debian/README.Debian | 97 - debian/README.Maintainer |

[pve-devel] [PATCH frr 3/5] control: remove libjson0 from build depend

2018-12-31 Thread Alexandre Derumier
--- debian/control | 1 - 1 file changed, 1 deletion(-) diff --git a/debian/control b/debian/control index d353039..0ed8bc2 100644 --- a/debian/control +++ b/debian/control @@ -15,7 +15,6 @@ Build-Depends: install-info, libc-ares-dev, libcap-dev, - libjson0, libjson0-dev | libjson-c-dev,

[pve-devel] [PATCH frr 1/5] Makefile : bump to 6.0.1

2018-12-31 Thread Alexandre Derumier
also, - debianpkg in frr repo is now debian - use DEB_BUILD_PROFILES=pkg.frr.nortrlib as librtr-dev is only in sid --- Makefile | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index 18861d0..18e994f 100644 --- a/Makefile +++ b/Makefile @@ -1,5

[pve-devel] [PATCH frr 0/5] bump to 6.0.1

2018-12-31 Thread Alexandre Derumier
Update frr package to 6.0.1. (This need to update frr_mirror to frr-6.0.1 tag) Alexandre Derumier (5): Makefile : bump to 6.0.1 update debian directory control: remove libjson0 from build depend add 0001-enable-bgp-daemon.patch changelog : bump to 6.0.1 Makefile

[pve-devel] [PATCH frr 4/5] add 0001-enable-bgp-daemon.patch

2018-12-31 Thread Alexandre Derumier
/debian/patches/pve/0001-enable-bgp-daemon.patch @@ -0,0 +1,25 @@ +From 13d0e8ea85fbbc4d53709a0fb28ebd66876884b4 Mon Sep 17 00:00:00 2001 +From: Alexandre Derumier +Date: Mon, 31 Dec 2018 09:50:06 +0100 +Subject: [PATCH] enable bgp daemon + +--- + tools/etc/frr/daemons | 2 +- + 1 file changed, 1

Re: [pve-devel] [PATCH v3 qemu-server 0/7] online vm migration to external cluster

2018-12-21 Thread Alexandre DERUMIER
On 12/19/18 11:07 AM, Alexandre DERUMIER wrote: >>> hat do you think about splitting the code path completely from the >>> current migration path? This would allow us to work on it without any >>> modification to the current migration code. Makes testing easier and we >>

Re: [pve-devel] [PATCH v3 qemu-server 4/7] migrate : phase2 : migrate external

2018-12-19 Thread Alexandre DERUMIER
8 10:35:46 Objet: Re: [pve-devel] [PATCH v3 qemu-server 4/7] migrate : phase2 : migrate external comments inline On 11/27/18 4:38 PM, Alexandre Derumier wrote: > --- > PVE/API2/Qemu.pm | 18 +++--- > PVE/QemuMigrate.pm | 21 ++--- > PVE/QemuServer.pm | 20

Re: [pve-devel] [PATCH v3 qemu-server 0/7] online vm migration to external cluster

2018-12-19 Thread Alexandre DERUMIER
r on (and combine common functionality again) some more comments follow on the respective patches On 11/27/18 4:38 PM, Alexandre Derumier wrote: > This add support to migrate a vm online to an different external cluster. > (This a rework of a previous patch series sent 2years ag

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-14 Thread Alexandre DERUMIER
always use the zone id as prefix for vlan/vxlan devices. Its simply to implement and avid problems in future. Although most people will only have only one zone? > On December 13, 2018 at 11:46 AM Alexandre DERUMIER > wrote: > > > looking at kernel code in this patch >

Re: [pve-devel] applied: [PATCH v4 firewall 0/2] firewall conntrack logging

2018-12-13 Thread Alexandre DERUMIER
Thanks you very much ! Just tested, works fine. If somebody is interested, I have build logstash parser + elastic template + kibana dashboards - Mail original - De: "Wolfgang Bumiller" À: "David Limbeck" Cc: "pve-devel" Envoyé: Jeudi 13 Décembre 2018 14:34:32 Objet: [pve-devel]

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-13 Thread Alexandre DERUMIER
for that, or is that just a bug? I really don't know if it's in vxlan rfc or not. Maybe because we have 16 million vxlan-id, they don't have implemented it in the kernel. - Mail original - De: "dietmar" À: "aderumier" Cc: "pve-devel" Envoyé: Je

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-13 Thread Alexandre DERUMIER
ot;pve-devel" Envoyé: Jeudi 13 Décembre 2018 10:03:52 Objet: Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing > On December 13, 2018 at 9:12 AM Alexandre DERUMIER > wrote: > > > >>I just noticed that can have v(x)lan IDs multiple times, > >>on

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-13 Thread Alexandre DERUMIER
>>I just noticed that can have v(x)lan IDs multiple times, >>once for each transport zone? So we need a better >>naming scheme, for example: >>vxlan2 in zone1 => z1vxlan2 >>vxlan2 in zone2 => z2vxlan2 it's not possible to have 2 vxlan vtep with same vxlan-id. auto vxlan2 iface vxlan2 inet

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-12 Thread Alexandre DERUMIER
used on this transport zone (this is what vmware is doing) It's more if we want to give differents permissions, - root admin role can define transport zones with allowed vlan/vxlan. - customer can add his own vxlan in this transport zone - Mail original - De: "dietmar" À: "A

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-12 Thread Alexandre DERUMIER
>>That "transport zone" looks interesting. >>We you just mark physical interfaces (or bridges?) as part of a transport >>zone. >>Then we have everything to setup the vxlan (each vlan belong to a zone)? yes, it should work. (interface could be better I think). /etc/network/interfaces iface

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-11 Thread Alexandre DERUMIER
>>Another way, could be make somekind of template on each local host. (as we >>only need to duplicate them for each vlan/vxlan). for vlan, - keep using an vmbr a template auto vmbr0 iface vmbr0 inet manual bridge-ports eno1 bridge-stp off bridge-fd 0 Maybe

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-11 Thread Alexandre DERUMIER
>>IMHO this looks like a hack - I wonder how VMware associates the global net >>to >>local devices on the host? From what I have seen, the wizard when you create global net, is able to do local configuration of all hosts interfaces too. (and associate them to global switch). Another way,

[pve-devel] [PATCH v2 qemu-server] fix #1013 : migrate : sync_disk : --targetstorage with offline disk

2018-12-10 Thread Alexandre Derumier
targetsid was not used, for disk unused (offline copy) --- PVE/QemuMigrate.pm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index e9e9075..0bae076 100644 --- a/PVE/QemuMigrate.pm +++ b/PVE/QemuMigrate.pm @@ -446,6 +446,7 @@ sub

Re: [pve-devel] [PATCH pve-docs 0/1] vxlan l3 routing

2018-12-10 Thread Alexandre DERUMIER
Hi Dietmar, I'll have time to work again on /etc/pve/network.cfg idea. I don't known if you have some time to check my idea about using ifupdown2 "alias" BTW,talking with students on last training (a lot coming from vmware world), they think it could be great to be able to define global

[pve-devel] [PATCH pve-docs] vxlan : add missing vxlan-id

2018-12-10 Thread Alexandre Derumier
--- vxlan-and-evpn.adoc | 51 +++ 1 file changed, 51 insertions(+) diff --git a/vxlan-and-evpn.adoc b/vxlan-and-evpn.adoc index 5115d78..bd0bd72 100644 --- a/vxlan-and-evpn.adoc +++ b/vxlan-and-evpn.adoc @@ -51,6 +51,7 @@ iface vmbr0 inet static

<    4   5   6   7   8   9   10   11   12   13   >