Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Dear Michel, ... but the ssh connection is continous open -- before and "during" the issue? Did you sniff (e.g. with tcpdump) the package flow on and outside (e.g. from your testing host)? Guido >-Original Message- >From: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] On Behalf >Of Michel Jansens >Sent: Friday, June 08, 2018 8:58 AM >To: LXC users mailing-list >Subject: Re: [lxc-users] Network instability with bridged nat and macvlan >interfaces > > > >Hi Guido, > >Thanks for your reply > >I’ve installed an apache2 on port 8082, and it falls at the same time as >haproxy ports 80 and 443. Only ssh keeps responding. >Weird! > >Michel > > > > > > On 8 Jun 2018, at 08:15, Jäkel, Guido <mailto:g.jae...@dnb.de> > wrote: > > Dear Michel, > > did you already take a look on the other parts of the involved network > environment? Maybe you have an issue on layer two >vs. three concerning the MAC <-> IP correlation on the involved next upstream >switch. You may check the ARP tables. > > And -- because you "loose" port 80 and 443, but not 22 --- as a test I > would arrange some other simple services (using >another product as you use for the httpd). > ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Hi Guido, Thanks for your reply I’ve installed an apache2 on port 8082, and it falls at the same time as haproxy ports 80 and 443. Only ssh keeps responding. Weird! Michel > On 8 Jun 2018, at 08:15, Jäkel, Guido wrote: > > Dear Michel, > > did you already take a look on the other parts of the involved network > environment? Maybe you have an issue on layer two vs. three concerning the > MAC <-> IP correlation on the involved next upstream switch. You may check > the ARP tables. > > And -- because you "loose" port 80 and 443, but not 22 --- as a test I would > arrange some other simple services (using another product as you use for the > httpd). > > Greetings > > Guido > >> -Original Message- >> From: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] On >> Behalf Of Michel Jansens >> Sent: Thursday, June 07, 2018 7:36 PM >> To: LXC users mailing-list >> Subject: Re: [lxc-users] Network instability with bridged nat and macvlan >> interfaces >> >> Hi Andrey, >> Thank you for your answer. >> I’ll try to avoid mixing macvlan with bridging/nat to test. >> I’m currently building the equivalent on a second server, but with a bridge >> built on top of the vlan. >> Somebody at Canonical also suggested it could be the physical switch playing >> bad with macvlan. We’re investigating. >> I’ll keep you informed of the evolution. >> >> Cheers, >> >> Michel > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Dear Michel, did you already take a look on the other parts of the involved network environment? Maybe you have an issue on layer two vs. three concerning the MAC <-> IP correlation on the involved next upstream switch. You may check the ARP tables. And -- because you "loose" port 80 and 443, but not 22 --- as a test I would arrange some other simple services (using another product as you use for the httpd). Greetings Guido >-Original Message- >From: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] On Behalf >Of Michel Jansens >Sent: Thursday, June 07, 2018 7:36 PM >To: LXC users mailing-list >Subject: Re: [lxc-users] Network instability with bridged nat and macvlan >interfaces > >Hi Andrey, >Thank you for your answer. >I’ll try to avoid mixing macvlan with bridging/nat to test. >I’m currently building the equivalent on a second server, but with a bridge >built on top of the vlan. >Somebody at Canonical also suggested it could be the physical switch playing >bad with macvlan. We’re investigating. >I’ll keep you informed of the evolution. > >Cheers, > >Michel ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Hi Andrey, Thank you for your answer. I’ll try to avoid mixing macvlan with bridging/nat to test. I’m currently building the equivalent on a second server, but with a bridge built on top of the vlan. Somebody at Canonical also suggested it could be the physical switch playing bad with macvlan. We’re investigating. I’ll keep you informed of the evolution. Cheers, Michel > On 7 Jun 2018, at 17:34, Andrey Repin wrote: > > Greetings, Michel Jansens! > >> I’m running on Ubuntu18.04 LXC 3.0.0. > >> I’ve created 5 debian9 containers with default eth0 networking on NAT: > >> # lxc network show lxdbr0 >> config: >> ipv4.address: 10.1.1.1/24 >> ipv4.dhcp.ranges: 10.1.1.2-10.1.1.99 >> ipv4.nat: "true" >> ipv6.address: fd42:6f79:c120:7701::1/64 >> ipv6.nat: "true" >> description: Natted network 0 >> name: lxdbr0 >> type: bridge > >> One of the containers (frontal) has an additional interface configured with: > >> # lxc network attach vlan7 frontal >> # lxc config show kspreprodfrontal >> … >> devices: >> vlan7: >>nictype: macvlan >>parent: vlan7 >>type: nic > >> vlan7 is a flan with id: 7 configured in /etc/netplan/01-netcfg.yaml >> ... >> vlans: >>vlan7: >> id: 7 >> link: enp1s0f0 > > I'm no expert, frankly, but it itching me to mix brctl and macvlan like that. > >> I’ve changed the frontal host internal networking so that eth1 comes first >> and default route is going through eth1. >> Everything works internal and external…except from time to time, the >> frontal starts refusing connexions from the outside for a few seconds (up to >> 50). >> It looks like general networking because all ports suddenly stop working >> (connexion refused) >> internally the frontal remains reachable >> I’m running haproxy on ports 80 and 443, but also tried running apache2 on >> port 8082. All ports go down at the same time. > >> I’ve now installed an Ubuntu (16.04) container and added the vlan7 network >> the same way. >> It worked fine…for about an hour and stopped working again, but for good. >> What is weird is that port 80 and 443 are refused but port 22 is working >> (maybe that’s the host ssh?). > >> Any idea? > > Your explanation is not very clear in parts where you describe the failure. > >> Thanks for any suggestion. > > My first suggestion would be to rebuild your networking a little bit > different. > > 1. Create a dummy internal interface and bind your containers' macvlan bridges > to it. Bind an additional bridged macvlan on host to be able to reach into > the containers' network. > 2. If your vlan7 is a dedicate network interface for your containers, pass it > as physical to the ingress container. > > > -- > With best regards, > Andrey Repin > Thursday, June 7, 2018 18:26:48 > > Sorry for my terrible english... > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Greetings, Michel Jansens! > I’m running on Ubuntu18.04 LXC 3.0.0. > I’ve created 5 debian9 containers with default eth0 networking on NAT: > # lxc network show lxdbr0 > config: > ipv4.address: 10.1.1.1/24 > ipv4.dhcp.ranges: 10.1.1.2-10.1.1.99 > ipv4.nat: "true" > ipv6.address: fd42:6f79:c120:7701::1/64 > ipv6.nat: "true" > description: Natted network 0 > name: lxdbr0 > type: bridge > One of the containers (frontal) has an additional interface configured with: > # lxc network attach vlan7 frontal > # lxc config show kspreprodfrontal > … > devices: > vlan7: > nictype: macvlan > parent: vlan7 > type: nic > vlan7 is a flan with id: 7 configured in /etc/netplan/01-netcfg.yaml > ... > vlans: > vlan7: > id: 7 > link: enp1s0f0 I'm no expert, frankly, but it itching me to mix brctl and macvlan like that. > I’ve changed the frontal host internal networking so that eth1 comes first > and default route is going through eth1. > Everything works internal and external…except from time to time, the > frontal starts refusing connexions from the outside for a few seconds (up to > 50). > It looks like general networking because all ports suddenly stop working > (connexion refused) > internally the frontal remains reachable > I’m running haproxy on ports 80 and 443, but also tried running apache2 on > port 8082. All ports go down at the same time. > I’ve now installed an Ubuntu (16.04) container and added the vlan7 network > the same way. > It worked fine…for about an hour and stopped working again, but for good. > What is weird is that port 80 and 443 are refused but port 22 is working > (maybe that’s the host ssh?). > Any idea? Your explanation is not very clear in parts where you describe the failure. > Thanks for any suggestion. My first suggestion would be to rebuild your networking a little bit different. 1. Create a dummy internal interface and bind your containers' macvlan bridges to it. Bind an additional bridged macvlan on host to be able to reach into the containers' network. 2. If your vlan7 is a dedicate network interface for your containers, pass it as physical to the ingress container. -- With best regards, Andrey Repin Thursday, June 7, 2018 18:26:48 Sorry for my terrible english... ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Hi Andrey, I don’t understand what you mean by hijack unrelated threads. I just created a mail with the title "Network instability with bridged nat and macvlan interfaces” Or did I miss something? Sorry if I did. Cheers, Michel > On 6 Jun 2018, at 19:54, Andrey Repin wrote: > > Greetings, Michel Jansens! > > Please don't hijack unrelated threads. If you want to post a new issue, post a > new message. > > > -- > With best regards, > Andrey Repin > Wednesday, June 6, 2018 20:54:31 > > Sorry for my terrible english... > > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Greetings, Michel Jansens! Please don't hijack unrelated threads. If you want to post a new issue, post a new message. -- With best regards, Andrey Repin Wednesday, June 6, 2018 20:54:31 Sorry for my terrible english... ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Network instability with bridged nat and macvlan interfaces
Hi again, In the meantime, I’ve installed an Ubuntu (16.04) container and added the vlan7 network the same way. It worked fine…for about an hour and stopped working again, but for good. What is weird is that port 80 and 443 are refused but port 22 is working (maybe that’s the host ssh?). Michel > On 6 Jun 2018, at 16:51, Michel Jansens wrote: > > Hi, > > I’m running on Ubuntu18.04 LXC 3.0.0. > > I’ve created 5 debian9 containers with default eth0 networking on NAT: > > # lxc network show lxdbr0 > config: > ipv4.address: 10.1.1.1/24 > ipv4.dhcp.ranges: 10.1.1.2-10.1.1.99 > ipv4.nat: "true" > ipv6.address: fd42:6f79:c120:7701::1/64 > ipv6.nat: "true" > description: Natted network 0 > name: lxdbr0 > type: bridge > > One of the containers (frontal) has an additional interface configured with: > > # lxc network attach vlan7 frontal > # lxc config show kspreprodfrontal > … > devices: > vlan7: > nictype: macvlan > parent: vlan7 > type: nic > > vlan7 is a flan with id: 7 configured in /etc/netplan/01-netcfg.yaml > ... > vlans: > vlan7: > id: 7 > link: enp1s0f0 > > I’ve changed the frontal host internal networking so that eth1 comes first > and default route is going through eth1. > Everything works internal and external…except from time to time, the frontal > starts refusing connexions from the outside for a few seconds (up to 50). > It looks like general networking because all ports suddenly stop working > (connexion refused) > internally the frontal remains reachable > I’m running haproxy on ports 80 and 443, but also tried running apache2 on > port 8082. All ports go down at the same time. > > > Any idea? > > Thanks for any suggestion. > > Cheers, > > Michel > > > > > > > > > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
