Cool! The problem seems to be:
GRE (VXLAN too?) tunnel of Data Network (between Instance and its namespace router) doesn't work with default mtu=1500. ??? It was fixed and there are two solutions for this problem (as I'm seeing... I didn't patched ovs_lib.py), as follows: 1- (Easier / + generic?) Lower the mtu to 1450 for Instances or; 2- (Recommended?) Enable Jumbo Frames for Data Network (your IPv4 subnet where OVS GRE/VXLAN tunnel flows). Both solutions have fixed the issue for me. I'm sticking with Jumbo Frames, problem fixed (plus performance boost? I'll test it more)... About 1- To lower the mtu for the Instances, add the following line to dhcp_agent.ini: --- dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf --- And write dnsmasq-neutron.conf with: --- dhcp-option-force=26,1450 --- About 2- Enable Jumbo Frames on interfaces X / Y (of "Data Network" / "local_ip var subnet")... I think that the IceHouse install document should add a note about this issue, maybe here: http://docs.openstack.org/icehouse/install-guide/install/apt/content/neutron-ml2-network-node.html Should I fill a documentation bug? Thanks! Thiago On 20 May 2014 17:56, McCann, Jack <[email protected]> wrote: > Yes. Personally I’d lean toward jumbo frames if you have that > flexibility. > > > > - Jack > > > > *From:* Martinx - ジェームズ [mailto:[email protected]] > *Sent:* Tuesday, May 20, 2014 2:36 PM > *To:* McCann, Jack > > *Subject:* Re: [Openstack] IceHouse Neutron L3 - Connectivity issue > > > > BTW Jack, do you think that, lowering the MTU for the Instances (via > dhcp_agent) and / or enabling jumbo-frames for DATA NETWORK (GRE / VXLAN > tunnels), might also help to solve this issue?! > > > > - > > Thiago > > > > On 20 May 2014 14:36, Martinx - ジェームズ <[email protected]> wrote: > > Cool! I'll try that tonight... Tks! > > > > On 20 May 2014 14:00, McCann, Jack <[email protected]> wrote: > > We ran into an issue (change of default behavior) with GRE/VxLAN tunnel > > fragmentation when we moved to the 3.13 kernel. One workaround for this > > issue is to set "options:df_default=false" on the ovs tunnels, see [1]. > > Fragmenting those packets is not ideal, but it would be interesting to > > see if this is the problem you’re running into. > > > > - Jack > > > > [1] > https://review.openstack.org/#/c/75281/9/neutron/agent/linux/ovs_lib.py > > > > *From:* Martinx - ジェームズ [mailto:[email protected]] > *Sent:* Tuesday, May 20, 2014 12:03 PM > *To:* Jason Bishop > *Cc:* [email protected] > *Subject:* Re: [Openstack] IceHouse Neutron L3 - Connectivity issue > > > > Apparently we're all facing the same issue. The fact is, Neutron is too > much complex... > > > > Back with Havana, we faced this: > https://bugs.launchpad.net/neutron/+bug/1252900 > > > > But now, with Ubuntu 14.04 + IceHouse, seems to be a different problem. > > > > "Neutron + GRE or VXLAN", is very hard to stabilize and doesn't scale very > well... > > > > I'm seeing that it works better with "VLAN / Flat Networks" (i.e. without > using the Neutron Network Node as Instance's default gateway). I'll try it > instead... > > > > Maybe James Denton, from Rackspace, might help us again! :-P > > > > Best! > > Thiago > > > > On 20 May 2014 11:21, Jason Bishop <[email protected]> wrote: > > > > that happened to me too. maybe same root cause? > > > > see http://ubuntuforums.org/showthread.php?t=2223789 > > > > cheers > > jason > > > > > > On Mon, May 19, 2014 at 10:59 PM, Martinx - ジェームズ < > [email protected]> wrote: > > I managed to login into a Ubuntu Instance via `SPICE Console` and there > is more network outages from within it, take a look ("apt-get update" froze > too): > > > > -- > > http://i.imgur.com/IeEXIjR.png > > -- > > > > Am I missing something?! > > > > On 20 May 2014 02:47, Martinx - ジェームズ <[email protected]> wrote: > > Hey Stackers! > > > > I'm facing a weird network issue here, with *IceHouse* and *Neutron L3*... > Using "Per-Project Router with Private Networks"... > > > > I am unable to connect via SSH into an Instance, from its own "Project > Namespace Router", look: > > > > -- > > root@net-controller-1:~# ip netns exec > qrouter-c252886a-1fd0-45dd-9e3b-4233786e7960 ssh -i ~/pem > [email protected] -v > > OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014 > > debug1: Reading configuration data /etc/ssh/ssh_config > > debug1: /etc/ssh/ssh_config line 19: Applying options for * > > debug1: Connecting to 192.168.1.17 [192.168.1.17] port 22. > > debug1: Connection established. > > debug1: permanently_set_uid: 0/0 > > debug1: identity file /root/pem type -1 > > debug1: identity file /root/pem-cert type -1 > > debug1: Enabling compatibility mode for protocol 2.0 > > debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 > > debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6p1 > Ubuntu-2ubuntu1 > > debug1: match: OpenSSH_6.6p1 Ubuntu-2ubuntu1 pat OpenSSH_6.5*,OpenSSH_6.6* > compat 0x14000000 > > debug1: SSH2_MSG_KEXINIT sent > > ... > > ... > > ...ssh stucked!!! > > ... > > -- > > > > The SSH connection froze at "debug1: SSH2_MSG_KEXINIT sent" and it never > establishes... > > > > From its attached `Floating IP`, the symptom is the same, I can't connect > via SSH into a Instance, I tried it with 12.04 and 14.04, same result for > both... > > > > The Instance looks good from SPICE Consoles (but I need to login with > "ubuntu user" via network first, using ssh key). > > > > - > > > > I'm running tcpdump on both "qr-$INT" and at the Compute Node DATA > NETWORK, to see the GREv0 traffic too, here it is: > > > > -- > > root@net-controller-1:~# ip netns exec > qrouter-c252886a-1fd0-45dd-9e3b-4233786e7960 tcpdump -v -ni qr-eeb0d2f3-42 > > > > http://paste.openstack.org/show/80969/ > > > > NOTE: From the line ~16 to 32, SSH was already frozen... > > -- > > > > root@compute-node-1:~# tcpdump -ni eth1 | grep -v IP6 > > > > http://paste.openstack.org/show/80970/ > > -- > > > > What can I do to deep investigate this?! > > > > I double checked everything, including OVS bridges, sysctl.conf, `ethtool > --offload ethX gro off` and etc... Don't know what's is wrong... > > > > I think that this is the last problem I'm facing with IceHouse, I would > like to put it into prod but, I am unable to do it right now... :-/ > > > > Tks in advance! > > > > Best, > > Thiago > > > > > > _______________________________________________ > Mailing list: > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : [email protected] > Unsubscribe : > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > > > > > > > >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
