I think you are looking at the wrong file.. if you are using ovs you need to look at that… example:
[root@os-poc-controller-0 ml2]# grep -i mtu openvswitch_agent.ini # MTU size of veth interfaces (integer value) #veth_mtu = 9000 > On Dec 18, 2017, at 20:06, Manuel Sopena Ballesteros > <manuel...@garvan.org.au> wrote: > > Yes, <> > > This is my neutron ml2 config on the compute node > > [ml2] > type_drivers = flat,vlan,vxlan > tenant_network_types = vxlan > path_mtu = 9000 > mechanism_drivers = openvswitch,l2population > extension_drivers = port_security > > [ml2_type_vlan] > network_vlan_ranges = > > [ml2_type_flat] > flat_networks = physnet1 > > [ml2_type_vxlan] > vni_ranges = 1:1000 > vxlan_group = 239.1.1.1 > > [securitygroup] > firewall_driver = > neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver > > [agent] > tunnel_types = vxlan > l2_population = true > arp_responder = true > > [ovs] > ovsdb_connection = tcp:127.0.0.1:6640 > local_ip = 10.0.32.23 > > thank you > > Manuel > > From: Remo Mattei [mailto:r...@italy1.com <mailto:r...@italy1.com>] > Sent: Tuesday, December 19, 2017 12:56 PM > To: Manuel Sopena Ballesteros > Cc: openstack@lists.openstack.org <mailto:openstack@lists.openstack.org> > Subject: Re: [Openstack] need help understanding neutron/OVS > > Did you change the mtu on neutron config? > > Sent from my iPad > > On Dec 18, 2017, at 5:38 PM, Manuel Sopena Ballesteros > <manuel...@garvan.org.au <mailto:manuel...@garvan.org.au>> wrote: > > Dear Openstack community, > > Openstack environment: > > 2 compute nodes each has: > · 1x AMD Opteron(tm) Processor 6282 SE (56 cpus) > · 512GB RAM > · 1x dual port 25Gbps Mellanox connect-4 XOR bond > > Physical network: > · Compute nodes connected through a mellanox 2700 switch (100Gbps) > · MTU 9000 > > Openstack network: > · OVS + VxLan > · Vxlan offloading setup on physical nics > · MTU on vms 8950 > > Compute nodes nics: > > [root@hercules-23 ~]# ip a > … > 6: p2p1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master > bond0 state UP qlen 1000 > link/ether 7c:fe:90:12:23:c4 brd ff:ff:ff:ff:ff:ff > 7: p2p2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master > bond0 state UP qlen 1000 > link/ether 7c:fe:90:12:23:c4 brd ff:ff:ff:ff:ff:ff > 8: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue > state UP qlen 1000 > link/ether 7c:fe:90:12:23:c4 brd ff:ff:ff:ff:ff:ff > inet 10.0.32.23/16 brd 10.0.255.255 scope global bond0 > valid_lft forever preferred_lft forever > inet6 fe80::3826:9daa:ef1b:43b0/64 scope link > valid_lft forever preferred_lft forever > … > > > VM nics: > > [centos@centos2 ~]$ ip a > … > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc pfifo_fast state UP > qlen 1000 > link/ether fa:16:3e:a3:d4:85 brd ff:ff:ff:ff:ff:ff > inet 192.168.1.116/24 brd 192.168.1.255 scope global dynamic eth0 > valid_lft 57885sec preferred_lft 57885sec > inet6 fe80::f816:3eff:fea3:d485/64 scope link > valid_lft forever preferred_lft forever > > > host to host bandwidth using muttcp (1 thread): > > [root@hercules-22 ~]# nuttcp -l65536 -fparse 10.0.32.23 > megabytes=19844.1759 real_seconds=10.00 rate_Mbps=16643.4189 tx_cpu=61 > rx_cpu=99 retrans=0 rtt_ms=0.28 > > VMTP Output: > > ================== > Total Scenarios: 29 > Passed Scenarios: 17 [100.00%] > Failed Scenarios: 0 [0.00%] > Skipped Scenarios: 12 > +----------+-------------------------------------------------------+-------------------+----------------------------------------------------------------------------------+ > | Scenario | Scenario Name | > Functional Status | Data > | > +----------+-------------------------------------------------------+-------------------+----------------------------------------------------------------------------------+ > | 1.1 | Same Network, Fixed IP, Intra-node, TCP | PASSED > | {'tp_kbps': '7613130', 'rtt_ms': '0.463333'} > | > | 1.2 | Same Network, Fixed IP, Intra-node, UDP | PASSED > | {128: {'tp_kbps': 59954, 'loss_rate': 0.02}, 1024: {'tp_kbps': > 444264, | > | | | > | 'loss_rate': 0.05}, 8192: {'tp_kbps': 2877674, 'loss_rate': 0.0}} > | > | 1.3 | Same Network, Fixed IP, Intra-node, ICMP | PASSED > | {'rtt avg/min/max/stddev msec': {'391-byte': > '0.376/0.285/0.439/0.047', | > | | | > | '64-byte': '0.617/0.361/1.513/0.388', '1500-byte': > '0.369/0.322/0.452/0.048'}} | > | 1.4 | Same Network, Fixed IP, Intra-node, Multicast | SKIPPED > | {} > | > | 2.1 | Same Network, Fixed IP, Inter-node, TCP | PASSED > | {'tp_kbps': '6864266', 'rtt_ms': '0.653333'} > | > | 2.2 | Same Network, Fixed IP, Inter-node, UDP | PASSED > | {128: {'tp_kbps': 80955, 'loss_rate': 0.46}, 1024: {'tp_kbps': > 166186, | > | | | > | 'loss_rate': 36.2}, 8192: {'tp_kbps': 2189389, 'loss_rate': 0.05}} > | > | 2.3 | Same Network, Fixed IP, Inter-node, ICMP | PASSED > | {'rtt avg/min/max/stddev msec': {'391-byte': > '0.484/0.315/0.561/0.068', | > | | | > | '64-byte': '0.719/0.421/2.714/0.666', '1500-byte': > '0.465/0.358/0.553/0.055'}} | > | 2.4 | Same Network, Fixed IP, Inter-node, Multicast | SKIPPED > | {} > | > | 3.1 | Different Network, Fixed IP, Intra-node, TCP | PASSED > | {'tp_kbps': '5207719', 'rtt_ms': '0.896667'} > | > | 3.2 | Different Network, Fixed IP, Intra-node, UDP | PASSED > | {128: {'tp_kbps': 76569, 'loss_rate': 0.9}, 1024: {'tp_kbps': > 571847, | > | | | > | 'loss_rate': 2.75}, 8192: {'tp_kbps': 1855166, 'loss_rate': 0.0}} > | > | 3.3 | Different Network, Fixed IP, Intra-node, ICMP | PASSED > | {'rtt avg/min/max/stddev msec': {'391-byte': > '0.859/0.658/1.103/0.133', | > | | | > | '64-byte': '0.976/0.576/1.946/0.361', '1500-byte': > '0.824/0.644/1.082/0.136'}} | > | 3.4 | Different Network, Fixed IP, Intra-node, Multicast | SKIPPED > | {} > | > | 4.1 | Different Network, Fixed IP, Inter-node, TCP | PASSED > | {'tp_kbps': '6730928', 'rtt_ms': '1.37667'} > | > | 4.2 | Different Network, Fixed IP, Inter-node, UDP | PASSED > | {128: {'tp_kbps': 31131, 'loss_rate': 4.84}, 1024: {'tp_kbps': > 579859, | > | | | > | 'loss_rate': 4.35}, 8192: {'tp_kbps': 1601092, 'loss_rate': 0.18}} > | > | 4.3 | Different Network, Fixed IP, Inter-node, ICMP | PASSED > | {'rtt avg/min/max/stddev msec': {'391-byte': > '0.915/0.740/1.138/0.144', | > | | | > | '64-byte': '1.134/0.713/2.210/0.539', '1500-byte': > '0.980/0.719/1.188/0.131'}} | > | 4.4 | Different Network, Fixed IP, Inter-node, Multicast | SKIPPED > | {} > | > | 5.1 | Different Network, Floating IP, Intra-node, TCP | PASSED > | {'tp_kbps': '5406554', 'rtt_ms': '0.836667'} > | > | 5.2 | Different Network, Floating IP, Intra-node, UDP | PASSED > | {128: {'tp_kbps': 71084, 'loss_rate': 4.26}, 1024: {'tp_kbps': > 507900, | > | | | > | 'loss_rate': 15.96}, 8192: {'tp_kbps': 2011414, 'loss_rate': > 3.72}} | > | 5.3 | Different Network, Floating IP, Intra-node, ICMP | PASSED > | {'rtt avg/min/max/stddev msec': {'391-byte': > '0.902/0.760/1.007/0.073', | > | | | > | '64-byte': '0.875/0.661/0.967/0.104', '1500-byte': > '0.889/0.754/0.988/0.082'}} | > | 5.4 | Different Network, Floating IP, Intra-node, Multicast | SKIPPED > | {} > | > | 6.1 | Different Network, Floating IP, Inter-node, TCP | PASSED > | {'tp_kbps': '6770995', 'rtt_ms': '0.973333'} > | > | 6.2 | Different Network, Floating IP, Inter-node, UDP | PASSED > | {128: {'tp_kbps': 81812, 'loss_rate': 2.25}, 1024: {'tp_kbps': > 617485, | > | | | > | 'loss_rate': 3.27}, 8192: {'tp_kbps': 1295351, 'loss_rate': 0.02}} > | > | 6.3 | Different Network, Floating IP, Inter-node, ICMP | PASSED > | {'rtt avg/min/max/stddev msec': {'391-byte': > '1.028/0.920/1.254/0.105', | > | | | > | '64-byte': '1.043/0.807/1.227/0.137', '1500-byte': > '0.988/0.759/1.194/0.146'}} | > | 6.4 | Different Network, Floating IP, Inter-node, Multicast | SKIPPED > | {} > | > | 7.1 | Native Throughput, TCP | SKIPPED > | {} > | > | 7.2 | Native Throughput, UDP | SKIPPED > | {} > | > | 7.3 | Native Throughput, ICMP | SKIPPED > | {} > | > | 7.4 | Native Throughput, Multicast | SKIPPED > | {} > | > | 8.1 | VM to Host Uploading | SKIPPED > | {} > | > | 8.2 | VM to Host Downloading | SKIPPED > | {} > | > +----------+-------------------------------------------------------+-------------------+----------------------------------------------------------------------------------+ > > > My concern is about east-west bandwidth: > > Inter-host and intra-subnet (vmtp Scenario 2.1): I am getting ~6.8Gbps which > I found quite poor compared to 16.0 Gbps I am getting from the hosts > intra-host and intra-subnet (vmtp Scenario 1.1): In this example traffic > remains inside the compute node and no tunneling is required. I am getting > ~7.6Gbps which I found quite poor compared to 16.0 Gbps I am getting from the > hosts > > NOTE: I am not running anything except these tests on the machines so cpu > usage is quite low. > NOTE2: I know vmtp uses > > I would like to understand what is stopping my vms to get full network > bandwidth. Is this related with an OVS configuration? ML2? Other part of > neutron? Something else? > > Thank you very much > > > Manuel Sopena Ballesteros | Big data Engineer > Garvan Institute of Medical Research > The Kinghorn Cancer Centre, 370 Victoria Street, Darlinghurst, NSW 2010 > T: + 61 (0)2 9355 5760 | F: +61 (0)2 9295 8507 | E: manuel...@garvan.org.au > <mailto:manuel...@garvan.org.au> > > NOTICE > Please consider the environment before printing this email. This message and > any attachments are intended for the addressee named and may contain legally > privileged/confidential/copyright information. If you are not the intended > recipient, you should not read, use, disclose, copy or distribute this > communication. If you have received this message in error please notify us at > once by return email and then delete both messages. We accept no liability > for the distribution of viruses or similar in electronic communications. This > notice should not be removed. > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack> > Post to : openstack@lists.openstack.org > <mailto:openstack@lists.openstack.org> > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack> > NOTICE > Please consider the environment before printing this email. This message and > any attachments are intended for the addressee named and may contain legally > privileged/confidential/copyright information. If you are not the intended > recipient, you should not read, use, disclose, copy or distribute this > communication. If you have received this message in error please notify us at > once by return email and then delete both messages. We accept no liability > for the distribution of viruses or similar in electronic communications. This > notice should not be removed.
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
