On Mon, Nov 4, 2019 at 5:14 PM shuangyang qian <[email protected]> wrote:
>
> Hi:
> I make rpm packages for ovs and ovn with this
> document:http://docs.openvswitch.org/en/latest/intro/install/fedora/ . For
> use the kernel module in ovs tree, i configure with the command: ./configure
> --with-linux=/lib/modules/$(uname -r)/build .
> Then install the rpm packages.
> when i finished, i check the openvswitch.ko is like:
> # lsmod | grep openvswitch
> openvswitch 291276 0
> tunnel6 3115 1 openvswitch
> nf_defrag_ipv6 25957 2 nf_conntrack_ipv6,openvswitch
> nf_nat_ipv6 6459 2 openvswitch,ip6table_nat
> nf_nat_ipv4 6187 2 openvswitch,iptable_nat
> nf_nat 18080 5
> xt_nat,openvswitch,nf_nat_ipv6,nf_nat_masquerade_ipv4,nf_nat_ipv4
> nf_conntrack 102766 10
> ip_vs,nf_conntrack_ipv6,openvswitch,nf_conntrack_ipv4,nf_conntrack_netlink,nf_nat_ipv6,nf_nat_masquerade_ipv4,xt_conntrack,nf_nat_ipv4,nf_nat
> libcrc32c 1388 3 ip_vs,openvswitch,xfs
> ipv6 400397 92
> ip_vs,nf_conntrack_ipv6,openvswitch,nf_defrag_ipv6,nf_nat_ipv6,bridge
> # modinfo openvswitch
> filename: /lib/modules/4.9.18-19080201/extra/openvswitch/openvswitch.ko
> alias: net-pf-16-proto-16-family-ovs_ct_limit
> alias: net-pf-16-proto-16-family-ovs_meter
> alias: net-pf-16-proto-16-family-ovs_packet
> alias: net-pf-16-proto-16-family-ovs_flow
> alias: net-pf-16-proto-16-family-ovs_vport
> alias: net-pf-16-proto-16-family-ovs_datapath
> version: 2.11.2
> license: GPL
> description: Open vSwitch switching datapath
> srcversion: 9DDA327F9DD46B9813628A4
> depends:
> nf_conntrack,tunnel6,ipv6,nf_nat,nf_defrag_ipv6,libcrc32c,nf_nat_ipv6,nf_nat_ipv4
> vermagic: 4.9.18-19080201 SMP mod_unload modversions
> parm: udp_port:Destination UDP port (ushort)
> # rpm -qf /lib/modules/4.9.18-19080201/extra/openvswitch/openvswitch.ko
> openvswitch-kmod-2.11.2-1.el7.x86_64
>
> Then i start to build my network structure. I have two node，and network
> namespace vm1 on node1, network namespace vm2 on node2. vm1's veth pair
> veth-vm1 is on node1's br-int. vm2's veth pair veth-vm2 is on node2's br-int.
> In logical layer, there is one logical switch test-subnet and two logical
> switch port node1 and node2 on it. like this:
> # ovn-nbctl show
> switch 70585c0e-3cd9-459e-9448-3c13f3c0bfa3 (test-subnet)
> port node2
> addresses: ["00:00:00:00:00:02 192.168.100.20"]
> port node1
> addresses: ["00:00:00:00:00:01 192.168.100.10"]
> on node1:
> # ovs-vsctl show
> 5180f74a-1379-49af-b265-4403bd0d82d8
> Bridge br-int
> fail_mode: secure
> Port "ovn-431b9e-0"
> Interface "ovn-431b9e-0"
> type: geneve
> options: {csum="true", key=flow, remote_ip="10.18.124.2"}
> Port br-int
> Interface br-int
> type: internal
> Port "veth-vm1"
> Interface "veth-vm1"
> ovs_version: "2.11.2"
> # ip netns exec vm1 ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
> default qlen 1
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> valid_lft forever preferred_lft forever
> inet6 ::1/128 scope host
> valid_lft forever preferred_lft forever
> 14: ovs-gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN
> group default qlen 1000
> link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> 15: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group
> default qlen 1000
> link/ether 22:02:1b:08:ec:53 brd ff:ff:ff:ff:ff:ff
> 16: ovs-ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN group default
> qlen 1
> link/gre6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd
> 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
> 17: ovs-ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default
> qlen 1
> link/tunnel6 :: brd ::
> 18: vm1-eth0@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue
> state UP group default qlen 1000
> link/ether 00:00:00:00:00:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> inet 192.168.100.10/24 scope global vm1-eth0
> valid_lft forever preferred_lft forever
> inet6 fe80::200:ff:fe00:1/64 scope link
> valid_lft forever preferred_lft forever
>
>
> on node2:# ovs-vsctl show
> 011332d0-78bc-47f7-be3c-fab0beb08e28
> Bridge br-int
> fail_mode: secure
> Port br-int
> Interface br-int
> type: internal
> Port "ovn-c655f8-0"
> Interface "ovn-c655f8-0"
> type: geneve
> options: {csum="true", key=flow, remote_ip="10.18.124.1"}
> Port "veth-vm2"
> Interface "veth-vm2"
> ovs_version: "2.11.2"
> #ip netns exec vm2 ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
> default qlen 1
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> valid_lft forever preferred_lft forever
> inet6 ::1/128 scope host
> valid_lft forever preferred_lft forever
> 10: ovs-gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN
> group default qlen 1000
> link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> 11: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group
> default qlen 1000
> link/ether 4a:1d:ca:65:e3:ca brd ff:ff:ff:ff:ff:ff
> 12: ovs-ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN group default
> qlen 1
> link/gre6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd
> 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
> 13: ovs-ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default
> qlen 1
> link/tunnel6 :: brd ::
> 17: vm2-eth0@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue
> state UP group default qlen 1000
> link/ether 00:00:00:00:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> inet 192.168.100.20/24 scope global vm2-eth0
> valid_lft forever preferred_lft forever
> inet6 fe80::200:ff:fe00:2/64 scope link
> valid_lft forever preferred_lft forever
>
> then i start to use iperf to check the network performence, oh, by the way, i
> use geneve protocol between the two nodes, ovn-sbctl show is :
> # ovn-sbctl show
> Chassis "c655f877-b7ed-4bb5-a047-23521426d541"
> hostname: "node1.com"
> Encap geneve
> ip: "10.18.124.1"
> options: {csum="true"}
> Port_Binding "node1"
> Chassis "431b9efb-b464-42a1-a6dd-7fc6e0176137"
> hostname: "node2.com"
> Encap geneve
> ip: "10.18.124.2"
> options: {csum="true"}
> Port_Binding "node2"
>
> on node1, in network namespace vm1 i start the iperf3 as server:
> #ip netns exec vm1 iperf3 -s
> on node2, in network namespace vm2 i start the iper3 as client:
> # ip netns exec vm2 iperf3 -c 192.168.100.10
> Connecting to host 192.168.100.10, port 5201
> [ 4] local 192.168.100.20 port 40708 connected to 192.168.100.10 port 5201
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 4] 0.00-1.00 sec 431 MBytes 3.61 Gbits/sec 34 253 KBytes
> [ 4] 1.00-2.00 sec 426 MBytes 3.58 Gbits/sec 0 253 KBytes
> [ 4] 2.00-3.00 sec 426 MBytes 3.57 Gbits/sec 0 253 KBytes
> [ 4] 3.00-4.00 sec 401 MBytes 3.37 Gbits/sec 0 255 KBytes
> [ 4] 4.00-5.00 sec 429 MBytes 3.60 Gbits/sec 0 255 KBytes
> [ 4] 5.00-6.00 sec 413 MBytes 3.46 Gbits/sec 0 253 KBytes
> [ 4] 6.00-7.00 sec 409 MBytes 3.43 Gbits/sec 0 250 KBytes
> [ 4] 7.00-8.00 sec 427 MBytes 3.58 Gbits/sec 0 253 KBytes
> [ 4] 8.00-9.00 sec 417 MBytes 3.49 Gbits/sec 0 250 KBytes
> [ 4] 9.00-10.00 sec 385 MBytes 3.23 Gbits/sec 0 5.27 KBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 4] 0.00-10.00 sec 4.07 GBytes 3.49 Gbits/sec 34 sender
> [ 4] 0.00-10.00 sec 4.07 GBytes 3.49 Gbits/sec receiver
>
> as you see, the bw is only 3.xxGbits/sec, but my physics eth1's bw 10000M:
Hi, I run the ovs on node1 and node2, using geneve tunnel but I didn't
reproduce your issue.
create the geneve tunnel:
# ovs-vsctl add-br br-int
# ovs-vsctl add-port br-int gnv0 -- set Interface gnv0 type=geneve
options:csum=true options:key=100 options:remote_ip=1.1.1.200
# ovs-vsctl show
9393485c-c64c-490e-884e-418ff5d90251
Bridge br-int
Port gnv0
Interface gnv0
type: geneve
options: {csum="true", key="100", remote_ip="1.1.1.200"}
Port __tap01
Interface __tap01
Port br-int
Interface br-int
type: internal
# ip netns exec ns100 ifconfig
__tap00: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 2.2.2.100 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::254:ff:fe00:1 prefixlen 64 scopeid 0x20<link>
ether 00:54:00:00:00:01 txqueuelen 1000 (Ethernet)
RX packets 605000 bytes 39951500 (38.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 819864 bytes 31247862764 (29.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# ip netns exec ns100 iperf -c 2.2.2.200 -i 2 -t 10
------------------------------------------------------------
Client connecting to 2.2.2.200, TCP port 5001
TCP window size: 482 KByte (default)
------------------------------------------------------------
[ 3] local 2.2.2.100 port 41428 connected with 2.2.2.200 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.0 sec 1.85 GBytes 7.93 Gbits/sec
[ 3] 2.0- 4.0 sec 1.94 GBytes 8.33 Gbits/sec
# modinfo openvswitch
filename: /lib/modules/3.10.0-957.1.3.el7.x86_64/extra/openvswitch.ko
so, can you use the commands as show above to reproduce your
issue.(the kernel version is different.)
> # ethtool eth1
> Settings for eth1:
> Supported ports: [ FIBRE ]
> Supported link modes: 10000baseT/Full
> Supported pause frame use: Symmetric
> Supports auto-negotiation: No
> Supported FEC modes: Not reported
> Advertised link modes: 10000baseT/Full
> Advertised pause frame use: Symmetric
> Advertised auto-negotiation: No
> Advertised FEC modes: Not reported
> Speed: 10000Mb/s
> Duplex: Full
> Port: Other
> PHYAD: 0
> Transceiver: external
> Auto-negotiation: off
> Supports Wake-on: d
> Wake-on: d
> Current message level: 0x00000007 (7)
> drv probe link
> Link detected: yes
>
> when i uninstall the openvswitch-kmod package and use the openvswitch.ko in
> the upstream linux kernel, like this:
> # lsmod | grep openvswitch
> openvswitch 95805 0
> nf_defrag_ipv6 25957 2 nf_conntrack_ipv6,openvswitch
> nf_nat_ipv6 6459 2 openvswitch,ip6table_nat
> nf_nat_ipv4 6187 2 openvswitch,iptable_nat
> nf_nat 18080 5
> xt_nat,openvswitch,nf_nat_ipv6,nf_nat_masquerade_ipv4,nf_nat_ipv4
> nf_conntrack 102766 10
> ip_vs,nf_conntrack_ipv6,openvswitch,nf_conntrack_ipv4,nf_conntrack_netlink,nf_nat_ipv6,nf_nat_masquerade_ipv4,xt_conntrack,nf_nat_ipv4,nf_nat
> libcrc32c 1388 3 ip_vs,openvswitch,xfs
> # modinfo openvswitch
> filename:
> /lib/modules/4.9.18-19080201/kernel/net/openvswitch/openvswitch.ko
> alias: net-pf-16-proto-16-family-ovs_packet
> alias: net-pf-16-proto-16-family-ovs_flow
> alias: net-pf-16-proto-16-family-ovs_vport
> alias: net-pf-16-proto-16-family-ovs_datapath
> license: GPL
> description: Open vSwitch switching datapath
> srcversion: 915B872C96FB1D38D107742
> depends:
> nf_conntrack,nf_nat,libcrc32c,nf_nat_ipv6,nf_nat_ipv4,nf_defrag_ipv6
> intree: Y
> vermagic: 4.9.18-19080201 SMP mod_unload modversions
>
> and do the same test in above, and i get the follow result:
> # ip netns exec vm2 iperf3 -c 192.168.100.10
> Connecting to host 192.168.100.10, port 5201
> [ 4] local 192.168.100.20 port 40652 connected to 192.168.100.10 port 5201
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 4] 0.00-1.00 sec 1000 MBytes 8.39 Gbits/sec 4 290 KBytes
> [ 4] 1.00-2.00 sec 994 MBytes 8.34 Gbits/sec 0 292 KBytes
> [ 4] 2.00-3.00 sec 1002 MBytes 8.41 Gbits/sec 0 287 KBytes
> 4] 3.00-4.00 sec 994 MBytes 8.34 Gbits/sec 0 292 KBytes
> ▽ 4] 4.00-5.00 sec 992 MBytes 8.32 Gbits/sec 0 298 KBytes
> [ 4] 5.00-6.00 sec 994 MBytes 8.34 Gbits/sec 0 305 KBytes
> [ 4] 6.00-7.00 sec 989 MBytes 8.29 Gbits/sec 0 313 KBytes
> [ 4] 7.00-8.00 sec 992 MBytes 8.32 Gbits/sec 0 290 KBytes
> [ 4] 8.00-9.00 sec 996 MBytes 8.36 Gbits/sec 0 303 KBytes
> [ 4] 9.00-10.00 sec 955 MBytes 8.01 Gbits/sec 0 5.27 KBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 4] 0.00-10.00 sec 9.67 GBytes 8.31 Gbits/sec 4 sender
> [ 4] 0.00-10.00 sec 9.67 GBytes 8.31 Gbits/sec receiver
>
> so, i cann't understand why the performence is so poor when i use the kernel
> module build from the ovs tree.
>
> anyone can give me some advice where is wrong?
>
> thx!
>
> _______________________________________________
> discuss mailing list
> [email protected]
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
