I'v used ftrace to see which paths we take in the kernel. I remind you that the packet (in my scenario) looks like the following: ETH | IP | UDP | VXLAN | L2 | VLAN | IP | TCP | payload
When 8021q is not loaded: dev_gro_receive <-napi_gro_receive inet_gro_receive <-dev_gro_receive udp4_gro_receive <-inet_gro_receive udp_gro_receive <-udp4_gro_receive udp4_lib_lookup_skb <-udp_gro_receive __udp4_lib_lookup <-udp4_lib_lookup_skb compute_score <-__udp4_lib_lookup vxlan_gro_receive <-udp_gro_receive __pskb_pull_tail <-vxlan_gro_receive skb_copy_bits <-__pskb_pull_tail eth_gro_receive <-vxlan_gro_receive __pskb_pull_tail <-eth_gro_receive skb_copy_bits <-__pskb_pull_tail gro_find_receive_by_type <-eth_gro_receive netif_receive_skb_internal <-napi_gro_receive ktime_get_real <-netif_receive_skb_internal getnstimeofday64 <-ktime_get_real __getnstimeofday64 <-getnstimeofday64 skb_defer_rx_timestamp <-netif_receive_skb_internal classify <-skb_defer_rx_timestamp __netif_receive_skb <-netif_receive_skb_internal __netif_receive_skb_core <-__netif_receive_skb ip_rcv <-__netif_receive_skb_core nf_hook_slow <-ip_rcv nf_iterate <-nf_hook_slow ipv4_conntrack_defrag <-nf_iterate ipv4_conntrack_in <-nf_iterate When 8021q was loaded manually: inet_gro_receive <-dev_gro_receive classify <-skb_clone_tx_timestamp udp4_gro_receive <-inet_gro_receive _raw_spin_lock <-sch_direct_xmit udp_gro_receive <-udp4_gro_receive udp4_lib_lookup_skb <-udp_gro_receive local_bh_enable <-__dev_queue_xmit __udp4_lib_lookup <-udp4_lib_lookup_skb __local_bh_enable_ip <-local_bh_enable compute_score <-__udp4_lib_lookup local_bh_enable <-ip_finish_output vxlan_gro_receive <-udp_gro_receive __local_bh_enable_ip <-local_bh_enable __pskb_pull_tail <-vxlan_gro_receive skb_copy_bits <-__pskb_pull_tail local_bh_enable <-__dev_queue_xmit eth_gro_receive <-vxlan_gro_receive __local_bh_enable_ip <-local_bh_enable __pskb_pull_tail <-eth_gro_receive skb_copy_bits <-__pskb_pull_tail gro_find_receive_by_type <-eth_gro_receive local_bh_enable <-__dev_queue_xmit __local_bh_enable_ip <-local_bh_enable vlan_gro_receive <-eth_gro_receive __pskb_pull_tail <-vlan_gro_receive local_bh_enable <-ip_finish_output skb_copy_bits <-__pskb_pull_tail __local_bh_enable_ip <-local_bh_enable gro_find_receive_by_type <-vlan_gro_receive inet_gro_receive <-vlan_gro_receive kfree_skb_partial <-tcp_rcv_established __pskb_pull_tail <-inet_gro_receive skb_release_all <-kfree_skb_partial skb_copy_bits <-__pskb_pull_tail skb_release_head_state <-skb_release_all skb_release_data <-skb_release_all tcp4_gro_receive <-inet_gro_receive tcp_gro_receive <-tcp4_gro_receive put_page <-skb_release_data You can see that eth_gro_receive calls vlan_gro_recieve when 8021q module is loaded and then performs ding gro_receive for the next layers as well (inet, tcp), whereas on default scenario where 8021q is not loaded the vlan device ndo cannot be used. בתאריך יום ה׳, 25 באוק׳ 2018 ב-9:40 מאת Jiri Pirko <j...@resnulli.us >: > Wed, Oct 24, 2018 at 10:12:54PM CEST, michaels...@gmail.com wrote: > >Hi, > >I noticed that there is a performance issue when running traffic on a vlan > >interface that was created by OVS. > >If we create a bridge with a vlan interface, the 8021q module is not > loaded. > >Then when packets with a 8021q tag arrives, the linux stack can't use the > >vlan ndos (such as gro_recieve) since there is no such vlan device. > >If I perform the same test after loading the 8021q module, I get 2x-5x > >better performance. > > Could you please describe why exacly do you think you see this increase? > > > >I personally test that using tunnels (vxlan) but I think the issue exists > >even without the tunnel. > > > >*Creating a bridge and vlan interface:* > >ovs-vsctl --no-wait add-br br_0 > >ip link set br_0 up > >ovs-vsctl --no-wait add-port br_0 ens5f0 > >ovs-vsctl --no-wait add-port br_0 vlan10 tag=10 -- set interface vlan10 > >type=internal > >ip link set vlan10 up > >ip addr add <ip> dev vlan10 > > > >run traffic (netperf on the vlan10 interface) and see the result. > >Then do 'modprobe 8021q' and perform the same test -> you will see much > >better numbers. > > > >I am suggesting to load 80021q module once vlan interface is added through > >ovs. > >similarly as a vlan interface would be created using 'ip link'. > > > >Michael >
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
