Hi Finn,
Finally I was able to reproduce the error behavior w/ VXLAN traffic.
I found 2 issues related to the design being made:
1. The VXLAN decap requires more fields apart from the TCP header like the
l3_ofs and l4_ofs which are missing because of bypassing the miniflow extract.
2. Tunnel packets are being recirculated after the outer headers decap
(dp_netdev_recirculate). On the second processing stage the packet still
carries the flow mark therefore the action for the inner headers Is also
pop_tunnel.
I haven't established yet the final way to solve those issues. My current
thoughts:
1. Even though attractive we cannot bypass the miniflow extract. VXLAN is
just one case. Probably other cases will need different fields from the flow.
2. Use the md_is_valid flag on dp_netdev_input__ to recognize the non-first
round and to skip the mark to flow search. I don't really like this approach
because
* It is a bit hackish to relay on the md_is_valid. Alternatively we can
clear the mbuf mark flag but this basically the same.
* w/ tunnel packets the massive amount of flow rules is in the inner
headers. The outer doesn't change much. w/ the current design it means we
offload the less consuming part.
Having said that, the above approach looks the fastest w/o changing the
current design. We can further improve this part later.
Be happy to hear more suggestions.
From: Finn Christensen [mailto:[email protected]]
Sent: Thursday, June 21, 2018 5:46 PM
To: Koujalagi, MalleshX <[email protected]>; Shahaf Shuler
<[email protected]>; [email protected]
Cc: [email protected]; Ergin, Mesut A <[email protected]>; Tsai, James
<[email protected]>
Subject: RE: [ovs-dev] [PATCH v9 0/7] OVS-DPDK flow offload with rte_flow
Hi Mallesh and Shahaf,
I have tried to reproduce the issue Mallesh is reporting, using a Napatech NIC.
The result is that I'm able to reproduce the error and are not getting the
decap functionality to work when using VxLan tunneling together with partial
hw-offload. Mainly the VxLAN POP is not performed in OVS in my setup.
The VxLan setup I'm using is straight forward with 2 hosts and br-int and
br-phy bridges, like the typical examples.
I don't know how far you have got into debugging this, but what I se is that
the vxlan pop functionality needs packet information originally gathered in the
miniflow extract function, which is omitted with hw-offload. Therefore, it does
not work.
I'm still investigating how this may be solved, but I wanted to hear where you
are in debugging this issue, and if you already have got to a solution.
Regards,
Finn
From: Koujalagi, MalleshX
<[email protected]<mailto:[email protected]>>
Sent: 20. juni 2018 00:59
To: Shahaf Shuler <[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>; Finn Christensen
<[email protected]<mailto:[email protected]>>
Cc: [email protected]<mailto:[email protected]>; Ergin, Mesut A
<[email protected]<mailto:[email protected]>>; Tsai, James
<[email protected]<mailto:[email protected]>>
Subject: RE: [ovs-dev] [PATCH v9 0/7] OVS-DPDK flow offload with rte_flow
Hi Shahaf,
Thanks for setting up VXLAN env. and trying to reproduce, appreciate!!
As you suggested, I moved to DPDK17.11.3 stable and OvS 2.9.0, still observed
same issue.
An attached Vxlan_diagram.jpg, to get clear idea, we are seeing issue while
traffic injected from Traffic Generator-->HOST B[eth1-->Vxlan-->eth0]-->Vxlan
Tunnel Packet-->HOST A[eth0-->Vxlan-->eth1]-->Traffic Generator direction, at
Host A (eth0-->Vxlan-->eth1), we see Vxlan DECAP issue in case of HW-offload
enabled. Vxlan Tunnel packets are not DECAP from eth0 to eth1, not observed any
packet receiving @ eth1.
We injected random destination ip address(196.0.0.(0/1)),
1. Find tcpdump @ Host A (eth0).
07:24:12.013848 68:05:ca:33:ff:99 > 68:05:ca:33:fe:f9, ethertype IPv4 (0x0800),
length 110: 172.168.1.2.46816 > 172.168.1.1.4789: VXLAN, flags [I] (0x08), vni 0
00:10:94:00:00:12 > 00:00:01:00:00:01, ethertype IPv4 (0x0800), length 60:
197.0.0.0.1024 > 196.0.0.1.1024: UDP, length 18
07:24:12.013860 68:05:ca:33:ff:99 > 68:05:ca:33:fe:f9, ethertype IPv4 (0x0800),
length 110: 172.168.1.2.50892 > 172.168.1.1.4789: VXLAN, flags [I] (0x08), vni 0
00:10:94:00:00:12 > 00:00:01:00:00:01, ethertype IPv4 (0x0800), length 60:
197.0.0.0.1024 > 196.0.0.0.1024: UDP, length 18
1. Ovs logs.
2. Sure, further we will debug this issue using Skype/webex session. Please
let me know good time.
Best regards
-/Mallesh
From: Shahaf Shuler [mailto:[email protected]]
Sent: Monday, June 18, 2018 4:05 AM
To: Koujalagi, MalleshX
<[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Cc: [email protected]<mailto:[email protected]>; Ergin, Mesut A
<[email protected]<mailto:[email protected]>>; Tsai, James
<[email protected]<mailto:[email protected]>>
Subject: RE: [ovs-dev] [PATCH v9 0/7] OVS-DPDK flow offload with rte_flow
Mallesh,
I was finally able to setup the vxlan testing w/ OVS. Instead of using OVS on
both sides I used vxlan i/f to inject the traffic on one host and run ovs with
vxlan tunnel configuration you specified on the other.
I was not able to reproduce your case. Actually I see the rules are created
successfully (see attached ovs log "vxlan_tep")
Few observations:
1. The rule created for the VXLAN is for the outer pattern up to the UDP,
hence supported by the device
* 2018-06-18T08:11:17.466Z|00057|dpif_netdev(pmd53)|DBG|flow match:
recirc_id=0,eth,udp,vlan_tci=0x0000,dl_src=e4:1d:2d:
ca:ca:6a,dl_dst=e4:1d:2d:a3:aa:74,nw_dst=2.2.2.1,nw_frag=no,tp_dst=4789
1. I do see segfault on mlx5 PMD on 17.11.0 and this was resolved on 17.11.3
stable.
2. I used MLNX_OFED_LINUX-4.3-2.0.1.0, however I don't expect it to cause
any diff
The packets being sent to the ovs w/ vxlan tunnel logic are[1]. tcpdump after
the ovs logic is [2], the packet outer headers were removed as expected.
On top of all, I tried to force the validate function to fail. Indeed the flow
was not created but the decap functionality still happens (see attached ovs log
"vxlan_tep_force_err")
Can you please:
1. Provide me the type of packets you inject and don't see being decap.
2. Try w/ 17.11.3 stable to avoid any issues from the DPDK side
3. If you still see the issue can we set webex/skype meeting for joint
debug, currently I don't see the error you reported on.
[1]
###[ Ethernet ]###
dst= e4:1d:2d:a3:aa:74
src= e4:1d:2d:ca:ca:6a
type= IPv4
###[ IP ]###
version= 4
ihl= None
tos= 0x0
len= None
id= 1
flags=
frag= 0
ttl= 64
proto= udp
chksum= None
src= 2.2.2.2
dst= 2.2.2.1
\options\
###[ UDP ]###
sport= 34550
dport= 4789
len= None
chksum= None
###[ VXLAN ]###
flags= I
reserved1= 0
vni= 1000
reserved2= 0x0
###[ Ethernet ]###
dst= 00:00:5e:00:01:01
src= 18:66:da:f5:37:64
type= IPv4
###[ IP ]###
version= 4
ihl= None
tos= 0x0
len= None
id= 1
flags=
frag= 0
ttl= 64
proto= tcp
chksum= None
src= 10.7.12.62
dst= 1.1.1.1
\options\
###[ TCP ]###
sport= ftp_data
dport= http
seq= 0
ack= 0
dataofs= None
reserved= 0
flags= S
window= 8192
chksum= None
urgptr= 0
options= {}
[2]
11:13:59.477667 18:66:da:f5:37:64 > 00:00:5e:00:01:01, ethertype IPv4 (0x0800),
length 60: (tos 0x0, ttl 64, id 1, off
set 0, flags [none], proto TCP (6), length 40)
10.7.12.62.20 > 1.1.1.1.80: Flags [S], cksum 0x7738 (correct), seq 0, win
8192, length 0
--Shahaf
From: Koujalagi, MalleshX
<[email protected]<mailto:[email protected]>>
Sent: Friday, June 15, 2018 9:29 PM
To: Shahaf Shuler <[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Cc: [email protected]<mailto:[email protected]>; Ergin, Mesut A
<[email protected]<mailto:[email protected]>>; Tsai, James
<[email protected]<mailto:[email protected]>>
Subject: RE: [ovs-dev] [PATCH v9 0/7] OVS-DPDK flow offload with rte_flow
Hi Shahaf,
Thanks for pointing NIC/protocol support.
Find inline comments:
>>When you say DECAP functionality is broken you mean the flow is not actually
>>inserted to the HW right?
[Mallesh]: Yes, DECAP flows are not inserted to HW.
>>The datapath should still DECAP the flow correctly.
[Mallesh]: Agree, However the datapath failed to DECAP.
If VXLAN protocol support in netdev_dpdk_validate_flow failed (return -1)
then, should be fall back normal or default behavior of datapath right?
Have you tried w/o VXLAN tunnel rules?
[Mallesh]: Yes, tried, but same behavior.
Best regards,
-/Mallesh
Disclaimer: This email and any files transmitted with it may contain
confidential information intended for the addressee(s) only. The information is
not to be surrendered or copied to unauthorized persons. If you have received
this communication in error, please notify the sender immediately and delete
this e-mail from your system.
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
