Hi,
I need some help with strange OVS behaviours.
ovs-vsctl (Open vSwitch) 3.2.2
ovn-controller 23.09.1
Open vSwitch Library 3.2.2
TLDR: We need to restart Open VSwitch in order for TLS traffic to work
between a VM and Cloudflare R2. After restarting Open VSwitch the TLS
connection works fine.
(see attached pcap tls-error.txt)
See the attached openflow traces - they show a flow trace from Open Vswitch.
Also there is a retis trace (retis tool discussed at Open VSwitch
conference 2023).
Note the drop (TC_INGRESS) in this file
+ 1702601116185568 [swapper/140] 0 [tp] skb:kfree_skb
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 3 drop
(TC_INGRESS)
if 21 (enp148s0f0_1) rxif 21 172.27.18.244.57394 > 104.18.2.35.443
ttl 63 tos 0x0 id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.]
seq 792060930:792061367 ack 951229219 win 11
Again , once I restart Open vSwitch the problem goes away for a time
and comes back sometime later (not sure what that time frame is but
its a recurring issue.)
No. Time Source Destination Protocol
Length Info Delta
1 09:59:25.235480 204.52.24.116 104.18.3.35 TCP
70 57552 → 443 [SYN] Seq=0 Win=42340 Len=0 MSS=1460 SACK_PERM WS=4096
0.000000
Frame 1: 70 bytes on wire (560 bits), 70 bytes captured (560 bits)
Ethernet II, Src: 4e:42:14:a1:2a:fb (4e:42:14:a1:2a:fb), Dst: IETF-VRRP-VRID_ff
(00:00:5e:00:01:ff)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 120
Internet Protocol Version 4, Src: 204.52.24.116, Dst: 104.18.3.35
Transmission Control Protocol, Src Port: 57552, Dst Port: 443, Seq: 0, Len: 0
Source Port: 57552
Destination Port: 443
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 0]
Sequence Number: 0 (relative sequence number)
Sequence Number (raw): 3725020240
[Next Sequence Number: 1 (relative sequence number)]
Acknowledgment Number: 0
Acknowledgment number (raw): 0
1000 .... = Header Length: 32 bytes (8)
Flags: 0x002 (SYN)
000. .... .... = Reserved: Not set
...0 .... .... = Accurate ECN: Not set
.... 0... .... = Congestion Window Reduced: Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...0 .... = Acknowledgment: Not set
.... .... 0... = Push: Not set
.... .... .0.. = Reset: Not set
.... .... ..1. = Syn: Set
[Expert Info (Chat/Sequence): Connection establish request (SYN):
server port 443]
.... .... ...0 = Fin: Not set
[TCP Flags: ··········S·]
Window: 42340
[Calculated window size: 42340]
Checksum: 0x6ce6 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
Options: (12 bytes), Maximum segment size, No-Operation (NOP), No-Operation
(NOP), SACK permitted, No-Operation (NOP), Window scale
[Timestamps]
No. Time Source Destination Protocol
Length Info Delta
2 09:59:25.241176 104.18.3.35 204.52.24.116 TCP
70 443 → 57552 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1400 SACK_PERM
WS=8192 0.005696
Frame 2: 70 bytes on wire (560 bits), 70 bytes captured (560 bits)
Ethernet II, Src: Mellanox_4a:c0:fd (9c:05:91:4a:c0:fd), Dst: 4e:42:14:a1:2a:fb
(4e:42:14:a1:2a:fb)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 120
Internet Protocol Version 4, Src: 104.18.3.35, Dst: 204.52.24.116
Transmission Control Protocol, Src Port: 443, Dst Port: 57552, Seq: 0, Ack: 1,
Len: 0
Source Port: 443
Destination Port: 57552
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 0]
Sequence Number: 0 (relative sequence number)
Sequence Number (raw): 3015987896
[Next Sequence Number: 1 (relative sequence number)]
Acknowledgment Number: 1 (relative ack number)
Acknowledgment number (raw): 3725020241
1000 .... = Header Length: 32 bytes (8)
Flags: 0x012 (SYN, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Accurate ECN: Not set
.... 0... .... = Congestion Window Reduced: Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 0... = Push: Not set
.... .... .0.. = Reset: Not set
.... .... ..1. = Syn: Set
[Expert Info (Chat/Sequence): Connection establish acknowledge
(SYN+ACK): server port 443]
.... .... ...0 = Fin: Not set
[TCP Flags: ·······A··S·]
Window: 65535
[Calculated window size: 65535]
Checksum: 0x0bf8 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
Options: (12 bytes), Maximum segment size, No-Operation (NOP), No-Operation
(NOP), SACK permitted, No-Operation (NOP), Window scale
[Timestamps]
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 1]
[The RTT to ACK the segment was: 0.005696000 seconds]
[iRTT: 0.007189000 seconds]
No. Time Source Destination Protocol
Length Info Delta
3 09:59:25.242669 204.52.24.116 104.18.3.35 TCP
58 57552 → 443 [ACK] Seq=1 Ack=1 Win=45056 Len=0 0.001493
Frame 3: 58 bytes on wire (464 bits), 58 bytes captured (464 bits)
Ethernet II, Src: 4e:42:14:a1:2a:fb (4e:42:14:a1:2a:fb), Dst: IETF-VRRP-VRID_ff
(00:00:5e:00:01:ff)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 120
Internet Protocol Version 4, Src: 204.52.24.116, Dst: 104.18.3.35
Transmission Control Protocol, Src Port: 57552, Dst Port: 443, Seq: 1, Ack: 1,
Len: 0
Source Port: 57552
Destination Port: 443
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 0]
Sequence Number: 1 (relative sequence number)
Sequence Number (raw): 3725020241
[Next Sequence Number: 1 (relative sequence number)]
Acknowledgment Number: 1 (relative ack number)
Acknowledgment number (raw): 3015987897
0101 .... = Header Length: 20 bytes (5)
Flags: 0x010 (ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Accurate ECN: Not set
.... 0... .... = Congestion Window Reduced: Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 0... = Push: Not set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·······A····]
Window: 11
[Calculated window size: 45056]
[Window size scaling factor: 4096]
Checksum: 0x4c89 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
[Timestamps]
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 2]
[The RTT to ACK the segment was: 0.001493000 seconds]
[iRTT: 0.007189000 seconds]
No. Time Source Destination Protocol
Length Info Delta
11 09:59:40.248755 104.18.3.35 204.52.24.116 TCP
60 443 → 57552 [FIN, ACK] Seq=1 Ack=1 Win=65536 Len=0 0.609588
Frame 11: 60 bytes on wire (480 bits), 60 bytes captured (480 bits)
Ethernet II, Src: Mellanox_4a:c0:fd (9c:05:91:4a:c0:fd), Dst: 4e:42:14:a1:2a:fb
(4e:42:14:a1:2a:fb)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 120
Internet Protocol Version 4, Src: 104.18.3.35, Dst: 204.52.24.116
Transmission Control Protocol, Src Port: 443, Dst Port: 57552, Seq: 1, Ack: 1,
Len: 0
Source Port: 443
Destination Port: 57552
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 0]
Sequence Number: 1 (relative sequence number)
Sequence Number (raw): 3015987897
[Next Sequence Number: 2 (relative sequence number)]
Acknowledgment Number: 1 (relative ack number)
Acknowledgment number (raw): 3725020241
0101 .... = Header Length: 20 bytes (5)
Flags: 0x011 (FIN, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Accurate ECN: Not set
.... 0... .... = Congestion Window Reduced: Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 0... = Push: Not set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...1 = Fin: Set
[Expert Info (Chat/Sequence): Connection finish (FIN)]
[TCP Flags: ·······A···F]
[Expert Info (Note/Sequence): This frame initiates the connection
closing]
Window: 8
[Calculated window size: 65536]
[Window size scaling factor: 8192]
Checksum: 0x4c8b [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
[Timestamps]
No. Time Source Destination Protocol
Length Info Delta
12 09:59:40.252248 204.52.24.116 104.18.3.35 TLSv1
65 [TCP Previous segment not captured] , Alert (Level: Fatal, Description:
Decode Error) 0.003493
Frame 12: 65 bytes on wire (520 bits), 65 bytes captured (520 bits)
Ethernet II, Src: 4e:42:14:a1:2a:fb (4e:42:14:a1:2a:fb), Dst: IETF-VRRP-VRID_ff
(00:00:5e:00:01:ff)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 120
Internet Protocol Version 4, Src: 204.52.24.116, Dst: 104.18.3.35
Transmission Control Protocol, Src Port: 57552, Dst Port: 443, Seq: 438, Ack:
2, Len: 7
Source Port: 57552
Destination Port: 443
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 7]
Sequence Number: 438 (relative sequence number)
Sequence Number (raw): 3725020678
[Next Sequence Number: 446 (relative sequence number)]
Acknowledgment Number: 2 (relative ack number)
Acknowledgment number (raw): 3015987898
0101 .... = Header Length: 20 bytes (5)
Flags: 0x019 (FIN, PSH, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Accurate ECN: Not set
.... 0... .... = Congestion Window Reduced: Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 1... = Push: Set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...1 = Fin: Set
[Expert Info (Chat/Sequence): Connection finish (FIN)]
[TCP Flags: ·······AP··F]
[Expert Info (Note/Sequence): This frame undergoes the connection
closing]
Window: 11
[Calculated window size: 45056]
[Window size scaling factor: 4096]
Checksum: 0x00be [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
[Timestamps]
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 11]
[The RTT to ACK the segment was: 0.003493000 seconds]
[iRTT: 0.007189000 seconds]
[TCP Analysis Flags]
[Expert Info (Warning/Sequence): Previous segment(s) not captured
(common at capture start)]
TCP payload (7 bytes)
Transport Layer Security
TLSv1 Record Layer: Alert (Level: Fatal, Description: Decode Error)
Content Type: Alert (21)
Version: TLS 1.0 (0x0301)
Length: 2
Alert Message
Level: Fatal (2)
Description: Decode Error (50)
No. Time Source Destination Protocol
Length Info Delta
13 09:59:40.257853 104.18.3.35 204.52.24.116 TCP
56 443 → 57552 [RST] Seq=2 Win=0 Len=0 0.005605
Frame 13: 56 bytes on wire (448 bits), 56 bytes captured (448 bits)
Ethernet II, Src: Mellanox_4a:c0:fd (9c:05:91:4a:c0:fd), Dst: 4e:42:14:a1:2a:fb
(4e:42:14:a1:2a:fb)
Internet Protocol Version 4, Src: 104.18.3.35, Dst: 204.52.24.116
Transmission Control Protocol, Src Port: 443, Dst Port: 57552, Seq: 2, Len: 0
Source Port: 443
Destination Port: 57552
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 0]
Sequence Number: 2 (relative sequence number)
Sequence Number (raw): 3015987898
[Next Sequence Number: 2 (relative sequence number)]
Acknowledgment Number: 0
Acknowledgment number (raw): 0
0101 .... = Header Length: 20 bytes (5)
Flags: 0x004 (RST)
000. .... .... = Reserved: Not set
...0 .... .... = Accurate ECN: Not set
.... 0... .... = Congestion Window Reduced: Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...0 .... = Acknowledgment: Not set
.... .... 0... = Push: Not set
.... .... .1.. = Reset: Set
[Expert Info (Warning/Sequence): Connection reset (RST)]
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·········R··]
Window: 0
[Calculated window size: 0]
[Window size scaling factor: 8192]
Checksum: 0x76f8 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
[Timestamps]
[root@vaeq-cu2b-r104-prod-hv-03 cloud-user]# ovs-appctl ofproto/trace br-int
in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,tcp,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_ttl=32,tcp_src=52776,tcp_dst=443,tcp_flags=2
Flow:
tcp,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
0. in_port=7753, priority 100, cookie 0xc33f39c4
set_field:0x16d->reg13
set_field:0x155->reg11
set_field:0x1cf->reg12
set_field:0x12->metadata
set_field:0xca->reg14
resubmit(,8)
8. metadata=0x12, priority 50, cookie 0x1645d3f2
set_field:0/0x1000->reg10
resubmit(,73)
73.
ip,reg14=0xca,metadata=0x12,dl_src=4e:42:14:a1:2a:fb,nw_src=172.27.18.244,
priority 90, cookie 0xc33f39c4
set_field:0/0x1000->reg10
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,9)
9. metadata=0x12, priority 0, cookie 0xcc4fd106
resubmit(,10)
10. metadata=0x12, priority 0, cookie 0x9e10ad0e
resubmit(,11)
11. metadata=0x12, priority 0, cookie 0x557f3249
resubmit(,12)
12. ip,metadata=0x12, priority 100, cookie 0x14131a67
set_field:0x1000000000000000000000000/0x1000000000000000000000000->xxreg0
resubmit(,13)
13. metadata=0x12, priority 0, cookie 0x85f9ed4f
resubmit(,14)
14. ip,reg0=0x1/0x1,metadata=0x12, priority 100, cookie 0x279651c
ct(table=15,zone=NXM_NX_REG13[0..15])
drop
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 15.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
tcp,reg0=0x1,reg11=0x155,reg12=0x1cf,reg13=0x16d,reg14=0xca,metadata=0x12,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
Megaflow:
recirc_id=0,eth,tcp,in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_frag=no
Datapath actions: ct(zone=365),recirc(0x13b5a)
===============================================================================
recirc(0x13b5a) - resume conntrack with default ct_state=trk|new (use --ct-next
to customize)
===============================================================================
Flow:
recirc_id=0x13b5a,ct_state=new|trk,ct_zone=365,eth,tcp,reg0=0x1,reg11=0x155,reg12=0x1cf,reg13=0x16d,reg14=0xca,metadata=0x12,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 15
15. ct_state=+new-est+trk,metadata=0x12, priority 7, cookie 0xa9e0ee6f
set_field:0x80000000000000000000000000/0x80000000000000000000000000->xxreg0
set_field:0x200000000000000000000000000/0x200000000000000000000000000->xxreg0
resubmit(,16)
16. conj_id=2865573479,tcp,reg0=0x80/0x80,reg14=0xca,metadata=0x12, priority
3000, cookie 0xabdec111
set_field:0x1000000000000/0x1000000000000->xreg4
set_field:0x2000000000000000000000000/0x2000000000000000000000000->xxreg0
resubmit(,17)
17. reg8=0x10000/0x10000,metadata=0x12, priority 1000, cookie 0x8171c04a
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,18)
18. metadata=0x12, priority 0, cookie 0x62454929
resubmit(,19)
19. metadata=0x12, priority 0, cookie 0x3bb47080
resubmit(,20)
20. metadata=0x12, priority 0, cookie 0x73face9d
resubmit(,21)
21. metadata=0x12, priority 0, cookie 0x8e46634d
resubmit(,22)
22. metadata=0x12, priority 0, cookie 0xbddac461
resubmit(,23)
23. metadata=0x12, priority 0, cookie 0x9a32c0de
resubmit(,24)
24. metadata=0x12, priority 0, cookie 0x79b1c074
resubmit(,25)
25. metadata=0x12, priority 0, cookie 0xc02374d3
resubmit(,26)
26. metadata=0x12, priority 0, cookie 0x218dc750
resubmit(,27)
27. metadata=0x12, priority 0, cookie 0xf6943631
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,28)
28. ip,reg0=0x2/0x2002,metadata=0x12, priority 100, cookie 0x7fdca4fb
ct(commit,zone=NXM_NX_REG13[0..15],nat(src),exec(set_field:0/0x1->ct_mark))
nat(src)
set_field:0/0x1->ct_mark
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
resubmit(,29)
29. metadata=0x12, priority 0, cookie 0x34e5fac7
resubmit(,30)
30. metadata=0x12, priority 0, cookie 0xac3f53bd
resubmit(,31)
31. metadata=0x12, priority 0, cookie 0x54000bd5
resubmit(,32)
32. metadata=0x12, priority 0, cookie 0x29ab49f3
resubmit(,33)
33. metadata=0x12, priority 0, cookie 0xae6aefcb
resubmit(,34)
34. metadata=0x12, priority 0, cookie 0x632dc93b
resubmit(,35)
35. metadata=0x12,dl_dst=1a:16:b1:58:e1:cd, priority 50, cookie 0x3fa33935
set_field:0x1->reg15
resubmit(,37)
37. priority 0
resubmit(,39)
39. priority 0
resubmit(,40)
40. reg15=0x1,metadata=0x12, priority 100, cookie 0x95c785db
set_field:0x155->reg11
set_field:0x1cf->reg12
resubmit(,41)
41. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,42)
42. ip,reg15=0x1,metadata=0x12, priority 110, cookie 0xc9a79824
resubmit(,43)
43. ip,reg15=0x1,metadata=0x12, priority 110, cookie 0xac7d5d78
resubmit(,44)
44. metadata=0x12, priority 0, cookie 0xa1ddb4f6
resubmit(,45)
45. ct_state=-trk,metadata=0x12, priority 5, cookie 0xb2622a65
set_field:0x100000000000000000000000000/0x100000000000000000000000000->xxreg0
set_field:0x200000000000000000000000000/0x200000000000000000000000000->xxreg0
resubmit(,46)
46. metadata=0x12, priority 0, cookie 0x267ae0e3
resubmit(,47)
47. metadata=0x12, priority 0, cookie 0x914392c3
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,48)
48. metadata=0x12, priority 0, cookie 0xc4f6a97f
resubmit(,49)
49. metadata=0x12, priority 0, cookie 0x8ebb0c71
resubmit(,50)
50. metadata=0x12, priority 0, cookie 0x61e385f4
resubmit(,51)
51. metadata=0x12, priority 0, cookie 0x4c722ce1
set_field:0/0x1000->reg10
resubmit(,75)
75. No match.
drop
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,52)
52. metadata=0x12, priority 0, cookie 0x3c01b89d
resubmit(,64)
64. priority 0
resubmit(,65)
65. reg15=0x1,metadata=0x12, priority 100, cookie 0x95c785db
clone(ct_clear,set_field:0->reg11,set_field:0->reg12,set_field:0->reg13,set_field:0x4cc->reg11,set_field:0x383->reg12,set_field:0x13->metadata,set_field:0x1->reg14,set_field:0->reg10,set_field:0->reg15,set_field:0->reg0,set_field:0->reg1,set_field:0->reg2,set_field:0->reg3,set_field:0->reg4,set_field:0->reg5,set_field:0->reg6,set_field:0->reg7,set_field:0->reg8,set_field:0->reg9,resubmit(,8))
ct_clear
set_field:0->reg11
set_field:0->reg12
set_field:0->reg13
set_field:0x4cc->reg11
set_field:0x383->reg12
set_field:0x13->metadata
set_field:0x1->reg14
set_field:0->reg10
set_field:0->reg15
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,8)
8. reg14=0x1,metadata=0x13,dl_dst=1a:16:b1:58:e1:cd, priority 50, cookie
0x8522832c
set_field:0x1a16b158e1cd0000000000000000/0xffffffffffff0000000000000000->xxreg0
resubmit(,9)
9. metadata=0x13, priority 0, cookie 0xef4dd8a9
set_field:0x4/0x4->xreg4
resubmit(,10)
10. reg9=0x4/0x4,metadata=0x13, priority 100, cookie 0x37718dd7
resubmit(,79)
79.
ip,reg14=0x1,metadata=0x13,dl_src=4e:42:14:a1:2a:fb,nw_src=172.27.18.244,
priority 100, cookie 0x2e544767
drop
resubmit(,11)
11. metadata=0x13, priority 0, cookie 0x3c2eb1b
resubmit(,12)
12. metadata=0x13, priority 0, cookie 0x44f62446
resubmit(,13)
13. metadata=0x13, priority 0, cookie 0x3842bec6
resubmit(,14)
14. metadata=0x13, priority 0, cookie 0xfd7b2ab9
resubmit(,15)
15. metadata=0x13, priority 0, cookie 0xefbd7e27
resubmit(,16)
16. metadata=0x13, priority 0, cookie 0xf439d853
resubmit(,17)
17. metadata=0x13, priority 0, cookie 0x123f01f0
resubmit(,18)
18. metadata=0x13, priority 0, cookie 0x142cd59b
resubmit(,19)
19. metadata=0x13, priority 0, cookie 0x297e0190
resubmit(,20)
20. metadata=0x13, priority 0, cookie 0x61e9e3c7
set_field:0/0xffffffff->xxreg1
resubmit(,21)
21. ip,reg7=0,metadata=0x13, priority 1, cookie 0x9f114f3d
dec_ttl()
set_field:0/0xffff00000000->xreg4
set_field:0x64646401000000000000000000000000/0xffffffff000000000000000000000000->xxreg0
set_field:0x646464030000000000000000/0xffffffff0000000000000000->xxreg0
set_field:2a:d5:d1:e8:89:cc->eth_src
set_field:0x2->reg15
set_field:0x1/0x1->reg10
resubmit(,22)
22. reg8=0/0xffff,metadata=0x13, priority 150, cookie 0x6ece899a
resubmit(,23)
23. metadata=0x13, priority 0, cookie 0x72437536
set_field:0/0xffff00000000->xreg4
resubmit(,24)
24. reg8=0/0xffff,metadata=0x13, priority 150, cookie 0xce06f1
resubmit(,25)
25. ip,metadata=0x13, priority 1, cookie 0xf690342c
push:NXM_NX_REG0[]
push:NXM_NX_XXREG0[96..127]
pop:NXM_NX_REG0[]
-> NXM_NX_REG0[] is now 0x64646401
set_field:00:00:00:00:00:00->eth_dst
resubmit(,66)
66. reg0=0x64646401,reg15=0x2,metadata=0x13, priority 100, cookie 0x97b4353d
set_field:00:00:5e:00:01:ff->eth_dst
set_field:0x40/0x40->reg10
pop:NXM_NX_REG0[]
-> NXM_NX_REG0[] is now 0x64646401
resubmit(,26)
26. metadata=0x13, priority 0, cookie 0x6c2ad4cc
resubmit(,27)
27. metadata=0x13, priority 0, cookie 0xadc9fb4f
resubmit(,28)
28. ip,reg15=0x2,metadata=0x13,nw_src=172.27.18.244, priority 100, cookie
0xfa923492
set_field:4e:42:14:a1:2a:fb->eth_src
set_field:0xcc3418740000000000000000/0xffffffff0000000000000000->xxreg0
resubmit(,29)
29. metadata=0x13, priority 0, cookie 0x571bbf77
resubmit(,37)
37. priority 0
resubmit(,39)
39. priority 0
resubmit(,40)
40. reg15=0x2,metadata=0x13, priority 100, cookie 0x407086d0
set_field:0x4cc->reg11
set_field:0x383->reg12
resubmit(,41)
41. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,42)
42. metadata=0x13, priority 0, cookie 0x4c3b7fcc
set_field:0/0x10->xreg4
resubmit(,43)
43. ip,reg15=0x2,metadata=0x13,nw_src=172.27.18.244, priority 100, cookie
0x540f90b2
set_field:4e:42:14:a1:2a:fb->eth_src
ct(table=44,zone=NXM_NX_REG11[0..15],nat)
nat
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 44.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
recirc_id=0x13b5a,eth,tcp,reg0=0x300,reg11=0x155,reg12=0x1cf,reg13=0x16d,reg14=0xca,reg15=0x1,metadata=0x12,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
Megaflow:
recirc_id=0x13b5a,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,tcp,in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.0.0.0/5,nw_ttl=32,nw_frag=no,tp_src=0x8000/0x8000,tp_dst=0x180/0xffc0
Datapath actions:
ct(commit,zone=365,mark=0/0x1,nat(src)),set(eth(dst=00:00:5e:00:01:ff)),set(ipv4(ttl=31)),ct(zone=1228,nat),recirc(0x13b5c)
===============================================================================
recirc(0x13b5c) - resume conntrack with default ct_state=trk|new (use --ct-next
to customize)
Replacing src/dst IP/ports to simulate NAT:
Initial flow:
Modified flow:
===============================================================================
Flow:
recirc_id=0x13b5c,ct_state=new|trk,ct_zone=1228,eth,tcp,reg10=0x41,reg11=0x4cc,reg12=0x383,reg14=0x1,reg15=0x2,metadata=0x13,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=31,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 44
44. metadata=0x13, priority 0, cookie 0x48ef0c42
resubmit(,45)
45. ct_state=-rpl+trk,ip,reg15=0x2,metadata=0x13,nw_src=172.27.18.244, priority
161, cookie 0x5414262a
set_field:4e:42:14:a1:2a:fb->eth_src
ct(commit,table=46,zone=NXM_NX_REG12[0..15],nat(src=204.52.24.116))
nat(src=204.52.24.116)
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 46.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
recirc_id=0x13b5c,eth,tcp,reg10=0x41,reg11=0x4cc,reg12=0x383,reg14=0x1,reg15=0x2,metadata=0x13,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_src=172.27.18.244,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=31,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
Megaflow:
recirc_id=0x13b5c,ct_state=-rpl+trk,eth,ip,in_port=7753,dl_src=4e:42:14:a1:2a:fb,nw_src=172.27.18.244,nw_frag=no
Datapath actions: ct(commit,zone=899,nat(src=204.52.24.116)),recirc(0x25f84)
===============================================================================
recirc(0x25f84) - resume conntrack with default ct_state=trk|new (use --ct-next
to customize)
Replacing src/dst IP/ports to simulate NAT:
Initial flow: nw_src=172.27.18.244,tp_src=52776,nw_dst=104.18.2.35,tp_dst=443
Modified flow: nw_src=204.52.24.116,tp_src=52776,nw_dst=104.18.2.35,tp_dst=443
===============================================================================
Flow:
recirc_id=0x25f84,ct_state=new|trk,ct_zone=899,eth,tcp,reg10=0x41,reg11=0x4cc,reg12=0x383,reg14=0x1,reg15=0x2,metadata=0x13,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_src=204.52.24.116,nw_dst=104.18.2.35,nw_tos=0,nw_ecn=0,nw_ttl=31,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 46
46. metadata=0x13, priority 0, cookie 0x3800cb88
resubmit(,47)
47. metadata=0x13, priority 0, cookie 0xc2575be2
resubmit(,48)
48. reg15=0x2,metadata=0x13, priority 100, cookie 0xaf19a22a
resubmit(,64)
64. reg10=0x1/0x1,reg15=0x2,metadata=0x13, priority 100, cookie 0x407086d0
push:NXM_OF_IN_PORT[]
set_field:ANY->in_port
resubmit(,65)
65. reg15=0x2,metadata=0x13, priority 100, cookie 0x407086d0
clone(ct_clear,set_field:0->reg11,set_field:0->reg12,set_field:0->reg13,set_field:0x2c2->reg11,set_field:0x327->reg12,set_field:0x1->metadata,set_field:0xa->reg14,set_field:0->reg10,set_field:0->reg15,set_field:0->reg0,set_field:0->reg1,set_field:0->reg2,set_field:0->reg3,set_field:0->reg4,set_field:0->reg5,set_field:0->reg6,set_field:0->reg7,set_field:0->reg8,set_field:0->reg9,resubmit(,8))
ct_clear
set_field:0->reg11
set_field:0->reg12
set_field:0->reg13
set_field:0x2c2->reg11
set_field:0x327->reg12
set_field:0x1->metadata
set_field:0xa->reg14
set_field:0->reg10
set_field:0->reg15
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,8)
8. metadata=0x1, priority 50, cookie 0x1645d3f2
set_field:0/0x1000->reg10
resubmit(,73)
73. No match.
drop
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,9)
9. metadata=0x1, priority 0, cookie 0xcc4fd106
resubmit(,10)
10. metadata=0x1, priority 0, cookie 0x9e10ad0e
resubmit(,11)
11. metadata=0x1, priority 0, cookie 0x557f3249
resubmit(,12)
12. metadata=0x1, priority 0, cookie 0x915c56b1
resubmit(,13)
13. ip,reg14=0xa,metadata=0x1, priority 110, cookie 0xccdcd3e1
resubmit(,14)
14. metadata=0x1, priority 0, cookie 0x134ce32f
resubmit(,15)
15. metadata=0x1, priority 65535, cookie 0x49627e5f
resubmit(,16)
16. metadata=0x1, priority 65535, cookie 0xc947843d
resubmit(,17)
17. metadata=0x1, priority 0, cookie 0xd2fb4d2
resubmit(,18)
18. metadata=0x1, priority 0, cookie 0x62454929
resubmit(,19)
19. metadata=0x1, priority 0, cookie 0x3bb47080
resubmit(,20)
20. metadata=0x1, priority 0, cookie 0x73face9d
resubmit(,21)
21. metadata=0x1, priority 0, cookie 0x8e46634d
resubmit(,22)
22. metadata=0x1, priority 0, cookie 0xbddac461
resubmit(,23)
23. metadata=0x1, priority 0, cookie 0x9a32c0de
resubmit(,24)
24. metadata=0x1, priority 0, cookie 0x79b1c074
resubmit(,25)
25. metadata=0x1, priority 0, cookie 0xc02374d3
resubmit(,26)
26. metadata=0x1, priority 0, cookie 0x218dc750
resubmit(,27)
27. metadata=0x1, priority 0, cookie 0x8db4eebc
resubmit(,28)
28. metadata=0x1, priority 0, cookie 0x6deecbbe
resubmit(,29)
29. metadata=0x1, priority 0, cookie 0x34e5fac7
resubmit(,30)
30. metadata=0x1, priority 0, cookie 0xac3f53bd
resubmit(,31)
31. metadata=0x1, priority 0, cookie 0x54000bd5
resubmit(,32)
32. metadata=0x1, priority 0, cookie 0x29ab49f3
resubmit(,33)
33. metadata=0x1, priority 0, cookie 0xae6aefcb
resubmit(,34)
34. metadata=0x1, priority 0, cookie 0x632dc93b
resubmit(,35)
35. metadata=0x1, priority 0, cookie 0x9961115c
set_field:0->reg15
resubmit(,71)
71. No match.
drop
resubmit(,36)
36. reg15=0,metadata=0x1, priority 50, cookie 0x62255993
set_field:0x8001->reg15
resubmit(,37)
37. priority 0
resubmit(,39)
39. priority 0
resubmit(,40)
40. reg15=0x8001,metadata=0x1, priority 100, cookie 0x11699429
set_field:0x14->reg13
set_field:0x1->reg15
resubmit(,41)
41. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,42)
42. metadata=0x1, priority 0, cookie 0x9ad1480d
resubmit(,43)
43. ip,reg15=0x1,metadata=0x1, priority 110, cookie 0xca4977d3
ct_clear
resubmit(,44)
44. metadata=0x1, priority 0, cookie 0xa1ddb4f6
resubmit(,45)
45. metadata=0x1, priority 65535, cookie 0x91b4350e
resubmit(,46)
46. metadata=0x1, priority 65535, cookie 0xfc9c351d
resubmit(,47)
47. metadata=0x1, priority 0, cookie 0xb984696e
resubmit(,48)
48. metadata=0x1, priority 0, cookie 0xc4f6a97f
resubmit(,49)
49. metadata=0x1, priority 0, cookie 0x8ebb0c71
resubmit(,50)
50. metadata=0x1, priority 0, cookie 0x61e385f4
resubmit(,51)
51. metadata=0x1, priority 0, cookie 0x4c722ce1
set_field:0/0x1000->reg10
resubmit(,75)
75. No match.
drop
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,52)
52. metadata=0x1, priority 0, cookie 0x3c01b89d
resubmit(,64)
64. priority 0
resubmit(,65)
65. reg15=0x1,metadata=0x1, priority 100, cookie 0x4581da82
push_vlan:0x8100
set_field:4216->vlan_vid
output:7754
bridge("br-provider")
---------------------
0. priority 0
NORMAL
-> forwarding to learned port
pop_vlan
set_field:0x8001->reg15
pop:NXM_OF_IN_PORT[]
-> NXM_OF_IN_PORT[] is now 7753
Final flow: unchanged
Megaflow:
recirc_id=0x25f84,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,ip,in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_dst=0.0.0.0/1,nw_frag=no
Datapath actions: ct_clear,push_vlan(vid=120,pcp=0),5
[root@vaeq-cu2b-r104-prod-hv-03 cloud-user]# ovs-appctl ofproto/trace br-int
in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,tcp,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_ttl=32,tcp_src=52776,tcp_dst=443,tcp_flags=2
Flow:
tcp,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
0. in_port=7753, priority 100, cookie 0xc33f39c4
set_field:0x16d->reg13
set_field:0x155->reg11
set_field:0x1cf->reg12
set_field:0x12->metadata
set_field:0xca->reg14
resubmit(,8)
8. metadata=0x12, priority 50, cookie 0x1645d3f2
set_field:0/0x1000->reg10
resubmit(,73)
73.
ip,reg14=0xca,metadata=0x12,dl_src=4e:42:14:a1:2a:fb,nw_src=172.27.18.244,
priority 90, cookie 0xc33f39c4
set_field:0/0x1000->reg10
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,9)
9. metadata=0x12, priority 0, cookie 0xcc4fd106
resubmit(,10)
10. metadata=0x12, priority 0, cookie 0x9e10ad0e
resubmit(,11)
11. metadata=0x12, priority 0, cookie 0x557f3249
resubmit(,12)
12. ip,metadata=0x12, priority 100, cookie 0x14131a67
set_field:0x1000000000000000000000000/0x1000000000000000000000000->xxreg0
resubmit(,13)
13. metadata=0x12, priority 0, cookie 0x85f9ed4f
resubmit(,14)
14. ip,reg0=0x1/0x1,metadata=0x12, priority 100, cookie 0x279651c
ct(table=15,zone=NXM_NX_REG13[0..15])
drop
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 15.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
tcp,reg0=0x1,reg11=0x155,reg12=0x1cf,reg13=0x16d,reg14=0xca,metadata=0x12,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
Megaflow:
recirc_id=0,eth,tcp,in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_frag=no
Datapath actions: ct(zone=365),recirc(0x13b5a)
===============================================================================
recirc(0x13b5a) - resume conntrack with default ct_state=trk|new (use --ct-next
to customize)
===============================================================================
Flow:
recirc_id=0x13b5a,ct_state=new|trk,ct_zone=365,eth,tcp,reg0=0x1,reg11=0x155,reg12=0x1cf,reg13=0x16d,reg14=0xca,metadata=0x12,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 15
15. ct_state=+new-est+trk,metadata=0x12, priority 7, cookie 0xa9e0ee6f
set_field:0x80000000000000000000000000/0x80000000000000000000000000->xxreg0
set_field:0x200000000000000000000000000/0x200000000000000000000000000->xxreg0
resubmit(,16)
16. conj_id=2865573479,tcp,reg0=0x80/0x80,reg14=0xca,metadata=0x12, priority
3000, cookie 0xabdec111
set_field:0x1000000000000/0x1000000000000->xreg4
set_field:0x2000000000000000000000000/0x2000000000000000000000000->xxreg0
resubmit(,17)
17. reg8=0x10000/0x10000,metadata=0x12, priority 1000, cookie 0x8171c04a
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,18)
18. metadata=0x12, priority 0, cookie 0x62454929
resubmit(,19)
19. metadata=0x12, priority 0, cookie 0x3bb47080
resubmit(,20)
20. metadata=0x12, priority 0, cookie 0x73face9d
resubmit(,21)
21. metadata=0x12, priority 0, cookie 0x8e46634d
resubmit(,22)
22. metadata=0x12, priority 0, cookie 0xbddac461
resubmit(,23)
23. metadata=0x12, priority 0, cookie 0x9a32c0de
resubmit(,24)
24. metadata=0x12, priority 0, cookie 0x79b1c074
resubmit(,25)
25. metadata=0x12, priority 0, cookie 0xc02374d3
resubmit(,26)
26. metadata=0x12, priority 0, cookie 0x218dc750
resubmit(,27)
27. metadata=0x12, priority 0, cookie 0xf6943631
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,28)
28. ip,reg0=0x2/0x2002,metadata=0x12, priority 100, cookie 0x7fdca4fb
ct(commit,zone=NXM_NX_REG13[0..15],nat(src),exec(set_field:0/0x1->ct_mark))
nat(src)
set_field:0/0x1->ct_mark
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
resubmit(,29)
29. metadata=0x12, priority 0, cookie 0x34e5fac7
resubmit(,30)
30. metadata=0x12, priority 0, cookie 0xac3f53bd
resubmit(,31)
31. metadata=0x12, priority 0, cookie 0x54000bd5
resubmit(,32)
32. metadata=0x12, priority 0, cookie 0x29ab49f3
resubmit(,33)
33. metadata=0x12, priority 0, cookie 0xae6aefcb
resubmit(,34)
34. metadata=0x12, priority 0, cookie 0x632dc93b
resubmit(,35)
35. metadata=0x12,dl_dst=1a:16:b1:58:e1:cd, priority 50, cookie 0x3fa33935
set_field:0x1->reg15
resubmit(,37)
37. priority 0
resubmit(,39)
39. priority 0
resubmit(,40)
40. reg15=0x1,metadata=0x12, priority 100, cookie 0x95c785db
set_field:0x155->reg11
set_field:0x1cf->reg12
resubmit(,41)
41. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,42)
42. ip,reg15=0x1,metadata=0x12, priority 110, cookie 0xc9a79824
resubmit(,43)
43. ip,reg15=0x1,metadata=0x12, priority 110, cookie 0xac7d5d78
resubmit(,44)
44. metadata=0x12, priority 0, cookie 0xa1ddb4f6
resubmit(,45)
45. ct_state=-trk,metadata=0x12, priority 5, cookie 0xb2622a65
set_field:0x100000000000000000000000000/0x100000000000000000000000000->xxreg0
set_field:0x200000000000000000000000000/0x200000000000000000000000000->xxreg0
resubmit(,46)
46. metadata=0x12, priority 0, cookie 0x267ae0e3
resubmit(,47)
47. metadata=0x12, priority 0, cookie 0x914392c3
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,48)
48. metadata=0x12, priority 0, cookie 0xc4f6a97f
resubmit(,49)
49. metadata=0x12, priority 0, cookie 0x8ebb0c71
resubmit(,50)
50. metadata=0x12, priority 0, cookie 0x61e385f4
resubmit(,51)
51. metadata=0x12, priority 0, cookie 0x4c722ce1
set_field:0/0x1000->reg10
resubmit(,75)
75. No match.
drop
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,52)
52. metadata=0x12, priority 0, cookie 0x3c01b89d
resubmit(,64)
64. priority 0
resubmit(,65)
65. reg15=0x1,metadata=0x12, priority 100, cookie 0x95c785db
clone(ct_clear,set_field:0->reg11,set_field:0->reg12,set_field:0->reg13,set_field:0x4cc->reg11,set_field:0x383->reg12,set_field:0x13->metadata,set_field:0x1->reg14,set_field:0->reg10,set_field:0->reg15,set_field:0->reg0,set_field:0->reg1,set_field:0->reg2,set_field:0->reg3,set_field:0->reg4,set_field:0->reg5,set_field:0->reg6,set_field:0->reg7,set_field:0->reg8,set_field:0->reg9,resubmit(,8))
ct_clear
set_field:0->reg11
set_field:0->reg12
set_field:0->reg13
set_field:0x4cc->reg11
set_field:0x383->reg12
set_field:0x13->metadata
set_field:0x1->reg14
set_field:0->reg10
set_field:0->reg15
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,8)
8. reg14=0x1,metadata=0x13,dl_dst=1a:16:b1:58:e1:cd, priority 50, cookie
0x8522832c
set_field:0x1a16b158e1cd0000000000000000/0xffffffffffff0000000000000000->xxreg0
resubmit(,9)
9. metadata=0x13, priority 0, cookie 0xef4dd8a9
set_field:0x4/0x4->xreg4
resubmit(,10)
10. reg9=0x4/0x4,metadata=0x13, priority 100, cookie 0x37718dd7
resubmit(,79)
79.
ip,reg14=0x1,metadata=0x13,dl_src=4e:42:14:a1:2a:fb,nw_src=172.27.18.244,
priority 100, cookie 0x2e544767
drop
resubmit(,11)
11. metadata=0x13, priority 0, cookie 0x3c2eb1b
resubmit(,12)
12. metadata=0x13, priority 0, cookie 0x44f62446
resubmit(,13)
13. metadata=0x13, priority 0, cookie 0x3842bec6
resubmit(,14)
14. metadata=0x13, priority 0, cookie 0xfd7b2ab9
resubmit(,15)
15. metadata=0x13, priority 0, cookie 0xefbd7e27
resubmit(,16)
16. metadata=0x13, priority 0, cookie 0xf439d853
resubmit(,17)
17. metadata=0x13, priority 0, cookie 0x123f01f0
resubmit(,18)
18. metadata=0x13, priority 0, cookie 0x142cd59b
resubmit(,19)
19. metadata=0x13, priority 0, cookie 0x297e0190
resubmit(,20)
20. metadata=0x13, priority 0, cookie 0x61e9e3c7
set_field:0/0xffffffff->xxreg1
resubmit(,21)
21. ip,reg7=0,metadata=0x13, priority 1, cookie 0x9f114f3d
dec_ttl()
set_field:0/0xffff00000000->xreg4
set_field:0x64646401000000000000000000000000/0xffffffff000000000000000000000000->xxreg0
set_field:0x646464030000000000000000/0xffffffff0000000000000000->xxreg0
set_field:2a:d5:d1:e8:89:cc->eth_src
set_field:0x2->reg15
set_field:0x1/0x1->reg10
resubmit(,22)
22. reg8=0/0xffff,metadata=0x13, priority 150, cookie 0x6ece899a
resubmit(,23)
23. metadata=0x13, priority 0, cookie 0x72437536
set_field:0/0xffff00000000->xreg4
resubmit(,24)
24. reg8=0/0xffff,metadata=0x13, priority 150, cookie 0xce06f1
resubmit(,25)
25. ip,metadata=0x13, priority 1, cookie 0xf690342c
push:NXM_NX_REG0[]
push:NXM_NX_XXREG0[96..127]
pop:NXM_NX_REG0[]
-> NXM_NX_REG0[] is now 0x64646401
set_field:00:00:00:00:00:00->eth_dst
resubmit(,66)
66. reg0=0x64646401,reg15=0x2,metadata=0x13, priority 100, cookie 0x97b4353d
set_field:00:00:5e:00:01:ff->eth_dst
set_field:0x40/0x40->reg10
pop:NXM_NX_REG0[]
-> NXM_NX_REG0[] is now 0x64646401
resubmit(,26)
26. metadata=0x13, priority 0, cookie 0x6c2ad4cc
resubmit(,27)
27. metadata=0x13, priority 0, cookie 0xadc9fb4f
resubmit(,28)
28. ip,reg15=0x2,metadata=0x13,nw_src=172.27.18.244, priority 100, cookie
0xfa923492
set_field:4e:42:14:a1:2a:fb->eth_src
set_field:0xcc3418740000000000000000/0xffffffff0000000000000000->xxreg0
resubmit(,29)
29. metadata=0x13, priority 0, cookie 0x571bbf77
resubmit(,37)
37. priority 0
resubmit(,39)
39. priority 0
resubmit(,40)
40. reg15=0x2,metadata=0x13, priority 100, cookie 0x407086d0
set_field:0x4cc->reg11
set_field:0x383->reg12
resubmit(,41)
41. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,42)
42. metadata=0x13, priority 0, cookie 0x4c3b7fcc
set_field:0/0x10->xreg4
resubmit(,43)
43. ip,reg15=0x2,metadata=0x13,nw_src=172.27.18.244, priority 100, cookie
0x540f90b2
set_field:4e:42:14:a1:2a:fb->eth_src
ct(table=44,zone=NXM_NX_REG11[0..15],nat)
nat
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 44.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
recirc_id=0x13b5a,eth,tcp,reg0=0x300,reg11=0x155,reg12=0x1cf,reg13=0x16d,reg14=0xca,reg15=0x1,metadata=0x12,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
Megaflow:
recirc_id=0x13b5a,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,tcp,in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=1a:16:b1:58:e1:cd,nw_src=172.27.18.244,nw_dst=104.0.0.0/5,nw_ttl=32,nw_frag=no,tp_src=0x8000/0x8000,tp_dst=0x180/0xffc0
Datapath actions:
ct(commit,zone=365,mark=0/0x1,nat(src)),set(eth(dst=00:00:5e:00:01:ff)),set(ipv4(ttl=31)),ct(zone=1228,nat),recirc(0x13b5c)
===============================================================================
recirc(0x13b5c) - resume conntrack with default ct_state=trk|new (use --ct-next
to customize)
Replacing src/dst IP/ports to simulate NAT:
Initial flow:
Modified flow:
===============================================================================
Flow:
recirc_id=0x13b5c,ct_state=new|trk,ct_zone=1228,eth,tcp,reg10=0x41,reg11=0x4cc,reg12=0x383,reg14=0x1,reg15=0x2,metadata=0x13,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=31,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 44
44. metadata=0x13, priority 0, cookie 0x48ef0c42
resubmit(,45)
45. ct_state=-rpl+trk,ip,reg15=0x2,metadata=0x13,nw_src=172.27.18.244, priority
161, cookie 0x5414262a
set_field:4e:42:14:a1:2a:fb->eth_src
ct(commit,table=46,zone=NXM_NX_REG12[0..15],nat(src=204.52.24.116))
nat(src=204.52.24.116)
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 46.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
recirc_id=0x13b5c,eth,tcp,reg10=0x41,reg11=0x4cc,reg12=0x383,reg14=0x1,reg15=0x2,metadata=0x13,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_src=172.27.18.244,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=31,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
Megaflow:
recirc_id=0x13b5c,ct_state=-rpl+trk,eth,ip,in_port=7753,dl_src=4e:42:14:a1:2a:fb,nw_src=172.27.18.244,nw_frag=no
Datapath actions: ct(commit,zone=899,nat(src=204.52.24.116)),recirc(0x25f9b)
===============================================================================
recirc(0x25f9b) - resume conntrack with default ct_state=trk|new (use --ct-next
to customize)
Replacing src/dst IP/ports to simulate NAT:
Initial flow: nw_src=172.27.18.244,tp_src=52776,nw_dst=104.18.3.35,tp_dst=443
Modified flow: nw_src=204.52.24.116,tp_src=52776,nw_dst=104.18.3.35,tp_dst=443
===============================================================================
Flow:
recirc_id=0x25f9b,ct_state=new|trk,ct_zone=899,eth,tcp,reg10=0x41,reg11=0x4cc,reg12=0x383,reg14=0x1,reg15=0x2,metadata=0x13,in_port=7753,vlan_tci=0x0000,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_src=204.52.24.116,nw_dst=104.18.3.35,nw_tos=0,nw_ecn=0,nw_ttl=31,nw_frag=no,tp_src=52776,tp_dst=443,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 46
46. metadata=0x13, priority 0, cookie 0x3800cb88
resubmit(,47)
47. metadata=0x13, priority 0, cookie 0xc2575be2
resubmit(,48)
48. reg15=0x2,metadata=0x13, priority 100, cookie 0xaf19a22a
resubmit(,64)
64. reg10=0x1/0x1,reg15=0x2,metadata=0x13, priority 100, cookie 0x407086d0
push:NXM_OF_IN_PORT[]
set_field:ANY->in_port
resubmit(,65)
65. reg15=0x2,metadata=0x13, priority 100, cookie 0x407086d0
clone(ct_clear,set_field:0->reg11,set_field:0->reg12,set_field:0->reg13,set_field:0x2c2->reg11,set_field:0x327->reg12,set_field:0x1->metadata,set_field:0xa->reg14,set_field:0->reg10,set_field:0->reg15,set_field:0->reg0,set_field:0->reg1,set_field:0->reg2,set_field:0->reg3,set_field:0->reg4,set_field:0->reg5,set_field:0->reg6,set_field:0->reg7,set_field:0->reg8,set_field:0->reg9,resubmit(,8))
ct_clear
set_field:0->reg11
set_field:0->reg12
set_field:0->reg13
set_field:0x2c2->reg11
set_field:0x327->reg12
set_field:0x1->metadata
set_field:0xa->reg14
set_field:0->reg10
set_field:0->reg15
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,8)
8. metadata=0x1, priority 50, cookie 0x1645d3f2
set_field:0/0x1000->reg10
resubmit(,73)
73. No match.
drop
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,9)
9. metadata=0x1, priority 0, cookie 0xcc4fd106
resubmit(,10)
10. metadata=0x1, priority 0, cookie 0x9e10ad0e
resubmit(,11)
11. metadata=0x1, priority 0, cookie 0x557f3249
resubmit(,12)
12. metadata=0x1, priority 0, cookie 0x915c56b1
resubmit(,13)
13. ip,reg14=0xa,metadata=0x1, priority 110, cookie 0xccdcd3e1
resubmit(,14)
14. metadata=0x1, priority 0, cookie 0x134ce32f
resubmit(,15)
15. metadata=0x1, priority 65535, cookie 0x49627e5f
resubmit(,16)
16. metadata=0x1, priority 65535, cookie 0xc947843d
resubmit(,17)
17. metadata=0x1, priority 0, cookie 0xd2fb4d2
resubmit(,18)
18. metadata=0x1, priority 0, cookie 0x62454929
resubmit(,19)
19. metadata=0x1, priority 0, cookie 0x3bb47080
resubmit(,20)
20. metadata=0x1, priority 0, cookie 0x73face9d
resubmit(,21)
21. metadata=0x1, priority 0, cookie 0x8e46634d
resubmit(,22)
22. metadata=0x1, priority 0, cookie 0xbddac461
resubmit(,23)
23. metadata=0x1, priority 0, cookie 0x9a32c0de
resubmit(,24)
24. metadata=0x1, priority 0, cookie 0x79b1c074
resubmit(,25)
25. metadata=0x1, priority 0, cookie 0xc02374d3
resubmit(,26)
26. metadata=0x1, priority 0, cookie 0x218dc750
resubmit(,27)
27. metadata=0x1, priority 0, cookie 0x8db4eebc
resubmit(,28)
28. metadata=0x1, priority 0, cookie 0x6deecbbe
resubmit(,29)
29. metadata=0x1, priority 0, cookie 0x34e5fac7
resubmit(,30)
30. metadata=0x1, priority 0, cookie 0xac3f53bd
resubmit(,31)
31. metadata=0x1, priority 0, cookie 0x54000bd5
resubmit(,32)
32. metadata=0x1, priority 0, cookie 0x29ab49f3
resubmit(,33)
33. metadata=0x1, priority 0, cookie 0xae6aefcb
resubmit(,34)
34. metadata=0x1, priority 0, cookie 0x632dc93b
resubmit(,35)
35. metadata=0x1, priority 0, cookie 0x9961115c
set_field:0->reg15
resubmit(,71)
71. No match.
drop
resubmit(,36)
36. reg15=0,metadata=0x1, priority 50, cookie 0x62255993
set_field:0x8001->reg15
resubmit(,37)
37. priority 0
resubmit(,39)
39. priority 0
resubmit(,40)
40. reg15=0x8001,metadata=0x1, priority 100, cookie 0x11699429
set_field:0x14->reg13
set_field:0x1->reg15
resubmit(,41)
41. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,42)
42. metadata=0x1, priority 0, cookie 0x9ad1480d
resubmit(,43)
43. ip,reg15=0x1,metadata=0x1, priority 110, cookie 0xca4977d3
ct_clear
resubmit(,44)
44. metadata=0x1, priority 0, cookie 0xa1ddb4f6
resubmit(,45)
45. metadata=0x1, priority 65535, cookie 0x91b4350e
resubmit(,46)
46. metadata=0x1, priority 65535, cookie 0xfc9c351d
resubmit(,47)
47. metadata=0x1, priority 0, cookie 0xb984696e
resubmit(,48)
48. metadata=0x1, priority 0, cookie 0xc4f6a97f
resubmit(,49)
49. metadata=0x1, priority 0, cookie 0x8ebb0c71
resubmit(,50)
50. metadata=0x1, priority 0, cookie 0x61e385f4
resubmit(,51)
51. metadata=0x1, priority 0, cookie 0x4c722ce1
set_field:0/0x1000->reg10
resubmit(,75)
75. No match.
drop
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,52)
52. metadata=0x1, priority 0, cookie 0x3c01b89d
resubmit(,64)
64. priority 0
resubmit(,65)
65. reg15=0x1,metadata=0x1, priority 100, cookie 0x4581da82
push_vlan:0x8100
set_field:4216->vlan_vid
output:7754
bridge("br-provider")
---------------------
0. priority 0
NORMAL
-> forwarding to learned port
pop_vlan
set_field:0x8001->reg15
pop:NXM_OF_IN_PORT[]
-> NXM_OF_IN_PORT[] is now 7753
Final flow: unchanged
Megaflow:
recirc_id=0x25f9b,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,ip,in_port=7753,dl_src=4e:42:14:a1:2a:fb,dl_dst=00:00:5e:00:01:ff,nw_dst=0.0.0.0/1,nw_frag=no
Datapath actions: ct_clear,push_vlan(vid=120,pcp=0),5
1702601116163628 [swapper/140] 0 [tp] net:napi_gro_receive_entry
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 0
if 21 (enp148s0f0_1)
+ 1702601116170733 [swapper/140] 0 [k] tcp_gro_receive
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 1
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.] seq 792060930:792061367
ack 951229219 win 11
+ 1702601116174841 [swapper/140] 0 [tp] net:netif_receive_skb
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 2
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.] seq 792060930:792061367
ack 951229219 win 11
+ 1702601116185568 [swapper/140] 0 [tp] skb:kfree_skb
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 3 drop (TC_INGRESS)
if 21 (enp148s0f0_1) rxif 21 172.27.18.244.57394 > 104.18.2.35.443 ttl 63
tos 0x0 id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.] seq
792060930:792061367 ack 951229219 win 11
1702602293157037 [swapper/140] 0 [tp] openvswitch:ovs_dp_upcall
#60c81fce09cadff284fb46965ec00 (skb 18386033671255364352) n 0
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos
0x0 id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack
792060930 win 8
upcall (miss) port 3578259683 cpu 140
+ 1702602293170012 [swapper/140] 0 [kr] queue_userspace_packet
#60c81fce09cadff284fb46965ec00 (skb 18386033671255364352) n 1
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack
792060930 win 8
upcall_enqueue (miss) (140/1702602293157037) q 3568384553 ret 0
+ 1702602293173687 [swapper/140] 0 [kr] ovs_dp_upcall
#60c81fce09cadff284fb46965ec00 (skb 18386033671255364352) n 2
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack
792060930 win 8
upcall_ret (140/1702602293157037) ret 0
1702602293879756 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 0
if 4 (enp148s0f0np0) 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos 0x0 id
9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930 win
8
exec pop_vlan
+ 1702602293883747 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 1
if 4 (enp148s0f0np0) 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec ct zone 899 nat
+ 1702602293890476 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 2
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec recirc 0x27c5d
:...skipping...
1702601116163628 [swapper/140] 0 [tp] net:napi_gro_receive_entry
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 0
if 21 (enp148s0f0_1)
+ 1702601116170733 [swapper/140] 0 [k] tcp_gro_receive
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 1
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.] seq 792060930:792061367
ack 951229219 win 11
+ 1702601116174841 [swapper/140] 0 [tp] net:netif_receive_skb
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 2
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.] seq 792060930:792061367
ack 951229219 win 11
+ 1702601116185568 [swapper/140] 0 [tp] skb:kfree_skb
#60c81b6b91e2cff284fb3a3d65800 (skb 18386033671255367680) n 3 drop (TC_INGRESS)
if 21 (enp148s0f0_1) rxif 21 172.27.18.244.57394 > 104.18.2.35.443 ttl 63
tos 0x0 id 26162 off 0 [DF] len 477 proto TCP (6) flags [P.] seq
792060930:792061367 ack 951229219 win 11
1702602293157037 [swapper/140] 0 [tp] openvswitch:ovs_dp_upcall
#60c81fce09cadff284fb46965ec00 (skb 18386033671255364352) n 0
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos
0x0 id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack
792060930 win 8
upcall (miss) port 3578259683 cpu 140
+ 1702602293170012 [swapper/140] 0 [kr] queue_userspace_packet
#60c81fce09cadff284fb46965ec00 (skb 18386033671255364352) n 1
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack
792060930 win 8
upcall_enqueue (miss) (140/1702602293157037) q 3568384553 ret 0
+ 1702602293173687 [swapper/140] 0 [kr] ovs_dp_upcall
#60c81fce09cadff284fb46965ec00 (skb 18386033671255364352) n 2
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack
792060930 win 8
upcall_ret (140/1702602293157037) ret 0
1702602293879756 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 0
if 4 (enp148s0f0np0) 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos 0x0 id
9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930 win
8
exec pop_vlan
+ 1702602293883747 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 1
if 4 (enp148s0f0np0) 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec ct zone 899 nat
+ 1702602293890476 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 2
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec recirc 0x27c5d
+ 1702602293895884 [handler6] 1698151/1698158 [tp] openvswitch:ovs_dp_upcall
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 3
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
upcall (miss) port 2391576636 cpu 48
+ 1702602293910051 [handler6] 1698151/1698158 [kr] queue_userspace_packet
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 4
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
upcall_enqueue (miss) (48/1702602293895884) q 1435855212 ret 0
+ 1702602293915167 [handler6] 1698151/1698158 [kr] ovs_dp_upcall
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 5
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
upcall_ret (48/1702602293895884) ret 0
+ 1702602293943357 [handler16] 1698151/1698168 [u] dpif_recv:recv_upcall
(ovs-vswitchd) #60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 6
upcall_recv q 1435855212 pkt_size 54
+ 1702602294637372 [handler16] 1698151/1698168 [u]
dpif_netlink_operate__:op_flow_execute (ovs-vswitchd)
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 7
flow_exec q 1435855212 ts 1702602293943357 (0)
+ 1702602294663927 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 8
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec ct_clear q 1435855212
+ 1702602294668918 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 9
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec set_masked q 1435855212
+ 1702602294670734 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 10
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 55 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec set_masked q 1435855212
+ 1702602294672276 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 11
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec ct zone 365 q 1435855212
+ 1702602294678162 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 12
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec recirc 0x13b59 q 1435855212
+ 1702602294683435 [handler16] 1698151/1698168 [tp] openvswitch:ovs_dp_upcall
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 13
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
upcall (miss) port 3738662439 cpu 50
+ 1702602294700977 [handler16] 1698151/1698168 [kr] queue_userspace_packet
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 14
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
upcall_enqueue (miss) (50/1702602294683435) q 3693000559 ret 0
+ 1702602294706978 [handler16] 1698151/1698168 [kr] ovs_dp_upcall
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 15
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
upcall_ret (50/1702602294683435) ret 0
+ 1702602294725719 [handler18] 1698151/1698170 [u] dpif_recv:recv_upcall
(ovs-vswitchd) #60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 16
upcall_recv q 3693000559 pkt_size 54
+ 1702602294788084 [handler18] 1698151/1698170 [u]
dpif_netlink_operate__:op_flow_execute (ovs-vswitchd)
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 17
flow_exec q 3693000559 ts 1702602294725719 (0)
+ 1702602294804515 [handler18] 1698151/1698170 [tp]
openvswitch:ovs_do_execute_action #60c81fceb7e66ff2850367a5ad400 (skb
18386033667193641216) n 18
if 4 (enp148s0f0np0) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
exec oport 6 q 3693000559
+ 1702602294807333 [handler18] 1698151/1698170 [tp] net:net_dev_queue
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 19
if 21 (enp148s0f0_1) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
+ 1702602294811422 [handler18] 1698151/1698170 [tp] net:net_dev_start_xmit
#60c81fceb7e66ff2850367a5ad400 (skb 18386033667193641216) n 20
if 21 (enp148s0f0_1) 104.18.2.35.443 > 172.27.18.244.57394 ttl 54 tos 0x0
id 9224 off 0 [DF] len 40 proto TCP (6) flags [F.] seq 951229219 ack 792060930
win 8
1702602294967663 [swapper/140] 0 [tp] net:napi_gro_receive_entry
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 0
if 21 (enp148s0f0_1)
+ 1702602294969582 [swapper/140] 0 [k] tcp_gro_receive
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 1
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 64 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
+ 1702602294970886 [swapper/140] 0 [tp] net:netif_receive_skb
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 2
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 64 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
+ 1702602294984686 [swapper/140] 0 [tp] openvswitch:ovs_dp_upcall
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 3
if 21 (enp148s0f0_1) rxif 21 172.27.18.244.57394 > 104.18.2.35.443 ttl 63
tos 0x0 id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq
792061367:792061374 ack 951229220 win 11
upcall (miss) port 3578259683 cpu 140
+ 1702602294990942 [swapper/140] 0 [kr] queue_userspace_packet
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 4
if 21 (enp148s0f0_1) rxif 21 172.27.18.244.57394 > 104.18.2.35.443 ttl 63
tos 0x0 id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq
792061367:792061374 ack 951229220 win 11
upcall_enqueue (miss) (140/1702602294984686) q 1205013910 ret 0
+ 1702602294993198 [swapper/140] 0 [kr] ovs_dp_upcall
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 5
if 21 (enp148s0f0_1) rxif 21 172.27.18.244.57394 > 104.18.2.35.443 ttl 63
tos 0x0 id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq
792061367:792061374 ack 951229220 win 11
upcall_ret (140/1702602294984686) ret 0
+ 1702602295004512 [handler6] 1698151/1698158 [u] dpif_recv:recv_upcall
(ovs-vswitchd) #60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 6
upcall_recv q 1205013910 pkt_size 61
+ 1702602295896167 [handler6] 1698151/1698158 [u]
dpif_netlink_operate__:op_flow_execute (ovs-vswitchd)
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 7
flow_exec q 1205013910 ts 1702602295004512 (0)
+ 1702602295903448 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fcfc3d6fff284fb5b8c24000 (skb
18386033669760228352) n 8
if 21 (enp148s0f0_1) 172.27.18.244.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
exec ct zone 899 nat(src=204.52.24.116) commit q 1205013910
+ 1702602295906484 [handler6] 1698151/1698158 [tp]
openvswitch:ovs_do_execute_action #60c81fcfc3d6fff284fb5b8c24000 (skb
18386033669760228352) n 9
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
exec recirc 0x27c89 q 1205013910
+ 1702602295908883 [handler6] 1698151/1698158 [tp] openvswitch:ovs_dp_upcall
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 10
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
upcall (miss) port 2391576636 cpu 48
+ 1702602295914051 [handler6] 1698151/1698158 [kr] queue_userspace_packet
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 11
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
upcall_enqueue (miss) (48/1702602295908883) q 401876464 ret 0
+ 1702602295916476 [handler6] 1698151/1698158 [kr] ovs_dp_upcall
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 12
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
upcall_ret (48/1702602295908883) ret 0
+ 1702602295930315 [handler16] 1698151/1698168 [u] dpif_recv:recv_upcall
(ovs-vswitchd) #60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 13
upcall_recv q 401876464 pkt_size 61
+ 1702602296476525 [handler16] 1698151/1698168 [u]
dpif_netlink_operate__:op_flow_execute (ovs-vswitchd)
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 14
flow_exec q 401876464 ts 1702602295930315 (0)
+ 1702602296494668 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fcfc3d6fff284fb5b8c24000 (skb
18386033669760228352) n 15
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
exec ct_clear q 401876464
+ 1702602296497195 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fcfc3d6fff284fb5b8c24000 (skb
18386033669760228352) n 16
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
exec push_vlan q 401876464
+ 1702602296498561 [handler16] 1698151/1698168 [tp]
openvswitch:ovs_do_execute_action #60c81fcfc3d6fff284fb5b8c24000 (skb
18386033669760228352) n 17
if 21 (enp148s0f0_1) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
exec oport 5 q 401876464
+ 1702602296500986 [handler16] 1698151/1698168 [tp] net:net_dev_queue
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 18
if 4 (enp148s0f0np0) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
+ 1702602296509824 [handler16] 1698151/1698168 [tp] net:net_dev_start_xmit
#60c81fcfc3d6fff284fb5b8c24000 (skb 18386033669760228352) n 19
if 4 (enp148s0f0np0) 204.52.24.116.57394 > 104.18.2.35.443 ttl 63 tos 0x0
id 26163 off 0 [DF] len 47 proto TCP (6) flags [FP.] seq 792061367:792061374
ack 951229220 win 11
1702602301915755 [swapper/140] 0 [tp] net:napi_gro_receive_entry
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 0
if 4 (enp148s0f0np0)
+ 1702602301918876 [swapper/140] 0 [k] tcp_gro_receive
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 1
if 4 (enp148s0f0np0) 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos 0x0
id 0 off 0 [DF] len 40 proto TCP (6) flags [R] seq 951229220 win 0
+ 1702602301920027 [swapper/140] 0 [tp] net:netif_receive_skb
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 2
if 4 (enp148s0f0np0) 104.18.2.35.443 > 204.52.24.116.57394 ttl 55 tos 0x0
id 0 off 0 [DF] len 40 proto TCP (6) flags [R] seq 951229220 win 0
+ 1702602301932030 [swapper/140] 0 [tp] openvswitch:ovs_dp_upcall
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 3
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 0 off 0 [DF] len 40 proto TCP (6) flags [R] seq 951229220 win 0
upcall (miss) port 3578259683 cpu 140
+ 1702602301938837 [swapper/140] 0 [kr] queue_userspace_packet
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 4
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 0 off 0 [DF] len 40 proto TCP (6) flags [R] seq 951229220 win 0
upcall_enqueue (miss) (140/1702602301932030) q 3540092838 ret 0
+ 1702602301941536 [swapper/140] 0 [kr] ovs_dp_upcall
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 5
if 4 (enp148s0f0np0) rxif 4 104.18.2.35.443 > 204.52.24.116.57394 ttl 55
tos 0x0 id 0 off 0 [DF] len 40 proto TCP (6) flags [R] seq 951229220 win 0
upcall_ret (140/1702602301932030) ret 0
+ 1702602301950272 [handler6] 1698151/1698158 [u] dpif_recv:recv_upcall
(ovs-vswitchd) #60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 6
upcall_recv q 3540092838 pkt_size 54
+ 1702607698744783 [handler6] 1698151/1698158 [u]
dpif_netlink_operate__:op_flow_execute (ovs-vswitchd)
#60c81fd66426bff284fb46965e400 (skb 18386033661279436544) n 7
flow_exec q 3540092838 ts 1702602301950272 (0)_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
