> Got it. > > Since I wanted to test both upstream and downstream with iperf3, I was > using -R option. > Even with disabling virtual-reassembly, packets are being dropped (see below). > > Switching server to 10.155.6.x and client on 10.155.3.x works. > So, for this kind of test, do you recommend switching client and > server subnets instead of running iperf3 with -R.
I’m recommending not triggering fragmentation. Fragments are NAT unfriendly. iperf with -l being sensible, as in not 8KB for TCP. Unless you have an 8KB+ MTU. The iperf authors have clearly not read our draft in intarea. ;-) Cheers, Ole > > here are the dropped packets: > > Packet 49 > > 00:19:41:823757: dpdk-input > GigabitEthernet5/0/0 rx queue 0 > buffer 0x2dd18: current data 0, length 834, free-list 0, clone-count > 0, totlen-nifb 0, trace 0x30 > ext-hdr-valid > l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 > PKT MBUF: port 3, nb_segs 1, pkt_len 834 > buf_len 2176, data_len 834, ol_flags 0x180, data_off 128, > phys_addr 0xe8b74680 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > UDP: 10.155.6.111 -> 10.155.3.201 > tos 0x00, ttl 64, length 820, checksum 0x843f > fragment id 0xd06f offset 7400, flags > UDP: 23507 -> 4311 > length 43450, checksum 0x6383 > 00:19:41:823759: ethernet-input > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > 00:19:41:823760: l2-input > l2-input: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > 00:19:41:823760: l2-learn > l2-learn: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac bd_index > 5 > 00:19:41:823760: l2-fwd > l2-fwd: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac bd_index > 5 > 00:19:41:823761: ip4-input > UDP: 10.155.6.111 -> 10.155.3.201 > tos 0x00, ttl 64, length 820, checksum 0x843f > fragment id 0xd06f offset 7400, flags > UDP: 23507 -> 4311 > length 43450, checksum 0x6383 > 00:19:41:823761: nat44-in2out > NAT44_IN2OUT_FAST_PATH: sw_if_index 19, next index 4, session -1 > 00:19:41:823761: nat44-in2out-reass > NAT44_REASS: sw_if_index 19, next index 1, status translated > 00:19:41:823762: error-drop > nat44-in2out-reass: Drop fragment > > Packet 50 > > 00:19:41:824581: dpdk-input > GigabitEthernet5/0/0 rx queue 0 > buffer 0x1b3a8: current data 0, length 1514, free-list 0, > clone-count 0, totlen-nifb 0, trace 0x31 > ext-hdr-valid > l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 > PKT MBUF: port 3, nb_segs 1, pkt_len 1514 > buf_len 2176, data_len 1514, ol_flags 0x180, data_off 128, > phys_addr 0xe86cea80 > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > Packet Offload Flags > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > Packet Types > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > UDP: 10.155.6.111 -> 10.155.3.201 > tos 0x00, ttl 64, length 1500, checksum 0x8700 > fragment id 0xaea3, flags MORE_FRAGMENTS > UDP: 5201 -> 47346 > length 8200, checksum 0x5570 > 00:19:41:824582: ethernet-input > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > 00:19:41:824583: l2-input > l2-input: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > 00:19:41:824583: l2-learn > l2-learn: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac bd_index > 5 > 00:19:41:824584: l2-fwd > l2-fwd: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac bd_index > 5 > 00:19:41:824584: ip4-input > UDP: 10.155.6.111 -> 10.155.3.201 > tos 0x00, ttl 64, length 1500, checksum 0x8700 > fragment id 0xaea3, flags MORE_FRAGMENTS > UDP: 5201 -> 47346 > length 8200, checksum 0x5570 > 00:19:41:824584: nat44-in2out > NAT44_IN2OUT_FAST_PATH: sw_if_index 19, next index 4, session -1 > 00:19:41:824585: nat44-in2out-reass > NAT44_REASS: sw_if_index 19, next index 1, status translated > 00:19:41:824585: error-drop > nat44-in2out-reass: Drop fragment > > Thanks > > On Sun, Mar 3, 2019 at 11:35 PM Ole Troan <[email protected]> wrote: >> >> Hi Carlito, >> >> Seems like you are sending IP fragments. >> Those need to be (virtually) reassembled before NATted. Reassembly is to a >> large extent an attack vector, and it’s rate limited. >> >> cheers, >> Ole >> >>> On 3 Mar 2019, at 22:46, carlito nueno <[email protected]> wrote: >>> >>> Hi all, >>> >>> While running more iperf3 udp tests, I noticed vpp status showing this: >>> >>> My current vpp conf: >>> https://gist.github.com/ironpillow/4b119b57e21b31a7ff6985bcb20f952b >>> >>> Setup to reproduce: >>> 1. iperf3 server on 10.155.3.2 (iperf3 -s -B 10.155.3.2) >>> 2. iperf3 client on 10.155.6.2 but with -R flag (iperf3 -B 10.155.6.2 >>> -c 10.155.3.2 -u -b0 -R) >>> >>> So basically, server on one subnet and client on another subnet and >>> run it with -R flag >>> >>> vpp.service - vector packet processing engine >>> Loaded: loaded (/lib/systemd/system/vpp.service; enabled; vendor >>> preset: enabled) >>> Active: active (running) since Fri 2019-03-01 17:09:24 PST; 18min ago >>> Process: 32079 ExecStopPost=/bin/rm -f /dev/shm/db >>> /dev/shm/global_vm /dev/shm/vpe-api (code=exited, status=0/SUCCESS) >>> Process: 32093 ExecStartPre=/sbin/modprobe uio_pci_generic >>> (code=exited, status=0/SUCCESS) >>> Process: 32087 ExecStartPre=/bin/rm -f /dev/shm/db >>> /dev/shm/global_vm /dev/shm/vpe-api (code=exited, status=0/SUCCESS) >>> Main PID: 32095 (vpp_main) >>> Tasks: 10 (limit: 4915) >>> CGroup: /system.slice/vpp.service >>> └─32095 /usr/bin/vpp -c /etc/vpp/startup.conf >>> >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot >>> Mar 01 17:20:17 test1 vnet[32095]: nat: --- message(s) throttled --- >>> >>> Thanks >>> -=-=-=-=-=-=-=-=-=-=-=- >>> Links: You receive all messages sent to this group. >>> >>> View/Reply Online (#12410): https://lists.fd.io/g/vpp-dev/message/12410 >>> Mute This Topic: https://lists.fd.io/mt/30206523/675193 >>> Group Owner: [email protected] >>> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] >>> -=-=-=-=-=-=-=-=-=-=-=- >> > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#12419): https://lists.fd.io/g/vpp-dev/message/12419 > Mute This Topic: https://lists.fd.io/mt/30206523/675193 > Group Owner: [email protected] > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] > -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12420): https://lists.fd.io/g/vpp-dev/message/12420 Mute This Topic: https://lists.fd.io/mt/30206523/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
