It does seem to be some sort of offload/MTU issue. If I increase the interfaces' MTU, I get significantly better results:
$ ip netns exec IPv4 ip link set ipv4_to_jool mtu 65535 $ ip netns exec nsJool ip link set to_ipv4 mtu 65535 $ ip netns exec nsJool ip link set to_ipv6 mtu 65535 $ ip netns exec IPv6 ip link set ipv6_to_jool mtu 65535 $ iperf -B 172.17.1.2 -c 198.10.10.2 -i 1 -t 2 ------------------------------------------------------------ Client connecting to 198.10.10.2, TCP port 5001 Binding to local address 172.17.1.2 TCP window size: 2.50 MByte (default) ------------------------------------------------------------ [ 3] local 172.17.1.2 port 59082 connected with 198.10.10.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 679 MBytes 5.69 Gbits/sec [ 3] 1.0- 2.0 sec 662 MBytes 5.55 Gbits/sec [ 3] 0.0- 2.0 sec 1.31 GBytes 5.61 Gbits/sec I'm still investigating. On Tue, Jul 17, 2018 at 12:36 PM Alberto Leiva <[email protected]> wrote: > > Muhammad: > > I might have successfully reproduced the problem. Can you please > confirm if these are the kinds of numbers that you're seeing? > > ------------------------------------------------------------ > Client connecting to 198.10.10.2, TCP port 5001 > Binding to local address 172.17.1.2 > TCP window size: 85.0 KByte (default) > ------------------------------------------------------------ > [ 3] local 172.17.1.2 port 5001 connected with 198.10.10.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0- 1.0 sec 384 KBytes 3.15 Mbits/sec > [ 3] 1.0- 2.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 4.0- 5.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 5.0- 6.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 6.0- 7.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 7.0- 8.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 8.0- 9.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 9.0-10.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 10.0-11.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 11.0-12.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 12.0-13.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 13.0-14.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 14.0-15.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 15.0-16.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 16.0-17.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 17.0-18.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 18.0-19.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 19.0-20.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 20.0-21.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 21.0-22.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 22.0-23.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 23.0-24.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 24.0-25.0 sec 128 KBytes 1.05 Mbits/sec > [ 3] 25.0-26.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 26.0-27.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 27.0-28.0 sec 384 KBytes 3.15 Mbits/sec > [ 3] 28.0-29.0 sec 384 KBytes 3.15 Mbits/sec > [ 3] 29.0-30.0 sec 256 KBytes 2.10 Mbits/sec > [ 3] 0.0-30.3 sec 6.38 MBytes 1.76 Mbits/sec > On Mon, Jul 16, 2018 at 3:00 PM <[email protected]> wrote: > > > > Hi Alberto and Tore, > > > > Thanks for your feedback. > > > > I've done removed the modules using following > > > > #rmmod jool > > #rmmod jool_siit > > > > Re inserted only jool_siit module using following (In global namespace) > > > > #modprobe jool_siit no_instance > > > > In Jool name space I did the following > > > > ip netns exec nsJool jool_siit --instance --add > > > > ip netns exec nsJool jool_siit --eamt --add 172.17.1.2 2001:db8::c60a:a03 > > ip netns exec nsJool jool_siit --eamt --add 198.10.10.2 2001:db8::c60a:a02 > > > > Note: Ping and connectivity bw IPv4 and IPv6 host works fine. > > > > Please find attached the "ethtool -k " output for interface. All the veth > > pairs have similar ethtool configurations. > > > > Also find attached traffic capture on IPv6 interface of nsJool. I've > > observed a lot of TCP out of order and retransmissions. > > > > Can you take a look at the configurations and provide your feedback on why > > could be there TCP out of order and retransmissions. > > > > Appreciate your help > > > > Thanks > > Muhammad Ali > > > > -----Original Message----- > > From: Alberto Leiva <[email protected]> > > Sent: Friday, July 13, 2018 12:09 AM > > To: [email protected] > > Cc: [email protected]; [email protected] > > Subject: Re: [Jool-list] Jool performance help > > > > Thanks, Tore! > > > > I would like to add the following: > > > > > $ modprobe jool > > > $ modprobe jool_siit > > > > Are you sure that this is what you want? > > > > I'm not sure why you would want to insert both modules in the same > > namespace. One is a SIIT and the other one is a NAT64. Particularly if > > you're performance-testing, I'd normally expect you to test one *or* the > > other. > > On Thu, Jul 12, 2018 at 2:09 AM Tore Anderson <[email protected]> wrote: > > > > > > * [email protected] > > > > > > > I’ve installed Jool kernel modules and userspace application on Ubuntu. > > > > jool_siit is running in a network namespace and I’m using veth pairs > > > > for network I/O. Please find attached details of my test environment. > > > > > > > > However, while running the TCP throughput test, I was able to achieve > > > > only 6Mbps of throughput. I’ve tested it by doing both *GRO on and off* > > > > on all the relevant interfaces with no performance improvements. > > > > > > > > We are evaluating Jool for carrier grade NAT64 in our network > > > > infrastructure. > > > > > > > > I was wondering if you can help me to improve performance results. Is > > > > there any tweaks or workarounds to overcome the performance limitation > > > > by Jool. > > > > > > Hi Muhammad, > > > > > > First off, you're definitively not hitting the performance limit of > > > Jool - it easily scales to multiple Gb/s of throughput. There must be > > > something else that is causing your issues. > > > > > > Even though you said you turned GRO off, my suspicion would be > > > something with packet sizes. Are there other offload settings you can > > > turn off? MTU settings on all the interfaces are all okay? > > > > > > Also, check with tcpdump on all the relevant interfaces to see if the > > > test traffic is causing lots of ICMP Frag Needed/Packet Too Big errors. > > > > > > Tore > > > _______________________________________________ > > > Jool-list mailing list > > > [email protected] > > > https://mail-lists.nic.mx/listas/listinfo/jool-list _______________________________________________ Jool-list mailing list [email protected] https://mail-lists.nic.mx/listas/listinfo/jool-list
