For responsiveness, the bounceback seems reasonable even with upstream
competition. Bunch more TCP retries though.
[rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e
--trip-times -i 1 --bounceback -t 3
------------------------------------------------------------
Client connecting to (**hidden**), TCP port 5001 with pid 111022 (1
flows)
Write buffer size: 100 Byte
Bursting: 100 Byte writes 10 times every 1.00 second(s)
Bounce-back test (size= 100 Byte) (server hold req=0 usecs &
tcp_quickack)
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 16.0 KByte (default)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[ 1] local *.*.*.86%enp7s0 port 36976 connected with *.*.*.123 port
5001 (prefetch=16384) (bb w/quickack len/hold=100/0) (trip-times)
(sock=3) (icwnd/mss/irtt=14/1448/9862) (ct=9.90 ms) on 2023-01-06
12:42:18 (PST)
[ ID] Interval Transfer Bandwidth BB
cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS
[ 1] 0.00-1.00 sec 1.95 KBytes 16.0 Kbits/sec
10=12.195/9.298/16.457/2.679 ms 0 14K/11327 us 82 rps
[ 1] 1.00-2.00 sec 1.95 KBytes 16.0 Kbits/sec
10=12.613/9.271/15.489/2.788 ms 0 14K/12165 us 79 rps
[ 1] 2.00-3.00 sec 1.95 KBytes 16.0 Kbits/sec
10=13.390/9.376/15.986/2.520 ms 0 14K/13164 us 75 rps
[ 1] 0.00-3.03 sec 5.86 KBytes 15.8 Kbits/sec
30=12.733/9.271/16.457/2.620 ms 0 14K/15138 us 79 rps
[ 1] 0.00-3.03 sec OWD Delays (ms) Cnt=30 To=7.937/4.634/11.327/2.457
From=4.778/4.401/5.350/0.258 Asymmetry=3.166/0.097/6.311/2.318 79 rps
[ 1] 0.00-3.03 sec BB8(f)-PDF:
bin(w=100us):cnt(30)=93:2,94:3,95:2,97:1,100:1,102:1,105:1,114:2,142:1,143:1,144:2,145:3,146:1,147:1,148:1,151:1,152:1,154:1,155:1,156:1,160:1,165:1
(5.00/95.00/99.7%=93/160/165,Outliers=0,obl/obu=0/0)
[rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e
--trip-times -i 1 --bounceback -t 3 --bounceback-congest=up,4
------------------------------------------------------------
Client connecting to (**hidden**), TCP port 5001 with pid 111069 (1
flows)
Write buffer size: 100 Byte
Bursting: 100 Byte writes 10 times every 1.00 second(s)
Bounce-back test (size= 100 Byte) (server hold req=0 usecs &
tcp_quickack)
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 16.0 KByte (default)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[ 2] local *.*.*.85%enp4s0 port 38342 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=3) (qack)
(icwnd/mss/irtt=14/1448/10613) (ct=10.66 ms) on 2023-01-06 12:42:36
(PST)
[ 1] local *.*.*.85%enp4s0 port 38360 connected with *.*.*.123 port
5001 (prefetch=16384) (bb w/quickack len/hold=100/0) (trip-times)
(sock=4) (icwnd/mss/irtt=14/1448/14901) (ct=14.96 ms) on 2023-01-06
12:42:36 (PST)
[ 3] local *.*.*.85%enp4s0 port 38386 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=7) (qack)
(icwnd/mss/irtt=14/1448/15295) (ct=15.31 ms) on 2023-01-06 12:42:36
(PST)
[ 4] local *.*.*.85%enp4s0 port 38348 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=5) (qack)
(icwnd/mss/irtt=14/1448/14901) (ct=14.95 ms) on 2023-01-06 12:42:36
(PST)
[ 5] local *.*.*.85%enp4s0 port 38372 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=6) (qack)
(icwnd/mss/irtt=14/1448/15371) (ct=15.42 ms) on 2023-01-06 12:42:36
(PST)
[ ID] Interval Transfer Bandwidth Write/Err Rtry
Cwnd/RTT(var) NetPwr
[ 3] 0.00-1.00 sec 1.29 MBytes 10.8 Mbits/sec 13502/0 115
28K/22594(904) us 59.76
[ 4] 0.00-1.00 sec 1.63 MBytes 13.6 Mbits/sec 17048/0 140
42K/22728(568) us 75.01
[ ID] Interval Transfer Bandwidth BB
cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS
[ 1] 0.00-1.00 sec 1.95 KBytes 16.0 Kbits/sec
10=76.140/17.224/123.195/43.168 ms 0 14K/68136 us 13 rps
[ 5] 0.00-1.00 sec 1.04 MBytes 8.72 Mbits/sec 10893/0 82
25K/23400(644) us 46.55
[SUM] 0.00-1.00 sec 3.95 MBytes 33.2 Mbits/sec 41443/0 337
[ 2] 0.00-1.00 sec 1.10 MBytes 9.25 Mbits/sec 11566/0 77
22K/23557(432) us 49.10
[ 3] 1.00-2.00 sec 1.24 MBytes 10.4 Mbits/sec 13037/0 20
28K/14427(503) us 90.37
[ 4] 1.00-2.00 sec 1.43 MBytes 12.0 Mbits/sec 14954/0 31
12K/13348(407) us 112
[ 1] 1.00-2.00 sec 1.95 KBytes 16.0 Kbits/sec
10=14.581/10.801/20.356/3.599 ms 0 14K/27791 us 69 rps
[ 5] 1.00-2.00 sec 1.26 MBytes 10.6 Mbits/sec 13191/0 16
12K/14749(675) us 89.44
[SUM] 1.00-2.00 sec 3.93 MBytes 32.9 Mbits/sec 41182/0 67
[ 2] 1.00-2.00 sec 1000 KBytes 8.19 Mbits/sec 10237/0 13
19K/14467(1068) us 70.76
[ 3] 2.00-3.00 sec 1.33 MBytes 11.2 Mbits/sec 13994/0 4
24K/20749(495) us 67.44
[ 4] 2.00-3.00 sec 1.20 MBytes 10.1 Mbits/sec 12615/0 3
31K/20877(718) us 60.43
[ 1] 2.00-3.00 sec 1.95 KBytes 16.0 Kbits/sec
10=11.298/9.407/14.245/1.330 ms 0 14K/15474 us 89 rps
[ 5] 2.00-3.00 sec 1.08 MBytes 9.03 Mbits/sec 11284/0 3
28K/21031(430) us 53.65
[SUM] 2.00-3.00 sec 3.61 MBytes 30.3 Mbits/sec 37893/0 10
[ 2] 2.00-3.00 sec 1.29 MBytes 10.8 Mbits/sec 13492/0 3
29K/20409(688) us 66.11
[ 3] 0.00-3.03 sec 3.87 MBytes 10.7 Mbits/sec 40534/0 139
25K/20645(557) us 64.85
[ 5] 0.00-3.03 sec 3.37 MBytes 9.35 Mbits/sec 35369/0 101
29K/20489(668) us 57.02
[ 4] 0.00-3.03 sec 4.26 MBytes 11.8 Mbits/sec 44618/0 174
32K/21240(961) us 69.40
[ 2] 0.00-3.03 sec 3.37 MBytes 9.31 Mbits/sec 35296/0 94
19K/21504(948) us 54.13
[ 1] 0.00-3.14 sec 7.81 KBytes 20.4 Kbits/sec
40=28.332/5.611/123.195/34.940 ms 0 14K/14000 us 35 rps
[ 1] 0.00-3.14 sec OWD Delays (ms) Cnt=40
To=23.730/1.110/118.744/34.957 From=4.567/4.356/5.171/0.141
Asymmetry=19.332/0.189/114.294/34.869 35 rps
[ 1] 0.00-3.14 sec BB8(f)-PDF:
bin(w=100us):cnt(40)=57:1,94:2,95:2,96:2,98:1,101:1,106:1,109:2,111:1,112:2,113:1,115:1,118:1,119:2,143:2,145:1,146:1,152:1,158:1,173:1,176:1,194:1,195:1,204:1,205:1,274:1,554:1,790:1,925:1,1125:1,1126:1,1225:1,1232:1
(5.00/95.00/99.7%=94/1225/1232,Outliers=0,obl/obu=0/0)
[SUM] 0.00-3.11 sec 11.5 MBytes 31.0 Mbits/sec 120521/0 414
[ CT] final connect times (min/avg/max/stdev) =
10.661/14.261/15.423/2023.369 ms (tot/err) = 5/0
Some thoughts are not to use UDP for testing here. Also, these speed
tests have little to no information for network engineers about what's
going on. Iperf 2 may better assist network engineers but then I'm
biased ;)
Running iperf 2 https://sourceforge.net/projects/iperf2/ with
--trip-times. Though the sampling and central limit theorem averaging
is hiding the real distributions (use --histograms to get those)
Below are 4 parallel TCP streams from my home to one of my servers in
the cloud. First where TCP is limited per CCA. Second is source side
write rate limiting. Things to note:
o) connect times for both at 10-15 ms
o) multiple TCP retries on a few rites - one case is 4 with 5 writes.
Source side pacing eliminates retries
o) Fairness with CCA isn't great but quite good with source side write
pacing
o) Queue depth with CCA is about 150 Kbytes about 100K byte with
source side pacing
o) min write to read is about 80 ms for both
o) max is 220 ms vs 97 ms
o) stdev for CCA write/read is 30 ms vs 3 ms
o) TCP RTT is 20ms w/CCA and 90 ms with ssp - seems odd here as
TCP_QUICACK and TCP_NODELAY are both enabled.
[ CT] final connect times (min/avg/max/stdev) =
10.326/13.522/14.986/2150.329 ms (tot/err) = 4/0
[rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e
--trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N
------------------------------------------------------------
Client connecting to (**hidden**), TCP port 5001 with pid 107678 (4
flows)
Write buffer size: 131072 Byte
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[ 1] local *.*.*.85%enp4s0 port 42480 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=3) (qack)
(icwnd/mss/irtt=14/1448/10534) (ct=10.63 ms) on 2023-01-06 12:17:56
(PST)
[ 4] local *.*.*.85%enp4s0 port 42488 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=5) (qack)
(icwnd/mss/irtt=14/1448/14023) (ct=14.08 ms) on 2023-01-06 12:17:56
(PST)
[ 3] local *.*.*.85%enp4s0 port 42502 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=6) (qack)
(icwnd/mss/irtt=14/1448/14642) (ct=14.70 ms) on 2023-01-06 12:17:56
(PST)
[ 2] local *.*.*.85%enp4s0 port 42484 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=4) (qack)
(icwnd/mss/irtt=14/1448/14728) (ct=14.79 ms) on 2023-01-06 12:17:56
(PST)
[ ID] Interval Transfer Bandwidth Write/Err Rtry
Cwnd/RTT(var) NetPwr
...
[ 4] 4.00-5.00 sec 1.38 MBytes 11.5 Mbits/sec 11/0 3
29K/21088(1142) us 68.37
[ 2] 4.00-5.00 sec 1.62 MBytes 13.6 Mbits/sec 13/0 2
31K/19284(612) us 88.36
[ 1] 4.00-5.00 sec 896 KBytes 7.34 Mbits/sec 7/0 5
16K/18996(658) us 48.30
[ 3] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 5
18K/18133(208) us 57.83
[SUM] 4.00-5.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 15
[ 4] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec 10/0 4
29K/14717(489) us 89.06
[ 1] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 4
16K/15874(408) us 66.06
[ 3] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 4
16K/15826(382) us 74.54
[ 2] 5.00-6.00 sec 1.50 MBytes 12.6 Mbits/sec 12/0 6
9K/14878(557) us 106
[SUM] 5.00-6.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 18
[ 4] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec 14/0 4
25K/15472(496) us 119
[ 2] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 2
26K/16417(427) us 63.87
[ 1] 6.00-7.00 sec 1.25 MBytes 10.5 Mbits/sec 10/0 5
16K/16268(679) us 80.57
[ 3] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 6
15K/16629(799) us 63.06
[SUM] 6.00-7.00 sec 5.00 MBytes 41.9 Mbits/sec 40/0 17
[ 4] 7.00-8.00 sec 1.75 MBytes 14.7 Mbits/sec 14/0 4
22K/13986(519) us 131
[ 1] 7.00-8.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 4
16K/12679(377) us 93.04
[ 3] 7.00-8.00 sec 896 KBytes 7.34 Mbits/sec 7/0 5
14K/12971(367) us 70.74
[ 2] 7.00-8.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 6
15K/14740(779) us 80.03
[SUM] 7.00-8.00 sec 4.88 MBytes 40.9 Mbits/sec 39/0 19
[root@bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m
------------------------------------------------------------
Server listening on TCP port 5001 with pid 233615
Read buffer size: 128 KByte (Dist bin width=16.0 KByte)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
------------------------------------------------------------
[ 1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
42480 (trip-times) (sock=4) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11636) on 2023-01-06 12:17:56 (PST)
[ 2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
42502 (trip-times) (sock=5) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11898) on 2023-01-06 12:17:56 (PST)
[ 3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
42484 (trip-times) (sock=6) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11938) on 2023-01-06 12:17:56 (PST)
[ 4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
42488 (trip-times) (sock=7) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11919) on 2023-01-06 12:17:56 (PST)
[ ID] Interval Transfer Bandwidth Burst Latency
avg/min/max/stdev (cnt/size) inP NetPwr Reads=Dist
...
[ 2] 4.00-5.00 sec 1.06 MBytes 8.86 Mbits/sec
129.819/90.391/186.075/31.346 ms (9/123080) 154 KByte 8.532803
467=461:6:0:0:0:0:0:0
[ 3] 4.00-5.00 sec 1.52 MBytes 12.8 Mbits/sec
103.552/82.653/169.274/28.382 ms (12/132854) 149 KByte 15.40
646=643:1:2:0:0:0:0:0
[ 4] 4.00-5.00 sec 1.39 MBytes 11.6 Mbits/sec
107.503/66.843/143.038/24.269 ms (11/132294) 149 KByte 13.54
619=617:1:1:0:0:0:0:0
[ 1] 4.00-5.00 sec 988 KBytes 8.10 Mbits/sec
141.389/119.961/178.785/18.812 ms (7/144593) 170 KByte 7.158641
409=404:5:0:0:0:0:0:0
[SUM] 4.00-5.00 sec 4.93 MBytes 41.4 Mbits/sec
2141=2125:13:3:0:0:0:0:0
[ 4] 5.00-6.00 sec 1.29 MBytes 10.8 Mbits/sec
118.943/86.253/176.128/31.248 ms (10/135098) 164 KByte 11.36
511=506:2:3:0:0:0:0:0
[ 2] 5.00-6.00 sec 1.09 MBytes 9.17 Mbits/sec
139.821/102.418/218.875/40.422 ms (9/127424) 148 KByte 8.202049
487=484:2:1:0:0:0:0:0
[ 3] 5.00-6.00 sec 1.51 MBytes 12.6 Mbits/sec
102.146/77.085/140.893/18.441 ms (13/121520) 151 KByte 15.47
640=636:1:3:0:0:0:0:0
[ 1] 5.00-6.00 sec 981 KBytes 8.04 Mbits/sec
161.901/105.582/219.931/36.260 ms (8/125614) 134 KByte 6.206944
415=413:2:0:0:0:0:0:0
[SUM] 5.00-6.00 sec 4.85 MBytes 40.7 Mbits/sec
2053=2039:7:7:0:0:0:0:0
[ 4] 6.00-7.00 sec 1.74 MBytes 14.6 Mbits/sec
88.846/74.297/101.859/7.118 ms (14/130526) 156 KByte 20.57
711=707:3:1:0:0:0:0:0
[ 1] 6.00-7.00 sec 1.22 MBytes 10.2 Mbits/sec
120.639/100.257/157.567/21.770 ms (10/127568) 157 KByte 10.57
494=488:5:1:0:0:0:0:0
[ 2] 6.00-7.00 sec 1015 KBytes 8.32 Mbits/sec
144.632/124.368/171.349/16.597 ms (8/129958) 143 KByte 7.188321
408=403:5:0:0:0:0:0:0
[ 3] 6.00-7.00 sec 1.02 MBytes 8.60 Mbits/sec
143.516/102.322/173.001/24.089 ms (8/134302) 146 KByte 7.486359
484=480:4:0:0:0:0:0:0
[SUM] 6.00-7.00 sec 4.98 MBytes 41.7 Mbits/sec
2097=2078:17:2:0:0:0:0:0
[ 4] 7.00-8.00 sec 1.77 MBytes 14.9 Mbits/sec
85.406/65.797/103.418/12.609 ms (14/132595) 153 KByte 21.74
692=687:2:3:0:0:0:0:0
[ 2] 7.00-8.00 sec 957 KBytes 7.84 Mbits/sec
153.936/131.452/191.464/19.361 ms (7/140042) 160 KByte 6.368199
429=425:4:0:0:0:0:0:0
[ 1] 7.00-8.00 sec 1.13 MBytes 9.44 Mbits/sec
131.146/109.737/166.774/22.035 ms (9/131124) 146 KByte 8.998528
520=516:4:0:0:0:0:0:0
[ 3] 7.00-8.00 sec 1.13 MBytes 9.51 Mbits/sec
126.512/88.404/220.175/42.237 ms (9/132089) 172 KByte 9.396784
527=524:1:2:0:0:0:0:0
[SUM] 7.00-8.00 sec 4.96 MBytes 41.6 Mbits/sec
2168=2152:11:5:0:0:0:0:0
With source side rate limiting to 9 mb/s per stream.
[rjmcmahon@ryzen3950 iperf2-code]$ iperf -c *** --hide-ips -e
--trip-times -i 1 -P 4 -t 10 -w 4m --tcp-quickack -N -b 9m
------------------------------------------------------------
Client connecting to (**hidden**), TCP port 5001 with pid 108884 (4
flows)
Write buffer size: 131072 Byte
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[ 1] local *.*.*.85%enp4s0 port 46448 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=3) (qack)
(icwnd/mss/irtt=14/1448/10666) (ct=10.70 ms) on 2023-01-06 12:27:45
(PST)
[ 3] local *.*.*.85%enp4s0 port 46460 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=6) (qack)
(icwnd/mss/irtt=14/1448/16499) (ct=16.54 ms) on 2023-01-06 12:27:45
(PST)
[ 2] local *.*.*.85%enp4s0 port 46454 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=4) (qack)
(icwnd/mss/irtt=14/1448/16580) (ct=16.86 ms) on 2023-01-06 12:27:45
(PST)
[ 4] local *.*.*.85%enp4s0 port 46458 connected with *.*.*.123 port
5001 (prefetch=16384) (trip-times) (sock=5) (qack)
(icwnd/mss/irtt=14/1448/16802) (ct=16.83 ms) on 2023-01-06 12:27:45
(PST)
[ ID] Interval Transfer Bandwidth Write/Err Rtry
Cwnd/RTT(var) NetPwr
...
[ 2] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
134K/88055(12329) us 11.91
[ 4] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
132K/74867(11755) us 14.01
[ 1] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
134K/89101(13134) us 11.77
[ 3] 4.00-5.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
131K/91451(11938) us 11.47
[SUM] 4.00-5.00 sec 4.00 MBytes 33.6 Mbits/sec 32/0 0
[ 2] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
134K/85135(14580) us 13.86
[ 4] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
132K/85124(15654) us 13.86
[ 1] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
134K/91336(11335) us 12.92
[ 3] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
131K/89185(13499) us 13.23
[SUM] 5.00-6.00 sec 4.50 MBytes 37.7 Mbits/sec 36/0 0
[ 2] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
134K/85687(13489) us 13.77
[ 4] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
132K/82803(13001) us 14.25
[ 1] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
134K/86869(15186) us 13.58
[ 3] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 9/0 0
131K/91447(12515) us 12.90
[SUM] 6.00-7.00 sec 4.50 MBytes 37.7 Mbits/sec 36/0 0
[ 2] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
134K/81814(13168) us 12.82
[ 4] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
132K/89008(13283) us 11.78
[ 1] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
134K/89494(12151) us 11.72
[ 3] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec 8/0 0
131K/91083(12797) us 11.51
[SUM] 7.00-8.00 sec 4.00 MBytes 33.6 Mbits/sec 32/0 0
[root@bobcat iperf2-code]# iperf -s -i 1 -e --hide-ips -w 4m
------------------------------------------------------------
Server listening on TCP port 5001 with pid 233981
Read buffer size: 128 KByte (Dist bin width=16.0 KByte)
TCP window size: 7.63 MByte (WARNING: requested 3.81 MByte)
------------------------------------------------------------
[ 1] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
46448 (trip-times) (sock=4) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11987) on 2023-01-06 12:27:45 (PST)
[ 2] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
46454 (trip-times) (sock=5) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11132) on 2023-01-06 12:27:45 (PST)
[ 3] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
46460 (trip-times) (sock=6) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/11097) on 2023-01-06 12:27:45 (PST)
[ 4] local *.*.*.123%eth0 port 5001 connected with *.*.*.171 port
46458 (trip-times) (sock=7) (peer 2.1.9-master) (qack)
(icwnd/mss/irtt=14/1448/17823) on 2023-01-06 12:27:45 (PST)
[ ID] Interval Transfer Bandwidth Burst Latency
avg/min/max/stdev (cnt/size) inP NetPwr Reads=Dist
[ 4] 0.00-1.00 sec 1.09 MBytes 9.15 Mbits/sec
93.383/90.103/95.661/2.232 ms (8/143028) 128 KByte 12.25
451=451:0:0:0:0:0:0:0
[ 3] 0.00-1.00 sec 1.08 MBytes 9.06 Mbits/sec
96.834/95.229/102.645/2.442 ms (8/141580) 131 KByte 11.70
472=472:0:0:0:0:0:0:0
[ 1] 0.00-1.00 sec 1.10 MBytes 9.19 Mbits/sec
95.183/92.623/97.579/1.431 ms (8/143571) 131 KByte 12.07
495=495:0:0:0:0:0:0:0
[ 2] 0.00-1.00 sec 1.09 MBytes 9.15 Mbits/sec
89.317/84.865/94.906/3.674 ms (8/143028) 122 KByte 12.81
489=489:0:0:0:0:0:0:0
[ ID] Interval Transfer Bandwidth Reads=Dist
[SUM] 0.00-1.00 sec 4.36 MBytes 36.6 Mbits/sec
1907=1907:0:0:0:0:0:0:0
[ 4] 1.00-2.00 sec 1.07 MBytes 8.95 Mbits/sec
92.649/89.987/95.036/1.828 ms (9/124314) 96.5 KByte 12.08
492=492:0:0:0:0:0:0:0
[ 3] 1.00-2.00 sec 1.06 MBytes 8.93 Mbits/sec
96.305/95.647/96.794/0.432 ms (9/123992) 100 KByte 11.59
480=480:0:0:0:0:0:0:0
[ 1] 1.00-2.00 sec 1.06 MBytes 8.89 Mbits/sec
92.578/90.866/94.145/1.371 ms (9/123510) 95.8 KByte 12.01
513=513:0:0:0:0:0:0:0
[ 2] 1.00-2.00 sec 1.07 MBytes 8.96 Mbits/sec
90.767/87.984/94.352/1.944 ms (9/124475) 94.7 KByte 12.34
489=489:0:0:0:0:0:0:0
[SUM] 1.00-2.00 sec 4.26 MBytes 35.7 Mbits/sec
1974=1974:0:0:0:0:0:0:0
[ 4] 2.00-3.00 sec 1.09 MBytes 9.13 Mbits/sec
93.977/91.795/96.561/1.693 ms (8/142656) 112 KByte 12.14
497=497:0:0:0:0:0:0:0
[ 3] 2.00-3.00 sec 1.08 MBytes 9.04 Mbits/sec
96.544/95.815/97.798/0.693 ms (8/141208) 114 KByte 11.70
503=503:0:0:0:0:0:0:0
[ 1] 2.00-3.00 sec 1.07 MBytes 9.01 Mbits/sec
93.970/91.193/96.325/1.796 ms (8/140846) 111 KByte 11.99
509=509:0:0:0:0:0:0:0
[ 2] 2.00-3.00 sec 1.08 MBytes 9.10 Mbits/sec
92.843/90.216/96.355/2.040 ms (8/142113) 111 KByte 12.25
509=509:0:0:0:0:0:0:0
[SUM] 2.00-3.00 sec 4.32 MBytes 36.3 Mbits/sec
2018=2018:0:0:0:0:0:0:0
[ 4] 3.00-4.00 sec 1.06 MBytes 8.86 Mbits/sec
93.222/89.063/96.104/2.346 ms (9/123027) 96.1 KByte 11.88
487=487:0:0:0:0:0:0:0
[ 3] 3.00-4.00 sec 1.07 MBytes 8.97 Mbits/sec
96.277/95.051/97.230/0.767 ms (9/124636) 101 KByte 11.65
489=489:0:0:0:0:0:0:0
[ 1] 3.00-4.00 sec 1.08 MBytes 9.02 Mbits/sec
93.899/88.732/96.972/2.737 ms (9/125280) 98.6 KByte 12.01
493=493:0:0:0:0:0:0:0
[ 2] 3.00-4.00 sec 1.07 MBytes 8.97 Mbits/sec
92.490/89.862/95.265/1.796 ms (9/124636) 96.6 KByte 12.13
493=493:0:0:0:0:0:0:0
[SUM] 3.00-4.00 sec 4.27 MBytes 35.8 Mbits/sec
1962=1962:0:0:0:0:0:0:0
[ 4] 4.00-5.00 sec 1.07 MBytes 9.00 Mbits/sec
92.431/81.888/96.221/4.524 ms (9/124958) 96.8 KByte 12.17
498=498:0:0:0:0:0:0:0
[ 1] 4.00-5.00 sec 1.07 MBytes 8.97 Mbits/sec
95.018/93.445/96.200/0.957 ms (9/124636) 99.3 KByte 11.81
490=490:0:0:0:0:0:0:0
[ 2] 4.00-5.00 sec 1.06 MBytes 8.90 Mbits/sec
93.874/86.485/95.672/2.810 ms (9/123671) 97.3 KByte 11.86
481=481:0:0:0:0:0:0:0
[ 3] 4.00-5.00 sec 1.08 MBytes 9.09 Mbits/sec
95.737/93.881/97.197/0.972 ms (9/126245) 101 KByte 11.87
484=484:0:0:0:0:0:0:0
[SUM] 4.00-5.00 sec 4.29 MBytes 36.0 Mbits/sec
1953=1953:0:0:0:0:0:0:0
[ 4] 5.00-6.00 sec 1.09 MBytes 9.13 Mbits/sec
92.908/86.844/95.994/3.012 ms (8/142656) 111 KByte 12.28
467=467:0:0:0:0:0:0:0
[ 3] 5.00-6.00 sec 1.07 MBytes 8.94 Mbits/sec
96.593/95.343/97.660/0.876 ms (8/139760) 113 KByte 11.58
478=478:0:0:0:0:0:0:0
[ 1] 5.00-6.00 sec 1.08 MBytes 9.03 Mbits/sec
95.021/91.421/97.167/1.893 ms (8/141027) 112 KByte 11.87
491=491:0:0:0:0:0:0:0
[ 2] 5.00-6.00 sec 1.08 MBytes 9.06 Mbits/sec
92.162/82.720/97.692/5.060 ms (8/141570) 109 KByte 12.29
488=488:0:0:0:0:0:0:0
[SUM] 5.00-6.00 sec 4.31 MBytes 36.2 Mbits/sec
1924=1924:0:0:0:0:0:0:0
[ 4] 6.00-7.00 sec 1.04 MBytes 8.70 Mbits/sec
92.793/85.343/96.967/3.552 ms (9/120775) 93.9 KByte 11.71
485=485:0:0:0:0:0:0:0
[ 2] 6.00-7.00 sec 1.05 MBytes 8.79 Mbits/sec
91.679/84.479/96.760/3.975 ms (9/122062) 93.8 KByte 11.98
472=472:0:0:0:0:0:0:0
[ 3] 6.00-7.00 sec 1.06 MBytes 8.88 Mbits/sec
96.982/95.933/98.371/0.680 ms (9/123349) 100 KByte 11.45
477=477:0:0:0:0:0:0:0
[ 1] 6.00-7.00 sec 1.05 MBytes 8.80 Mbits/sec
94.342/91.660/96.025/1.660 ms (9/122223) 96.7 KByte 11.66
494=494:0:0:0:0:0:0:0
[SUM] 6.00-7.00 sec 4.19 MBytes 35.2 Mbits/sec
1928=1928:0:0:0:0:0:0:0
[ 4] 7.00-8.00 sec 1.10 MBytes 9.25 Mbits/sec
92.515/88.182/96.351/2.538 ms (8/144466) 112 KByte 12.49
510=510:0:0:0:0:0:0:0
[ 3] 7.00-8.00 sec 1.09 MBytes 9.13 Mbits/sec
96.580/95.737/98.977/1.098 ms (8/142656) 115 KByte 11.82
480=480:0:0:0:0:0:0:0
[ 1] 7.00-8.00 sec 1.10 MBytes 9.21 Mbits/sec
95.269/91.719/97.514/2.126 ms (8/143923) 115 KByte 12.09
515=515:0:0:0:0:0:0:0
[ 2] 7.00-8.00 sec 1.11 MBytes 9.29 Mbits/sec
90.073/84.700/96.176/4.324 ms (8/145190) 110 KByte 12.90
508=508:0:0:0:0:0:0:0
[SUM] 7.00-8.00 sec 4.40 MBytes 36.9 Mbits/sec
2013=2013:0:0:0:0:0:0:0
Bob
-----Original Message-----
From: LibreQoS <[email protected]> On Behalf Of
Dave Taht
via LibreQoS
Sent: Wednesday, January 4, 2023 12:26 PM
Subject: [LibreQoS] the grinch meets cloudflare's christmas present
Please try the new, the shiny, the really wonderful test here:
https://urldefense.com/v3/__https://speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S
[1]
9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
[1]
I would really appreciate some independent verification of
measurements using this tool. In my brief experiments it appears -
as
all the commercial tools to date - to dramatically understate the
bufferbloat, on my LTE, (and my starlink terminal is out being
hacked^H^H^H^H^H^Hworked on, so I can't measure that)
[acm]
Hi Dave, I made some time to test "cloudflare's christmas present"
yesterday.
I'm on DOCSIS 3.1 service with 1Gbps Down. The Upstream has a "turbo"
mode with 40-50Mbps for the first ~3 sec, then steady-state about
23Mbps.
When I saw the ~620Mbps Downstream measurement, I was ready to
complain that even the IP-Layer Capacity was grossly underestimated.
In addition, the Latency measurements seem very low (as you asserted),
although the cloud server was “nearby”.
However, I found that Ookla and the ISP-provided measurement were also
reporting ~600Mbps! So the cloudflare Downstream capacity (or
throughput?) measurement was consistent with others. Our UDPST server
was unreachable, otherwise I would have added that measurement, too.
The Upstream measurement graph seems to illustrate the “turbo”
mode, with the dip after attaining 44.5Mbps.
UDPST saturates the uplink and we measure the full 250ms of the
Upstream buffer. Cloudflare’s latency measurements don’t even come
close.
Al
Links:
------
[1]
https://urldefense.com/v3/__https:/speed.cloudflare.com/__;!!BhdT!iZcFJ8WVU9S9zz5t456oxkfObrC5Xb9j5AG8UO3DqD5x4GAJkawZr0iGwEUtF0_09U8mCDnAkrJ9QEMHGbCMKVw$
_______________________________________________
Rpm mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/rpm
_______________________________________________
Bloat mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/bloat
