When using one of the mirrors, mirror.team-cymru.org, ftp connectivity works sporadically--and the issue is resolved when net.inet.tcp.rfc1323=0. My first thought is some issue with the mirror's firewall, but I have no issue using this mirror from a Windows machine. With pf enabled (pass out/block in), the connection eventually fails. With pf disabled, the same ftp problem is exhibited, but the connection get reestablished and, eventually, succeeds.
Ultimately, I am interested in understanding the cause of this issue as I can reproduce similar issues (and resolutions) with a small number of non-FTP sites from OpenBSD clients as well. That said, the following are two tcpdumps of FTP sessions with mirror.team-cymru.org in the hope someone well versed in the art of TCP can shed some light. If needed, I can provide the actual tcpdump files. Thanks. OpenBSD 5.0 (latest snapshot) net.inet.tcp.rfc1323=1 (pf is enabled and the last five SYN packets from mirror.team-cymru.org are blocked) 10:44:20.417944 192.0.2.1.26968 > 38.229.66.100.21: S 2981129915:2981129915(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 2750203874 0> (DF) 10:44:20.464624 38.229.66.100.21 > 192.0.2.1.26968: S 3887768391:3887768391(0) ack 2981129916 win 5792 <mss 1460,sackOK,timestamp 953139935 2750203874,nop,wscale 9> (DF) 10:44:20.464652 192.0.2.1.26968 > 38.229.66.100.21: . ack 1 win 2048 <nop,nop,timestamp 2750203874 953139935> (DF) 10:44:20.550703 38.229.66.100.21 > 192.0.2.1.26968: P 1:52(51) ack 1 win 12 <nop,nop,timestamp 953140022 2750203874> (DF) 10:44:20.553077 192.0.2.1.26968 > 38.229.66.100.21: P 1:17(16) ack 52 win 2048 <nop,nop,timestamp 2750203875 953140022> (DF) [tos 0x10] 10:44:20.601843 38.229.66.100.21 > 192.0.2.1.26968: . ack 17 win 12 <nop,nop,timestamp 953140072 2750203875> (DF) 10:44:20.601940 38.229.66.100.21 > 192.0.2.1.26968: P 52:75(23) ack 17 win12 <nop,nop,timestamp 953140073 2750203875> (DF) 10:44:20.602421 192.0.2.1.26968 > 38.229.66.100.21: P 17:23(6) ack 75 win 2048 <nop,nop,timestamp 2750203875 953140073> (DF) [tos 0x10] 10:44:20.650842 38.229.66.100.21 > 192.0.2.1.26968: P 75:94(19) ack 23 win12 <nop,nop,timestamp 953140121 2750203875> (DF) 10:44:20.842893 192.0.2.1.26968 > 38.229.66.100.21: . ack 94 win 2048 <nop,nop,timestamp 2750203875 953140121> (DF) [tos 0x10] 10:44:40.790007 192.0.2.1.26968 > 38.229.66.100.21: P 23:29(6) ack 94 win 2048 <nop,nop,timestamp 2750203915 953140121> (DF) [tos 0x10] 10:44:40.837662 38.229.66.100.21 > 192.0.2.1.26968: P 94:142(48) ack 29 win 12 <nop,nop,timestamp 953160308 2750203915> (DF) 10:44:40.837741 192.0.2.1.2742 > 38.229.66.100.49878: S 542068158:542068158(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 1456430528 0> (DF) 10:44:40.885580 38.229.66.100.49878 > 192.0.2.1.2742: S 3919155647:3919155647(0) ack 542068159 win 5792 <mss 1460,sackOK,timestamp 953160355 1456430528,nop,wscale 9> (DF) 10:44:40.885603 192.0.2.1.2742 > 38.229.66.100.49878: . ack 1 win 2048 <nop,nop,timestamp 1456430528 953160355> (DF) 10:44:40.885656 192.0.2.1.26968 > 38.229.66.100.21: P 29:35(6) ack 142 win2048 <nop,nop,timestamp 2750203915 953160308> (DF) [tos 0x10] 10:44:40.935149 38.229.66.100.21 > 192.0.2.1.26968: P 142:181(39) ack 35 win 12 <nop,nop,timestamp 953160405 2750203915> (DF) 10:44:40.937706 38.229.66.100.49878 > 192.0.2.1.2742: P 1:507(506) ack 1 win 12 <nop,nop,timestamp 953160406 1456430528> (DF) [tos 0x8] 10:44:40.937710 38.229.66.100.49878 > 192.0.2.1.2742: F 507:507(0) ack 1 win 12 <nop,nop,timestamp 953160406 1456430528> (DF) [tos 0x8] 10:44:40.937730 192.0.2.1.2742 > 38.229.66.100.49878: . ack 508 win 1984 <nop,nop,timestamp 1456430528 953160406> (DF) [tos 0x8] 10:44:40.939520 192.0.2.1.2742 > 38.229.66.100.49878: F 1:1(0) ack 508 win2048 <nop,nop,timestamp 1456430528 953160406> (DF) [tos 0x8] 10:44:40.985303 38.229.66.100.21 > 192.0.2.1.26968: P 181:205(24) ack 35 win 12 <nop,nop,timestamp 953160455 2750203915> (DF) 10:44:40.985322 192.0.2.1.26968 > 38.229.66.100.21: . ack 205 win 2045 <nop,nop,timestamp 2750203915 953160405> (DF) [tos 0x10] 10:44:40.987348 38.229.66.100.49878 > 192.0.2.1.2742: . ack 2 win 12 <nop,nop,timestamp 953160456 1456430528> (DF) [tos 0x8] 10:44:51.627405 192.0.2.1.26968 > 38.229.66.100.21: P 35:41(6) ack 205 win2048 <nop,nop,timestamp 2750203937 953160405> (DF) [tos 0x10] 10:44:51.674301 38.229.66.100.21 > 192.0.2.1.26968: P 205:253(48) ack 41 win 12 <nop,nop,timestamp 953171143 2750203937> (DF) 10:44:51.674389 192.0.2.1.47015 > 38.229.66.100.55628: S 1372232033:1372232033(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 2079433561 0> (DF) 10:44:51.720919 38.229.66.100.55628 > 192.0.2.1.47015: S 3929278385:3929278385(0) ack 1372232034 win 5792 <mss 1460,sackOK,timestamp 953171191 2079433561,nop,wscale 9> (DF) 10:44:51.720942 192.0.2.1.47015 > 38.229.66.100.55628: . ack 1 win 2048 <nop,nop,timestamp 2079433561 953171191> (DF) 10:44:51.720985 192.0.2.1.26968 > 38.229.66.100.21: P 41:47(6) ack 253 win2048 <nop,nop,timestamp 2750203937 953171143> (DF) [tos 0x10] 10:44:51.768541 38.229.66.100.21 > 192.0.2.1.26968: P 253:292(39) ack 47 win 12 <nop,nop,timestamp 953171238 2750203937> (DF) 10:44:51.770934 38.229.66.100.55628 > 192.0.2.1.47015: P 1:507(506) ack 1 win 12 <nop,nop,timestamp 953171239 2079433561> (DF) [tos 0x8] 10:44:51.774054 38.229.66.100.55628 > 192.0.2.1.47015: F 507:507(0) ack 1 win 12 <nop,nop,timestamp 953171243 2079433561> (DF) [tos 0x8] 10:44:51.774075 192.0.2.1.47015 > 38.229.66.100.55628: . ack 508 win 2048 <nop,nop,timestamp 2079433561 953171239> (DF) [tos 0x8] 10:44:51.774117 192.0.2.1.47015 > 38.229.66.100.55628: F 1:1(0) ack 508 win 2048 <nop,nop,timestamp 2079433561 953171239> (DF) [tos 0x8] 10:44:51.822668 38.229.66.100.55628 > 192.0.2.1.47015: . ack 2 win 12 <nop,nop,timestamp 953171291 2079433561> (DF) [tos 0x8] 10:44:51.826299 38.229.66.100.21 > 192.0.2.1.26968: P 292:316(24) ack 47 win 12 <nop,nop,timestamp 953171296 2750203937> (DF) 10:44:51.826319 192.0.2.1.26968 > 38.229.66.100.21: . ack 316 win 2045 <nop,nop,timestamp 2750203937 953171238> (DF) [tos 0x10] 10:45:02.403998 192.0.2.1.26968 > 38.229.66.100.21: P 47:53(6) ack 316 win2048 <nop,nop,timestamp 2750203958 953171238> (DF) [tos 0x10] 10:45:02.454138 38.229.66.100.21 > 192.0.2.1.26968: P 316:364(48) ack 53 win 12 <nop,nop,timestamp 953181923 2750203958> (DF) 10:45:02.454236 192.0.2.1.4191 > 38.229.66.100.60707: S 1545628794:1545628794(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 1999225603 0> (DF) 10:45:02.653794 192.0.2.1.26968 > 38.229.66.100.21: . ack 364 win 2048 <nop,nop,timestamp 2750203959 953181923> (DF) [tos 0x10] 10:45:08.453929 192.0.2.1.4191 > 38.229.66.100.60707: S 1545628794:1545628794(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 1999225615 0> (DF) 10:45:20.454192 192.0.2.1.4191 > 38.229.66.100.60707: S 1545628794:1545628794(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 1999225639 0> (DF) 10:45:44.454718 192.0.2.1.4191 > 38.229.66.100.60707: S 1545628794:1545628794(0) win 16384 <mss 1452,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 1999225687 0> (DF) 10:46:17.462006 192.0.2.1.26968 > 38.229.66.100.21: P 53:84(31) ack 364 win 2048 <nop,nop,timestamp 2750204108 953181923> (DF) [tos 0x10] 10:46:17.510094 38.229.66.100.21 > 192.0.2.1.26968: P 364:415(51) ack 84 win 12 <nop,nop,timestamp 953256974 2750204108> (DF) 10:46:17.510213 192.0.2.1.26968 > 38.229.66.100.21: P 84:90(6) ack 415 win2048 <nop,nop,timestamp 2750204108 953256974> (DF) [tos 0x10] 10:46:17.559724 38.229.66.100.38959 > 192.0.2.1.49391: S 4000593739:4000593739(0) win 5840 <mss 1460,sackOK,timestamp 953257023 0,nop,wscale 9> (DF) 10:46:17.598311 38.229.66.100.21 > 192.0.2.1.26968: . ack 90 win 12 <nop,nop,timestamp 953257063 2750204108> (DF) 10:46:20.559894 38.229.66.100.38959 > 192.0.2.1.49391: S 4000593739:4000593739(0) win 5840 <mss 1460,sackOK,timestamp 953260023 0,nop,wscale 9> (DF) 10:46:26.560796 38.229.66.100.38959 > 192.0.2.1.49391: S 4000593739:4000593739(0) win 5840 <mss 1460,sackOK,timestamp 953266024 0,nop,wscale 9> (DF) 10:46:38.562214 38.229.66.100.38959 > 192.0.2.1.49391: S 4000593739:4000593739(0) win 5840 <mss 1460,sackOK,timestamp 953278024 0,nop,wscale 9> (DF) 10:47:02.562624 38.229.66.100.38959 > 192.0.2.1.49391: S 4000593739:4000593739(0) win 5840 <mss 1460,sackOK,timestamp 953302024 0,nop,wscale 9> (DF) 10:47:17.562560 38.229.66.100.21 > 192.0.2.1.26968: P 415:452(37) ack 90 win 12 <nop,nop,timestamp 953317023 2750204108> (DF) 10:47:17.756758 192.0.2.1.26968 > 38.229.66.100.21: . ack 452 win 2048 <nop,nop,timestamp 2750204229 953317023> (DF) [tos 0x10] This is what is being done on the OpenBSD client at this time: $ ftp -a mirror.team-cymru.org Trying 38.229.66.100... Connected to mirror.team-cymru.org. 220 Welcome to the Team Cymru public FTP service. 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. ftp> ls 150 Here comes the directory listing. drwxrwxr-x 12 3002 1000 4096 Apr 15 17:51 CPAN drwxrwxr-x 27 503 503 4096 Jul 10 13:14 CentOS drwxr-xr-x 12 989 989 4096 Aug 28 14:02 OpenBSD drwxrwsr-x 5 0 2573 4096 Aug 28 14:40 cygwin drwxr-xr-x 313 0 0 12288 Aug 28 12:00 gnu drwxr-xr-x 12 0 1 4096 Nov 11 2010 mozilla drwxr-xr-x 2 0 0 4096 Jan 28 2011 pub drwxr-xr-x 6 1000 1000 4096 Aug 28 14:48 ubuntu 226 Directory send OK. ftp> ls 150 Here comes the directory listing. drwxrwxr-x 12 3002 1000 4096 Apr 15 17:51 CPAN drwxrwxr-x 27 503 503 4096 Jul 10 13:14 CentOS drwxr-xr-x 12 989 989 4096 Aug 28 14:02 OpenBSD drwxrwsr-x 5 0 2573 4096 Aug 28 14:40 cygwin drwxr-xr-x 313 0 0 12288 Aug 28 12:00 gnu drwxr-xr-x 12 0 1 4096 Nov 11 2010 mozilla drwxr-xr-x 2 0 0 4096 Jan 28 2011 pub drwxr-xr-x 6 1000 1000 4096 Aug 28 14:48 ubuntu 226 Directory send OK. ftp> ls 200 EPRT command successful. Consider using EPSV. 425 Failed to establish connection. ftp> At 10:45:02 in the above capture, the third ls is issued. OpenBSD 5.0 (latest snapshot) net.inet.tcp.rfc1323=0 (pf is enabled and no packets from mirror.team-cymru.org are blocked) 11:04:33.298797 192.0.2.1.18407 > 38.229.66.100.21: S 3998507996:3998507996(0) win 16384 <mss 1452,nop,nop,sackOK> (DF) 11:04:33.346070 38.229.66.100.21 > 192.0.2.1.18407: S 876251824:876251824(0) ack 3998507997 win 5840 <mss 1460,nop,nop,sackOK> (DF) 11:04:33.346097 192.0.2.1.18407 > 38.229.66.100.21: . ack 1 win 16384 (DF) 11:04:33.411936 38.229.66.100.21 > 192.0.2.1.18407: P 1:52(51) ack 1 win 5840 (DF) 11:04:33.412165 192.0.2.1.18407 > 38.229.66.100.21: P 1:17(16) ack 52 win 16384 (DF) [tos 0x10] 11:04:33.459532 38.229.66.100.21 > 192.0.2.1.18407: . ack 17 win 5840 (DF) 11:04:33.462181 38.229.66.100.21 > 192.0.2.1.18407: P 52:75(23) ack 17 win 5840 (DF) 11:04:33.462331 192.0.2.1.18407 > 38.229.66.100.21: P 17:23(6) ack 75 win 16384 (DF) [tos 0x10] 11:04:33.510033 38.229.66.100.21 > 192.0.2.1.18407: P 75:94(19) ack 23 win 5840 (DF) 11:04:33.709482 192.0.2.1.18407 > 38.229.66.100.21: . ack 94 win 16384 (DF) [tos 0x10] 11:04:37.948133 192.0.2.1.18407 > 38.229.66.100.21: P 23:29(6) ack 94 win 16384 (DF) [tos 0x10] 11:04:37.997293 38.229.66.100.21 > 192.0.2.1.18407: P 94:142(48) ack 29 win 5840 (DF) 11:04:37.997368 192.0.2.1.17190 > 38.229.66.100.61243: S 209694139:209694139(0) win 16384 <mss 1452,nop,nop,sackOK> (DF) 11:04:38.043923 38.229.66.100.61243 > 192.0.2.1.17190: S 883774422:883774422(0) ack 209694140 win 5840 <mss 1460,nop,nop,sackOK> (DF) 11:04:38.043947 192.0.2.1.17190 > 38.229.66.100.61243: . ack 1 win 16384 (DF) 11:04:38.044001 192.0.2.1.18407 > 38.229.66.100.21: P 29:35(6) ack 142 win16384 (DF) [tos 0x10] 11:04:38.091698 38.229.66.100.61243 > 192.0.2.1.17190: P 1:507(506) ack 1 win 5840 (DF) [tos 0x8] 11:04:38.091702 38.229.66.100.61243 > 192.0.2.1.17190: F 507:507(0) ack 1 win 5840 (DF) [tos 0x8] 11:04:38.091723 192.0.2.1.17190 > 38.229.66.100.61243: . ack 508 win 15878 (DF) [tos 0x8] 11:04:38.092037 38.229.66.100.21 > 192.0.2.1.18407: P 142:181(39) ack 35 win 5840 (DF) 11:04:38.093770 192.0.2.1.17190 > 38.229.66.100.61243: F 1:1(0) ack 508 win 16384 (DF) [tos 0x8] 11:04:38.141058 38.229.66.100.61243 > 192.0.2.1.17190: . ack 2 win 5840 (DF) [tos 0x8] 11:04:38.142641 38.229.66.100.21 > 192.0.2.1.18407: P 181:205(24) ack 35 win 5840 (DF) 11:04:38.142658 192.0.2.1.18407 > 38.229.66.100.21: . ack 205 win 16360 (DF) [tos 0x10] 11:04:46.245928 192.0.2.1.18407 > 38.229.66.100.21: P 35:41(6) ack 205 win16384 (DF) [tos 0x10] 11:04:46.294482 38.229.66.100.21 > 192.0.2.1.18407: P 205:253(48) ack 41 win 5840 (DF) 11:04:46.294559 192.0.2.1.17241 > 38.229.66.100.62645: S 1204601243:1204601243(0) win 16384 <mss 1452,nop,nop,sackOK> (DF) 11:04:46.342851 38.229.66.100.62645 > 192.0.2.1.17241: S 892990481:892990481(0) ack 1204601244 win 5840 <mss 1460,nop,nop,sackOK> (DF) 11:04:46.342873 192.0.2.1.17241 > 38.229.66.100.62645: . ack 1 win 16384 (DF) 11:04:46.342914 192.0.2.1.18407 > 38.229.66.100.21: P 41:47(6) ack 253 win 16384 (DF) [tos 0x10] 11:04:46.390972 38.229.66.100.21 > 192.0.2.1.18407: P 253:292(39) ack 47 win 5840 (DF) 11:04:46.391372 38.229.66.100.62645 > 192.0.2.1.17241: P 1:507(506) ack 1 win 5840 (DF) [tos 0x8] 11:04:46.391376 38.229.66.100.62645 > 192.0.2.1.17241: F 507:507(0) ack 1 win 5840 (DF) [tos 0x8] 11:04:46.391396 192.0.2.1.17241 > 38.229.66.100.62645: . ack 508 win 15878 (DF) [tos 0x8] 11:04:46.393136 192.0.2.1.17241 > 38.229.66.100.62645: F 1:1(0) ack 508 win 16384 (DF) [tos 0x8] 11:04:46.439070 38.229.66.100.21 > 192.0.2.1.18407: P 292:316(24) ack 47 win 5840 (DF) 11:04:46.439090 192.0.2.1.18407 > 38.229.66.100.21: . ack 316 win 16360 (DF) [tos 0x10] 11:04:46.440738 38.229.66.100.62645 > 192.0.2.1.17241: . ack 2 win 5840 (DF) [tos 0x8] 11:04:50.269367 192.0.2.1.18407 > 38.229.66.100.21: P 47:53(6) ack 316 win16384 (DF) [tos 0x10] 11:04:50.317070 38.229.66.100.21 > 192.0.2.1.18407: P 316:364(48) ack 53 win 5840 (DF) 11:04:50.317142 192.0.2.1.35944 > 38.229.66.100.56210: S 103148629:103148629(0) win 16384 <mss 1452,nop,nop,sackOK> (DF) 11:04:50.364444 38.229.66.100.56210 > 192.0.2.1.35944: S 896633888:896633888(0) ack 103148630 win 5840 <mss 1460,nop,nop,sackOK> (DF) 11:04:50.364466 192.0.2.1.35944 > 38.229.66.100.56210: . ack 1 win 16384 (DF) 11:04:50.364507 192.0.2.1.18407 > 38.229.66.100.21: P 53:59(6) ack 364 win 16384 (DF) [tos 0x10] 11:04:50.411975 38.229.66.100.56210 > 192.0.2.1.35944: P 1:507(506) ack 1 win 5840 (DF) [tos 0x8] 11:04:50.411979 38.229.66.100.56210 > 192.0.2.1.35944: F 507:507(0) ack 1 win 5840 (DF) [tos 0x8] 11:04:50.412001 192.0.2.1.35944 > 38.229.66.100.56210: . ack 508 win 15878(DF) [tos 0x8] 11:04:50.412309 38.229.66.100.21 > 192.0.2.1.18407: P 364:403(39) ack 59 win 5840 (DF) 11:04:50.414019 192.0.2.1.35944 > 38.229.66.100.56210: F 1:1(0) ack 508 win 16384 (DF) [tos 0x8] 11:04:50.460068 38.229.66.100.21 > 192.0.2.1.18407: P 403:427(24) ack 59 win 5840 (DF) 11:04:50.460086 192.0.2.1.18407 > 38.229.66.100.21: . ack 427 win 16360 (DF) [tos 0x10] 11:04:50.460929 38.229.66.100.56210 > 192.0.2.1.35944: . ack 2 win 5840 (DF) [tos 0x8] 11:05:23.220736 192.0.2.1.18407 > 38.229.66.100.21: P 59:65(6) ack 427 win 16384 (DF) [tos 0x10] 11:05:23.268803 38.229.66.100.21 > 192.0.2.1.18407: P 427:441(14) ack 65 win 5840 (DF) 11:05:23.268954 192.0.2.1.18407 > 38.229.66.100.21: F 65:65(0) ack 441 win 16384 (DF) [tos 0x10] 11:05:23.271670 38.229.66.100.21 > 192.0.2.1.18407: F 441:441(0) ack 65 win 5840 (DF) 11:05:23.271697 192.0.2.1.18407 > 38.229.66.100.21: F 65:65(0) ack 442 win 16384 (DF) [tos 0x10] 11:05:23.316571 38.229.66.100.21 > 192.0.2.1.18407: . ack 66 win 5840 (DF)
