Probably worth pinging this one since l2k18 is on. s_graf is trying to fetch ports sources from https://pypi.org/ and is getting a hang and eventually a timeout when attempting connection from ftp(1) on armv7. (From recent ports@ posts it seems like this still occurs).
curl/wget were working ok when tested before. Can anyone with armv7 confirm/deny that they can replicate this? (just try "ftp https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz";). Any ideas? > ftp: SSL write error: handshake failed: Operation timed out ----- Forwarded message from [email protected] ----- From: [email protected] Date: Wed, 15 Aug 2018 15:02:40 -0700 To: [email protected] Cc: 'Stuart Henderson' <[email protected]> X-Mailer: Microsoft Outlook 16.0 Subject: FW: SSL connection failure with ftp but not wget [was Re: python files moved] -- compare with another system Trying to get a file during a php build fails on arm but not on i386 systems. Stuart Henderson suggested I forward this. From the arm system (op1bsdtest2) op1bsdtest2# curl -v https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz * Trying 151.101.0.223... * TCP_NODELAY set * Connected to pypi.io (151.101.0.223) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: businessCategory=Private Organization; jurisdictionCountryName=US; jurisdictionStateOrProvinceName=Delaware; serialNumber=3359300; C=US; ST=New Hampshire; L=Wolfeboro; O=Python Software Foundation; CN=www.python.org * start date: Mar 28 00:00:00 2018 GMT * expire date: Sep 27 12:00:00 2018 GMT * subjectAltName: host "pypi.io" matched cert's "pypi.io" * issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 Extended Validation Server CA * SSL certificate verify ok. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x8063b000) > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/2 > Host: pypi.io > User-Agent: curl/7.61.0 > Accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS == 100)! < HTTP/2 301 < server: Varnish < retry-after: 0 < location: https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz < content-type: text/html; charset=UTF-8 < accept-ranges: bytes < date: Wed, 15 Aug 2018 21:55:38 GMT < x-served-by: cache-sea1033-SEA < x-cache: HIT < x-cache-hits: 0 < x-timer: S1534370139.898309,VS0,VE0 < strict-transport-security: max-age=31536000; includeSubDomains; preload < x-frame-options: deny < x-xss-protection: 1; mode=block < x-content-type-options: nosniff < x-permitted-cross-domain-policies: none < content-length: 122 < * Connection #0 to host pypi.io left intact <html><head><title>301 Moved Permanently</title></head><body><center><h1>301 Moved Permanently</h1></center></body></html>op1bsdtest2# op1bsdtest2# dmesg OpenBSD 6.3-current (GENERIC) #32: Fri Aug 10 10:32:37 MDT 2018 [email protected]:/usr/src/sys/arch/armv7/compile/GENERIC real mem = 536870912 (512MB) avail mem = 516112384 (492MB) mainbus0 at root: Xunlong Orange Pi One cpu0 at mainbus0: ARM Cortex-A7 r0p5 (ARMv7) cpu0: DC enabled IC enabled WB disabled EABT branch prediction enabled cpu0: 32KB(32b/l,2way) I-cache, 32KB(64b/l,4way) wr-back D-cache cortex0 at mainbus0 psci0 at mainbus0: PSCI 0.0 sxiccmu0 at mainbus0 simplebus0 at mainbus0: "soc" syscon0 at simplebus0: "syscon" sxiccmu1 at simplebus0 sxipio0 at simplebus0: 94 pins ampintc0 at simplebus0 nirq 160, ncpu 4: "interrupt-controller" sxiccmu2 at simplebus0 sxipio1 at simplebus0: 12 pins sximmc0 at simplebus0 sdmmc0 at sximmc0: 4-bit, sd high-speed, mmc high-speed, dma ehci0 at simplebus0 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1 ehci1 at simplebus0 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1 sxitemp0 at simplebus0 dwxe0 at simplebus0: address 02:81:b1:07:76:5e ukphy0 at dwxe0 phy 1: Generic IEEE 802.3u media interface, rev. 0: OUI 0x001105, model 0x0000 sxidog0 at simplebus0 com0 at simplebus0: ns16550, no working fifo com0: console sxitwi0 at simplebus0 iic0 at sxitwi0 "bosch,bme280" at iic0 addr 0x76 not configured sxitwi1 at simplebus0 iic1 at sxitwi1 "bosch,bme280" at iic1 addr 0x77 not configured sxirtc0 at simplebus0 gpio0 at sxipio0: 32 pins gpio1 at sxipio0: 32 pins gpio2 at sxipio0: 32 pins gpio3 at sxipio0: 32 pins gpio4 at sxipio0: 32 pins gpio5 at sxipio0: 32 pins gpio6 at sxipio0: 32 pins gpio7 at sxipio1: 32 pins agtimer0 at mainbus0: tick rate 24000 KHz scsibus0 at sdmmc0: 2 targets, initiator 0 sd0 at scsibus0 targ 1 lun 0: <SD/MMC, SL16G, 0080> SCSI2 0/direct removable sd0: 15193MB, 512 bytes/sector, 31116288 sectors vscsi0 at root scsibus1 at vscsi0: 256 targets softraid0 at root scsibus2 at softraid0: 256 targets bootfile: sd0a:/bsd boot device: sd0 root on sd0a (88106578f2222800.a) swap on sd0b dump on sd0b op1bsdtest2# -----Original Message----- From: Stuart Henderson <[email protected]> Sent: August 15, 2018 1:47 PM To: [email protected] Subject: RE: SSL connection failure with ftp but not wget [was Re: python files moved] -- compare with another system Very interesting! Could you forward to bugs@ so people who might have a better idea what's wrong will see it please? -- Sent from a phone, apologies for poor formatting. On 15 August 2018 20:01:10 <[email protected]> wrote: > It looks like the problem is specific to the arm system. I ran the ftp > -d on both systems one after the other. Both are on the same network. > The arm system is a recent snapshot base install with src and ports > loaded and really nothing else. > I have not seen any other connection problems on the arm system and it > is doing many as part of the php build. > > I will try some of the network reconfigs when the build of php finishes. > > From my 6.2 stable server: > > # ftp -d https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz > host pypi.io, port https, path > packages/source/s/six/six-1.11.0.tar.gz, > save as six-1.11.0.tar.gz, auth none. > Trying 151.101.0.223... > Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0 > Host: pypi.io > User-Agent: OpenBSD ftp > > received 'HTTP/1.1 301 Redirect to Primary Domain' > received 'Server: Varnish' > received 'Retry-After: 0' > received 'Location: https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz' > Redirected to https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz > host pypi.org, port https, path > packages/source/s/six/six-1.11.0.tar.gz, > save as six-1.11.0.tar.gz, auth none. > Trying 151.101.0.223... > Requesting https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0 > Host: pypi.org > User-Agent: OpenBSD ftp > > received 'HTTP/1.1 301 Moved Permanently' > received 'Content-Security-Policy: base-uri 'self'; > block-all-mixed-content; connect-src 'self' > https://api.github.com/repos/ *.fastly-insights.com sentry.io > https://2p66nmmycsj3.statuspage.io; > default-src 'none'; font-src 'self' fonts.gstatic.com; form-action > 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self' > https://warehouse-camo.cmh1.psfhosted.org/ www.google-analytics.com > *.fastly-insights.com; script-src 'self' www.googletagmanager.com > www.google-analytics.com *.fastly-insights.com > https://cdn.ravenjs.com; style-src 'self' fonts.googleapis.com; worker-src *.fastly-insights.com' > received 'Content-Type: text/plain; charset=UTF-8' > received 'Location: > https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz' > Redirected to > https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz > host files.pythonhosted.org, port https, path > packages/source/s/six/six-1.11.0.tar.gz, save as six-1.11.0.tar.gz, auth none. > Trying 151.101.41.63... > Requesting > https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0 > Host: files.pythonhosted.org > User-Agent: OpenBSD ftp > > received 'HTTP/1.1 302 Found' > received 'Cache-Control: max-age=604800, public' > received 'Content-Type: application/octet-stream' > received 'Location: > https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c 111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz' > Redirected to > https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c917 > 42194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz > host files.pythonhosted.org, port https, path > packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefa > f7afe/six-1.11.0.tar.gz, > save as six-1.11.0.tar.gz, auth none. > Trying 151.101.41.63... > Requesting > https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c917 > 42194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz > GET > /packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adef > af7afe/six-1.11.0.tar.gz > HTTP/1.0 > Host: files.pythonhosted.org > User-Agent: OpenBSD ftp > > received 'HTTP/1.1 200 OK' > received 'x-amz-id-2: > ZPG4LCvWjZhEUNqY9PvtfV2e2YaS3x2TDj/kcEDliRXzdWXLkp8nYE68NEGm0yD2GIomC5Ns1hw= ' > received 'x-amz-request-id: 27A0CF68EA8E91AB' > received 'Last-Modified: Sun, 17 Sep 2017 18:46:56 GMT' > received 'ETag: "d12789f9baf7e9fb2524c0c64f1773f8"' > received 'x-amz-version-id: RwRLQ60RynDAt7f8Xqbv.StV0y_SRxXJ' > received 'Content-Type: binary/octet-stream' > received 'Server: AmazonS3' > received 'Cache-Control: max-age=365000000, immutable' > received 'Content-Length: 29860' > received 'Accept-Ranges: bytes' > received 'Date: Wed, 15 Aug 2018 18:38:52 GMT' > received 'Age: 4781773' > received 'Connection: close' > received 'X-Served-By: cache-sea1041-SEA, cache-sjc3122-SJC' > received 'X-Cache: HIT, HIT' > received 'X-Cache-Hits: 1, 4' > received 'X-Timer: S1534358332.351879,VS0,VE0' > received 'Strict-Transport-Security: max-age=31536000; > includeSubDomains; preload' > received 'X-Frame-Options: deny' > received 'X-XSS-Protection: 1; mode=block' > received 'X-Content-Type-Options: nosniff' > received 'X-Permitted-Cross-Domain-Policies: none' > received 'X-Robots-Header: noindex' > 100% > |********************************************************************* > |*****| > 29860 00:00 > 29860 bytes received in 0.04 seconds (784.32 KB/s) > > From arm system: > > op1bsdtest2# ftp -d > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz > host pypi.io, port https, path > packages/source/s/six/six-1.11.0.tar.gz, > save as six-1.11.0.tar.gz, auth none. > Trying 151.101.0.223... > Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz > ftp: SSL write error: handshake failed: Operation timed out > op1bsdtest2# > > > > -----Original Message----- > From: [email protected] <[email protected]> On Behalf Of > Stuart Henderson > Sent: August 15, 2018 1:37 AM > To: [email protected] > Cc: [email protected] > Subject: SSL connection failure with ftp but not wget [was Re: python > files moved] > > On 2018/08/14 17:41, [email protected] wrote: >> The current setup failed on the last three builds I have done. >> >> Wget seems to understand redirection. Note one line from wget output >> seems to imply that the site has moved permanently. >> >> Connecting to pypi.org (pypi.org)|151.101.0.223|:443... connected. >> HTTP request sent, awaiting response... 301 Moved Permanently >> Location: >> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.g >> z >> [following] >> --2018-08-14 15:57:26-- >> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.g >> z >> >> ftp -d failed after a long time. >> >> op1bsdtest2# ftp -d >> https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz >> host pypi.io, port https, path >> packages/source/s/six/six-1.11.0.tar.gz, save as six-1.11.0.tar.gz, auth none. >> Trying 151.101.0.223... >> Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz >> ftp: SSL write error: handshake failed: Operation timed out > > The redirection thing is a red herring. ftp and wget both understand > it and it should happen quickly. As you aren't able to successfully > connect to https://pypi.io/ with ftp it doesn't even see the > redirection, just eventually times out and falls back to ftp.openbsd.org. > > I'm not sure why wget can connect but ftp can't - I don't think either > are doing anything particularly unusual with the TLS connection and > both use libressl for this.. > > Can you try curl -v -o /dev/null > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz ? > Does that succeed or fail, and can you paste the output? (it has > better TLS debug than ftp or wget). > > Is this on armv7 again? If so are you able to test on a machine of > another arch on the same network? > > Do you have problems connecting to anything else hosted on fastly? > > Does the problem go away if you reduce MTU on the network interface? > ("ifconfig em0 mtu 1200" or something?) > > > >> -----Original Message----- >> From: Stuart Henderson <[email protected]> >> Sent: August 14, 2018 4:23 PM >> To: [email protected] >> Cc: [email protected] >> Subject: Re: python files moved >> >> On 2018/08/14 16:03, [email protected] wrote: >> > When building php I get the following error which causes a multi >> > minute timeout. From a wget request it looks like the web site has >> > moved. This happens on many files and causes quite a slowdown in >> > building >> a port. >> >> If we point too far into the redirection chain for pypi we're more >> likely to have failures next time they change things, the pypi.io >> ones seems a more stable endpoint. >> >> > ===> Checking files for py-six-1.11.0 >> > >> > >> Fetch https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz >> > >> > ftp: SSL write error: handshake failed: Operation timed out >> > >> > >> Fetch >> > >> https://ftp.openbsd.org/pub/OpenBSD/distfiles/six-1.11.0.tar.gz >> > >> > six-1.11.0.tar.gz 100% >> > |********************************************************| 29860 >> 00:00 >> > >> > >> > >> > With wget: >> > >> > >> > >> > op1bsdtest2# wget >> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz >> > >> > --2018-08-14 15:57:26-- >> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz >> > >> > Resolving pypi.io (pypi.io)... 151.101.0.223, 151.101.64.223, >> > 151.101.128.223, ... >> > >> > Connecting to pypi.io (pypi.io)|151.101.0.223|:443... connected. >> >> It's rather odd that ftp(1) times out and wget succeeds. Does ftp -d >> throw any light on it? >> >> ----- End forwarded message -----
