On Wednesday, January 14th, 2026 at 4:11 AM, Ronald Klop <[email protected]> wrote:
> > > Van: ruby R53 <[email protected]> > Datum: woensdag, 14 januari 2026 03:26 > Aan: [email protected] > Onderwerp: Change fetch addresses for Rust crates, Python packages and > whatnot? > > > I've been trying to compile www/chromium for literal days now, all because > > whenever `make` runs `fetch`, it goes to an address that's actually > > supposed to redirect to another one, but `fetch` instead keeps trying for a > > whole minute until it ultimately falls back to distcache.freebsd.org, which > > is another whole minute of waiting for the file to download. The cherry on > > top is that the file is, most of the time, just hundreds of kilobytes in > > size. > > > > Hell, I even had to manually download part of these individual files using > > that Links web browser··· it'd do it a lot faster than `fetch`, all because > > it actually followed the redirection and didn't just keep trying over and > > over. But of course I wouldn't manually download hundreds upon hundreds of > > archive files, this is the reason why such tool exists. > > > > So here's an example: currently one of the missing dependencies for > > www/chromium is GTK3, which depends on graphics/librsvg2-rust, which then > > depends on cargo-c. It's now trying to fetch dunce-1.0.5.crate, which is > > one of the crates required for it, from > > https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate. > > Running `curl` on this link does literally nothing, no files are > > downloaded. But running it on any web browser instantly downloads the crate > > in question, all because that link actually redirects to > > https://static.crates.io/crates/dunce/dunce-1.0.5.crate. Even `curl`ing > > that new link does give the file I want. Same applies for Python packages. > > > > Now imagine this for all the remaining files··· if it takes about 2 minutes > > to download a single file, and there's 398 crate entries on cargo-c's > > `Makefile.crates`, that means it'd take a whole 796 minutes to download > > everything, which is literally more than half a day (~13.6 hours) :) > > > > > > Can you copy-paste the output of the following command in a reply? > fetch -vv > "https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate"; > > For me it immediately follows the redirect and downloads the file in half a > second. See the output below. > > Regards, > Ronald. > > > -- > > time fetch -vv > "https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate"; > scheme: "https" > user: "" > password: "" > host: "crates.io" > port: "0" > document: > "/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate" > ---> crates.io:443 > resolving server address: crates.io:443 > SSL options: 82004850 > Peer verification enabled > Using OpenSSL default CA cert file and path > Verify hostname > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > Certificate subject: /CN=crates.io > Certificate issuer: /C=US/O=Amazon/CN=Amazon RSA 2048 M01 > requesting > https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate > >>> GET > >>> /api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate > >>> HTTP/1.1 > >>> Host: crates.io > >>> Accept: */* > >>> User-Agent: fetch libfetch/2.0 > >>> Connection: close > >>> > <<< HTTP/1.1 302 Moved Temporarily > <<< Content-Length: 0 > <<< Connection: close > content length: [0] > <<< Access-Control-Allow-Origin: * > <<< Date: Wed, 14 Jan 2026 07:10:06 GMT > <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate > <<< Nel: > {"report_to":"heroku-nel","response_headers":["Via"],"max_age":3600,"success_fraction":0.01,"failure_fraction":0.1} > 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate > scheme: "https" > user: "" > password: "" > host: "static.crates.io" > port: "0" > document: "/crates/dunce/dunce-1.0.5.crate" > <<< Report-To: > {"group":"heroku-nel","endpoints":[{"url":"https://nel.heroku.com/reports?s=VOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D\u0026sid=af571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=1768374606"}],"max_age":3600} > <<< Reporting-Endpoints: > heroku-nel="https://nel.heroku.com/reports?s=VOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&ts=1768374606"; > <<< Server: Heroku > <<< Strict-Transport-Security: max-age=31536000; includeSubDomains > <<< Via: 1.1 heroku-router, 1.1 > c2905f891f96a0ec9c7fab16916dbb46.cloudfront.net (CloudFront) > <<< X-Cache: Miss from cloudfront > <<< X-Amz-Cf-Pop: AMS58-P6 > <<< X-Amz-Cf-Id: Xa10pwbYp4MCA5eToVa0shahNpzr2epWFUuQZS63bMpmbiDLKc2dmQ== > <<< > ---> static.crates.io:443 > resolving server address: static.crates.io:443 > SSL options: 82004850 > Peer verification enabled > Using OpenSSL default CA cert file and path > Verify hostname > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > Certificate subject: /CN=static.crates.io > Certificate issuer: /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Atlas R3 DV TLS CA > 2025 Q3 > requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate > >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1 > >>> Host: static.crates.io > >>> Accept: */* > >>> User-Agent: fetch libfetch/2.0 > >>> Connection: close > >>> > <<< HTTP/1.1 200 > <<< Connection: close > <<< Content-Length: 8244 > <<< etag: "198d7f7c37f632bc83502b28870b6d60" > content length: [8244] > <<< cache-control: public,max-age=31536000,immutable > <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D > <<< x-amz-id-2: > Dj7mFoaqQJUc61mB1FcouUQcy0tajnacJvU+xOR3Be5//jo3GaSmdCwSSgtYyDMnYE4cPCiPv7PV9m7z1Jk0WazYHTFPbYhm > <<< x-amz-request-id: KWXPQ4JP20HSYSY2 > <<< date: Wed, 14 Jan 2026 00:09:56 GMT > <<< x-amz-server-side-encryption: AES256 > <<< x-amz-replication-status: COMPLETED > <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT > <<< accept-ranges: bytes > last modified: [2024-08-04 01:36:47] > <<< content-type: application/gzip > <<< x-served-by: cache-ams21058-AMS > <<< server: AmazonS3 > <<< age: 25212 > <<< x-cache: HIT > <<< x-cache-hits: 148 > <<< > offset 0, length -1, size -1, clength 8244 > local size / mtime: 8244 / 1722735407 > remote size / mtime: 8244 / 1722735407 > dunce-1.0.5.crate 8244 B 10 MBps 00s > > real 0m0.429s > user 0m0.023s > sys 0m0.027s ``` # time fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate"; scheme: "https" user: "" password: "" host: "crates.io" port: "0" document: "/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate" ---> crates.io:443 resolving server address: crates.io:443 (<- it gets stuck here for like a minute) SSL options: 82004850 Peer verification enabled Using OpenSSL default CA cert file and path Verify hostname TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 Certificate subject: /CN=crates.io Certificate issuer: /C=US/O=Amazon/CN=Amazon RSA 2048 M01 requesting https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate >>> GET >>> /api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate >>> HTTP/1.1 >>> Host: crates.io >>> Accept: */* >>> User-Agent: fetch libfetch/2.0 >>> Connection: close >>> <<< HTTP/1.1 302 Moved Temporarily <<< Content-Length: 0 <<< Connection: close content length: [0] <<< Access-Control-Allow-Origin: * <<< Date: Wed, 14 Jan 2026 14:34:21 GMT <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate <<< Nel: {"report_to":"heroku-nel","response_headers":["Via"],"max_age":3600,"success_fraction":0.01,"failure_fraction":0.1} 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate scheme: "https" user: "" password: "" host: "static.crates.io" port: "0" document: "/crates/dunce/dunce-1.0.5.crate" <<< Report-To: {"group":"heroku-nel","endpoints":[{"url":"https://nel.heroku.com/reports?s=9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D\u0026sid=af571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=1768401261"}],"max_age":3600} <<< Reporting-Endpoints: heroku-nel="https://nel.heroku.com/reports?s=9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&ts=1768401261"; <<< Server: Heroku <<< Strict-Transport-Security: max-age=31536000; includeSubDomains <<< Via: 1.1 heroku-router, 1.1 608555ff8069f08e821e1679cd412294.cloudfront.net (CloudFront) <<< X-Cache: Miss from cloudfront <<< X-Amz-Cf-Pop: GRU3-P10 <<< X-Amz-Cf-Id: LvtqT2cBMH7thFfdy3ktelkhg-ALuajh-ucfDhnv7l_s73okAxXp7Q== <<< ---> static.crates.io:443 resolving server address: static.crates.io:443 SSL options: 82004850 Peer verification enabled Using OpenSSL default CA cert file and path Verify hostname TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 Certificate subject: /CN=static.crates.io Certificate issuer: /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Atlas R3 DV TLS CA 2025 Q3 requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1 >>> Host: static.crates.io >>> Accept: */* >>> User-Agent: fetch libfetch/2.0 >>> Connection: close >>> <<< HTTP/1.1 200 <<< Connection: close <<< Content-Length: 8244 <<< cache-control: public,max-age=31536000,immutable content length: [8244] <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D <<< x-amz-id-2: EzDoao06Q7mYZgq0ecISLqKGCrHOV1gBvzNGaaAfOg4QCWZ/d3G4pjY4ZAoo2BNA0jACxYFWJQQ= <<< x-amz-request-id: 7JJDJ8X4GESHDW5P <<< date: Wed, 14 Jan 2026 04:21:53 GMT <<< x-amz-replication-status: COMPLETED <<< x-amz-server-side-encryption: AES256 <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT <<< etag: "198d7f7c37f632bc83502b28870b6d60" last modified: [2024-08-04 01:36:47] <<< accept-ranges: bytes <<< content-type: application/gzip <<< x-served-by: cache-gru-sbgr1930032-GRU <<< server: AmazonS3 <<< age: 36813 <<< x-cache: HIT <<< x-cache-hits: 107 <<< offset 0, length -1, size -1, clength 8244 fetch: transfer timed out fetch: dunce-1.0.5.crate appears to be truncated: 0/8244 bytes 184.03 real 0.02 user 0.00 sys so it's even worse, it does follow the redirect but somehow can't get the file at all, and all of that actually takes 3 minutes to do
