Van: ruby R53 <[email protected]>
Datum: woensdag, 14 januari 2026 03:26
Aan: [email protected]
Onderwerp: Change fetch addresses for Rust crates, Python packages and whatnot?
I've been trying to compile www/chromium for literal days now, all because
whenever `make` runs `fetch`, it goes to an address that's actually supposed to
redirect to another one, but `fetch` instead keeps trying for a whole minute
until it ultimately falls back to distcache.freebsd.org, which is another whole
minute of waiting for the file to download. The cherry on top is that the file
is, most of the time, just hundreds of kilobytes in size.
Hell, I even had to manually download part of these individual files using that
Links web browser··· it'd do it a lot faster than `fetch`, all because it
actually followed the redirection and didn't just keep trying over and over.
But of course I wouldn't manually download hundreds upon hundreds of archive
files, this is the reason why such tool exists.
So here's an example: currently one of the missing dependencies for
www/chromium is GTK3, which depends on graphics/librsvg2-rust, which then
depends on cargo-c. It's now trying to fetch dunce-1.0.5.crate, which is one of
the crates required for it, from
https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate.
Running `curl` on this link does literally nothing, no files are downloaded.
But running it on any web browser instantly downloads the crate in question,
all because that link actually redirects to
https://static.crates.io/crates/dunce/dunce-1.0.5.crate. Even `curl`ing that
new link does give the file I want. Same applies for Python packages.
Now imagine this for all the remaining files··· if it takes about 2 minutes to
download a single file, and there's 398 crate entries on cargo-c's
`Makefile.crates`, that means it'd take a whole 796 minutes to download
everything, which is literally more than half a day (~13.6 hours) :)
Can you copy-paste the output of the following command in a reply?
fetch -vv
"https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate";
For me it immediately follows the redirect and downloads the file in half a
second. See the output below.
Regards,
Ronald.
--
time fetch -vv
"https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate";
scheme: "https"
user: ""
password: ""
host: "crates.io"
port: "0"
document:
"/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate"
---> crates.io:443
resolving server address: crates.io:443
SSL options: 82004850
Peer verification enabled
Using OpenSSL default CA cert file and path
Verify hostname
TLSv1.3 connection established using TLS_AES_128_GCM_SHA256
Certificate subject: /CN=crates.io
Certificate issuer: /C=US/O=Amazon/CN=Amazon RSA 2048 M01
requesting
https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate
GET /api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate
HTTP/1.1
Host: crates.io
Accept: */*
User-Agent: fetch libfetch/2.0
Connection: close
<<< HTTP/1.1 302 Moved Temporarily
<<< Content-Length: 0
<<< Connection: close
content length: [0]
<<< Access-Control-Allow-Origin: *
<<< Date: Wed, 14 Jan 2026 07:10:06 GMT
<<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate
<<< Nel:
{"report_to":"heroku-nel","response_headers":["Via"],"max_age":3600,"success_fraction":0.01,"failure_fraction":0.1}
302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate
scheme: "https"
user: ""
password: ""
host: "static.crates.io"
port: "0"
document: "/crates/dunce/dunce-1.0.5.crate"
<<< Report-To:
{"group":"heroku-nel","endpoints":[{"url":"https://nel.heroku.com/reports?s=VOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D\u0026sid=af571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=1768374606"}],"max_age":3600}
<<< Reporting-Endpoints:
heroku-nel="https://nel.heroku.com/reports?s=VOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&ts=1768374606";
<<< Server: Heroku
<<< Strict-Transport-Security: max-age=31536000; includeSubDomains
<<< Via: 1.1 heroku-router, 1.1 c2905f891f96a0ec9c7fab16916dbb46.cloudfront.net
(CloudFront)
<<< X-Cache: Miss from cloudfront
<<< X-Amz-Cf-Pop: AMS58-P6
<<< X-Amz-Cf-Id: Xa10pwbYp4MCA5eToVa0shahNpzr2epWFUuQZS63bMpmbiDLKc2dmQ==
<<<
---> static.crates.io:443
resolving server address: static.crates.io:443
SSL options: 82004850
Peer verification enabled
Using OpenSSL default CA cert file and path
Verify hostname
TLSv1.3 connection established using TLS_AES_128_GCM_SHA256
Certificate subject: /CN=static.crates.io
Certificate issuer: /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Atlas R3 DV TLS CA
2025 Q3
requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate
GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1
Host: static.crates.io
Accept: */*
User-Agent: fetch libfetch/2.0
Connection: close
<<< HTTP/1.1 200
<<< Connection: close
<<< Content-Length: 8244
<<< etag: "198d7f7c37f632bc83502b28870b6d60"
content length: [8244]
<<< cache-control: public,max-age=31536000,immutable
<<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D
<<< x-amz-id-2:
Dj7mFoaqQJUc61mB1FcouUQcy0tajnacJvU+xOR3Be5//jo3GaSmdCwSSgtYyDMnYE4cPCiPv7PV9m7z1Jk0WazYHTFPbYhm
<<< x-amz-request-id: KWXPQ4JP20HSYSY2
<<< date: Wed, 14 Jan 2026 00:09:56 GMT
<<< x-amz-server-side-encryption: AES256
<<< x-amz-replication-status: COMPLETED
<<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT
<<< accept-ranges: bytes
last modified: [2024-08-04 01:36:47]
<<< content-type: application/gzip
<<< x-served-by: cache-ams21058-AMS
<<< server: AmazonS3
<<< age: 25212
<<< x-cache: HIT
<<< x-cache-hits: 148
<<<
offset 0, length -1, size -1, clength 8244
local size / mtime: 8244 / 1722735407
remote size / mtime: 8244 / 1722735407
dunce-1.0.5.crate 8244 B 10 MBps 00s
real 0m0.429s
user 0m0.023s
sys 0m0.027s
