Van: ruby R53 <[email protected]>
Datum: woensdag, 14 januari 2026 15:41
Aan: Ronald Klop <[email protected]>, Current FreeBSD
<[email protected]>
Onderwerp: Re: Change fetch addresses for Rust crates, Python packages and
whatnot?
On Wednesday, January 14th, 2026 at 4:11 AM, Ronald Klop <[email protected]>
wrote:
>
>
> Van: ruby R53 <[email protected]>
> Datum: woensdag, 14 januari 2026 03:26
> Aan: [email protected]
> Onderwerp: Change fetch addresses for Rust crates, Python packages and
whatnot?
>
> > I've been trying to compile www/chromium for literal days now, all because
whenever `make` runs `fetch`, it goes to an address that's actually supposed to
redirect to another one, but `fetch` instead keeps trying for a whole minute until it
ultimately falls back to distcache.freebsd.org, which is another whole minute of
waiting for the file to download. The cherry on top is that the file is, most of the
time, just hundreds of kilobytes in size.
> >
> > Hell, I even had to manually download part of these individual files using
that Links web browser··· it'd do it a lot faster than `fetch`, all because it
actually followed the redirection and didn't just keep trying over and over. But of
course I wouldn't manually download hundreds upon hundreds of archive files, this is
the reason why such tool exists.
> >
> > So here's an example: currently one of the missing dependencies for
www/chromium is GTK3, which depends on graphics/librsvg2-rust, which then depends on
cargo-c. It's now trying to fetch dunce-1.0.5.crate, which is one of the crates
required for it, from
https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate.
Running `curl` on this link does literally nothing, no files are downloaded. But
running it on any web browser instantly downloads the crate in question, all because
that link actually redirects to
https://static.crates.io/crates/dunce/dunce-1.0.5.crate. Even `curl`ing that new link
does give the file I want. Same applies for Python packages.
> >
> > Now imagine this for all the remaining files··· if it takes about 2 minutes
to download a single file, and there's 398 crate entries on cargo-c's
`Makefile.crates`, that means it'd take a whole 796 minutes to download everything,
which is literally more than half a day (~13.6 hours) :)
> >
>
>
>
> Can you copy-paste the output of the following command in a reply?
> fetch -vv
"https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate";
>
> For me it immediately follows the redirect and downloads the file in half a
second. See the output below.
>
> Regards,
> Ronald.
>
>
> --
>
> time fetch -vv
"https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate";
> scheme: "https"
> user: ""
> password: ""
> host: "crates.io"
> port: "0"
> document:
"/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate"
> ---> crates.io:443
> resolving server address: crates.io:443
> SSL options: 82004850
> Peer verification enabled
> Using OpenSSL default CA cert file and path
> Verify hostname
> TLSv1.3 connection established using TLS_AES_128_GCM_SHA256
> Certificate subject: /CN=crates.io
> Certificate issuer: /C=US/O=Amazon/CN=Amazon RSA 2048 M01
> requesting
https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate
> >>> GET
/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate HTTP/1.1
> >>> Host: crates.io
> >>> Accept: */*
> >>> User-Agent: fetch libfetch/2.0
> >>> Connection: close
> >>>
> <<< HTTP/1.1 302 Moved Temporarily
> <<< Content-Length: 0
> <<< Connection: close
> content length: [0]
> <<< Access-Control-Allow-Origin: *
> <<< Date: Wed, 14 Jan 2026 07:10:06 GMT
> <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate
> <<< Nel:
{"report_to":"heroku-nel","response_headers":["Via"],"max_age":3600,"success_fraction":0.01,"failure_fraction":0.1}
> 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate
> scheme: "https"
> user: ""
> password: ""
> host: "static.crates.io"
> port: "0"
> document: "/crates/dunce/dunce-1.0.5.crate"
> <<< Report-To:
{"group":"heroku-nel","endpoints":[{"url":"https://nel.heroku.com/reports?s=VOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D\u0026sid=af571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=1768374606"}],"max_age":3600}
> <<< Reporting-Endpoints:
heroku-nel="https://nel.heroku.com/reports?s=VOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&ts=1768374606";
> <<< Server: Heroku
> <<< Strict-Transport-Security: max-age=31536000; includeSubDomains
> <<< Via: 1.1 heroku-router, 1.1
c2905f891f96a0ec9c7fab16916dbb46.cloudfront.net (CloudFront)
> <<< X-Cache: Miss from cloudfront
> <<< X-Amz-Cf-Pop: AMS58-P6
> <<< X-Amz-Cf-Id: Xa10pwbYp4MCA5eToVa0shahNpzr2epWFUuQZS63bMpmbiDLKc2dmQ==
> <<<
> ---> static.crates.io:443
> resolving server address: static.crates.io:443
> SSL options: 82004850
> Peer verification enabled
> Using OpenSSL default CA cert file and path
> Verify hostname
> TLSv1.3 connection established using TLS_AES_128_GCM_SHA256
> Certificate subject: /CN=static.crates.io
> Certificate issuer: /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Atlas R3 DV TLS CA
2025 Q3
> requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate
> >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1
> >>> Host: static.crates.io
> >>> Accept: */*
> >>> User-Agent: fetch libfetch/2.0
> >>> Connection: close
> >>>
> <<< HTTP/1.1 200
> <<< Connection: close
> <<< Content-Length: 8244
> <<< etag: "198d7f7c37f632bc83502b28870b6d60"
> content length: [8244]
> <<< cache-control: public,max-age=31536000,immutable
> <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D
> <<< x-amz-id-2:
Dj7mFoaqQJUc61mB1FcouUQcy0tajnacJvU+xOR3Be5//jo3GaSmdCwSSgtYyDMnYE4cPCiPv7PV9m7z1Jk0WazYHTFPbYhm
> <<< x-amz-request-id: KWXPQ4JP20HSYSY2
> <<< date: Wed, 14 Jan 2026 00:09:56 GMT
> <<< x-amz-server-side-encryption: AES256
> <<< x-amz-replication-status: COMPLETED
> <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT
> <<< accept-ranges: bytes
> last modified: [2024-08-04 01:36:47]
> <<< content-type: application/gzip
> <<< x-served-by: cache-ams21058-AMS
> <<< server: AmazonS3
> <<< age: 25212
> <<< x-cache: HIT
> <<< x-cache-hits: 148
> <<<
> offset 0, length -1, size -1, clength 8244
> local size / mtime: 8244 / 1722735407
> remote size / mtime: 8244 / 1722735407
> dunce-1.0.5.crate 8244 B 10 MBps 00s
>
> real 0m0.429s
> user 0m0.023s
> sys 0m0.027s
```
# time fetch -vv
"https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate";
scheme: "https"
user: ""
password: ""
host: "crates.io"
port: "0"
document:
"/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate"
---> crates.io:443
resolving server address: crates.io:443 (<- it gets stuck here for like a
minute)
SSL options: 82004850
Peer verification enabled
Using OpenSSL default CA cert file and path
Verify hostname
TLSv1.3 connection established using TLS_AES_128_GCM_SHA256
Certificate subject: /CN=crates.io
Certificate issuer: /C=US/O=Amazon/CN=Amazon RSA 2048 M01
requesting
https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate
>>> GET
/api/v1/crates/dunce/1.0.5/download?dummy=/rust/crates/dunce-1.0.5.crate HTTP/1.1
>>> Host: crates.io
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 302 Moved Temporarily
<<< Content-Length: 0
<<< Connection: close
content length: [0]
<<< Access-Control-Allow-Origin: *
<<< Date: Wed, 14 Jan 2026 14:34:21 GMT
<<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate
<<< Nel:
{"report_to":"heroku-nel","response_headers":["Via"],"max_age":3600,"success_fraction":0.01,"failure_fraction":0.1}
302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate
scheme: "https"
user: ""
password: ""
host: "static.crates.io"
port: "0"
document: "/crates/dunce/dunce-1.0.5.crate"
<<< Report-To:
{"group":"heroku-nel","endpoints":[{"url":"https://nel.heroku.com/reports?s=9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D\u0026sid=af571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=1768401261"}],"max_age":3600}
<<< Reporting-Endpoints:
heroku-nel="https://nel.heroku.com/reports?s=9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&ts=1768401261";
<<< Server: Heroku
<<< Strict-Transport-Security: max-age=31536000; includeSubDomains
<<< Via: 1.1 heroku-router, 1.1 608555ff8069f08e821e1679cd412294.cloudfront.net
(CloudFront)
<<< X-Cache: Miss from cloudfront
<<< X-Amz-Cf-Pop: GRU3-P10
<<< X-Amz-Cf-Id: LvtqT2cBMH7thFfdy3ktelkhg-ALuajh-ucfDhnv7l_s73okAxXp7Q==
<<<
---> static.crates.io:443
resolving server address: static.crates.io:443
SSL options: 82004850
Peer verification enabled
Using OpenSSL default CA cert file and path
Verify hostname
TLSv1.3 connection established using TLS_AES_128_GCM_SHA256
Certificate subject: /CN=static.crates.io
Certificate issuer: /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Atlas R3 DV TLS CA
2025 Q3
requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate
>>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1
>>> Host: static.crates.io
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 200
<<< Connection: close
<<< Content-Length: 8244
<<< cache-control: public,max-age=31536000,immutable
content length: [8244]
<<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D
<<< x-amz-id-2:
EzDoao06Q7mYZgq0ecISLqKGCrHOV1gBvzNGaaAfOg4QCWZ/d3G4pjY4ZAoo2BNA0jACxYFWJQQ=
<<< x-amz-request-id: 7JJDJ8X4GESHDW5P
<<< date: Wed, 14 Jan 2026 04:21:53 GMT
<<< x-amz-replication-status: COMPLETED
<<< x-amz-server-side-encryption: AES256
<<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT
<<< etag: "198d7f7c37f632bc83502b28870b6d60"
last modified: [2024-08-04 01:36:47]
<<< accept-ranges: bytes
<<< content-type: application/gzip
<<< x-served-by: cache-gru-sbgr1930032-GRU
<<< server: AmazonS3
<<< age: 36813
<<< x-cache: HIT
<<< x-cache-hits: 107
<<<
offset 0, length -1, size -1, clength 8244
fetch: transfer timed out
fetch: dunce-1.0.5.crate appears to be truncated: 0/8244 bytes
184.03 real 0.02 user 0.00 sys
so it's even worse, it does follow the redirect but somehow can't get the file
at all, and all of that actually takes 3 minutes to do
That looks weird. It is hard to diagnose from a distance. How is you general
network access? Do you have a lot of package loss?
Regards,
Ronald.
