Hi Ryan, It is happier. If-Modified-Since is now properly handled. Thank you. Would you mind sharing what you changed? I noticed that the server is now nginx instead of varnish. I don't know how Fastly works, but it would be nice if the config was in an open repository.
Regards, Ronald. Van: Ryan Steinmetz <[email protected]> Datum: zaterdag, 10 januari 2026 17:32 Aan: Ronald Klop <[email protected]> CC: [email protected] Onderwerp: Re: fetch on pkg.freebsd.org does not handle If-Modified-Since properly
This should be happier now. -r On (01/10/26 15:10), Ronald Klop wrote: >Hi, > >I have a script which regularly fetch some files from pkg.freebsd.org for keeping statistics. (https://www.klop.ws/pkgstats/pkg-age.html) > >The command it executes is: "fetch -vv -o /home/ronald/verify_pkgs.sh/downloads/FreeBSD:13:armv7_latest_data.pkg --if-modified-since=/home/ronald/verify_pkgs.sh/downloads/FreeBSD:13:armv7_latest_data.pkg https://pkg.freebsd.org/FreeBSD:13:armv7/latest/data.pkg"; > >Mind that the request header If-Modified-Since has the exact same time as the response header Last-Modified. >If-Modified-Since: Sun, 05 Oct 2025 04:47:06 GMT >Last-Modified: Sun, 05 Oct 2025 04:47:06 GMT > >Still the response says HTTP/1.1 200 OK instead of 304 Not Modified. > >I've had this in my projects if the File.lastModified() returns a date with milliseconds, while the http-request time always uses 0 for milliseconds. In that case the file is always newer. >NB: not all fetch requests have this behaviour. Some give a 304 response. > >Anyway, maybe the cause is something else. Would be nice to fix this optimization. > >Here the verbose output of the fetch command: >scheme: "https" >user: "" >password: "" >host: "pkg.freebsd.org" >port: "0" >document: "/FreeBSD:13:armv7/latest/data.pkg" >---> pkg.freebsd.org:443 >resolving server address: pkg.freebsd.org:443 >SSL options: 82004850 >Peer verification enabled >Using OpenSSL default CA cert file and path >Verify hostname >TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 >Certificate subject: /CN=pkg.freebsd.org >Certificate issuer: /C=US/O=Certainly/CN=Certainly Intermediate R1 >requesting https://pkg.freebsd.org/FreeBSD:13:armv7/latest/data.pkg >>>>GET /FreeBSD:13:armv7/latest/data.pkg HTTP/1.1 >If-Modified-Since: Sun, 05 Oct 2025 04:47:06 GMT >>>>If-Modified-Since: Sun, 05 Oct 2025 04:47:06 GMT >>>>Host: pkg.freebsd.org >>>>Accept: */* >>>>User-Agent: fetch libfetch/2.0 >>>>Connection: close >>>> ><<< HTTP/1.1 200 OK > ><<< Connection: close > ><<< Content-Length: 9137918 > ><<< Server: nginx > >content length: [9137918] ><<< Content-Type: application/octet-stream > ><<< Last-Modified: Sun, 05 Oct 2025 04:47:06 GMT > ><<< ETag: "68e1f84a-8b6efe" > >last modified: [2025-10-05 04:47:06] ><<< X-Content-Type-Options: nosniff > ><<< X-XSS-Protection: 1; mode=block > ><<< X-Frame-Options: SAMEORIGIN > ><<< Accept-Ranges: bytes > ><<< Via: 1.1 varnish, 1.1 varnish > ><<< Date: Sat, 10 Jan 2026 11:04:02 GMT > ><<< X-Served-By: cache-chi-kigq8000150-CHI, cache-ams2100142-AMS > ><<< X-Cache: MISS, MISS > ><<< X-Cache-Hits: 0, 0 > ><<< X-Timer: S1768043043.639628,VS0,VE106 > ><<< Cache-Control: no-cache,no-store,max-age=0 > ><<< > >offset 0, length -1, size -1, clength 9137918 >local size / mtime: 9137918 / 1759639626 >remote size / mtime: 9137918 / 1759639626 >/home/ronald/verify_pkgs.sh/downloads/FreeBSD:13:armv7_latest_d 8923 kB 20 MBps 01s > > >Regards, >Ronald. > -- Ryan Steinmetz PGP: 4143 0BDC 21B0 B3D9 6D47 F0DA 504D C825 7FAD 8C58
