On Thu, 24 Oct 2019 07:30 +11:00, L A Walsh <ast...@tlinx.org> wrote: > On 2019/10/21 17:39, Bradley Dean wrote: > > > > Using if-modified-since relies on the server you're querying supporting that > ---- > It "seems" it must support it separately, "I guess"...because (see below)...
>From perldoc LWP::UserAgent (see >https://metacpan.org/pod/LWP::UserAgent#mirror) $ua->mirror( $url, $filename ) This method will get the document identified by $url and store it in file called $filename. If the file already exists, then the request will contain an "If-Modified-Since" header matching the modification time of the file. If the document on the server has not changed since this time, then nothing happens. If the document has been updated, it will be downloaded again. The modification time of the file will be forced to match that of the server. The return value is the the response object. Which is using the If-Modified-Since header (when the local file exists) and the server makes the decision (whether or not the Last-Modified header is available or making it through a proxy to you). mirror accepts a HTTP 304 Not Modified and makes no changes. > >> Anyone know the call/header value to use for getting at the remote > >> object's last-modified time directly? It will be in the response if it's supported by the server (and making it through proxies between you and the server). > One thing I was curious about -- when I call mirror (vs. get), I get > back > an HTTP::Response object. One of the fields of that object is the > 'content' value. I'm wondering if that is read in dynamically if > 'accessed', since I'm > pulling down RPM packages which can be rather large (the 10 largest packages > are over 1GB each!). It certainly wouldn't just store them in the response > object, by default (would it?), since if I had references to several objects > at the same time, in memory, that could be a huge memory load. > > I can see I'm gonna want to find a way to filter some of these packages > before I mirror them since I don't really need the debuginfo for > multiple kernel > versions I'm not running nor various cross-compiler packages (Oi!) > > Whether dynamic or not, guess I'll find out -- can't believe how huge > some of those packages are! I've not checked specifically - but I believe mirror uses a content_file so the HTTP::Response will not have the content in memory. See content file details in https://metacpan.org/pod/LWP::UserAgent#get , https://metacpan.org/pod/LWP::UserAgent#request Cheerio, Brad -- Bradley Dean Email: bjd...@bjdean.id.au WWW: http://bjdean.id.au/
