My apologies if this ends up at wrong place in the mailing list. For some
reason an email wasn't sent to me when a reply was made. As such, I was unable
to directly reply to the previous response by Guenter.
I've tracked it down, or so I believe, to LWP+https being the cause here. I've
managed to replicate the same issue on Debian 6.0 (Squeeze) as well as the OS X
Lion system I run. As best I can tell, when LWP is downloading the file over
HTTPS sometimes the request ends early. I was not able to replicate using
curl, wget, or Google Chrome.
I tested this my commenting out the '$ua->mirror($url, $txt);' line, and
replacing it with the following line:
print $ua->get($url)->content . "\n"; exit(0);
The first time I ran the script the entire file was downloaded. However, it
was a fluke, as the following request attempts varying amounts of the file
contents were downloaded. And I was never able to print the full file again.
I even tried playing with the "max_size" option in UserAgent. This had no
affect, as expected, because the "max_size" option defaults to undef
(unlimited). When switching to http, it works perfectly every single time.
I've tried to do some digging and have been unable to find a discussion
regarding the change from HTTP to HTTPS. Additionally, I can't say I see any
benefit of doing so as the information is publicly accessible. I would be
interested in why it should not remain HTTP.
-Tim
---
Tim Heckman
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
