I am doing GET's on a long list of URL's to detect 404's etc. Out of 1000's of URL's, I have seen a few that consistently yield inconsistent behavior. :) http://www.ndu.edu/ndu/inss/strforum/forum83.html -- This url should return a 404, but LWP almost always times out with a server error (500). It's not a server problem, though, b/c GET works fine on http://www.ndu.edu. To complicate matters, when I run this script from a different machine, it consistently gets the 404. In a related note, on a small number of other URLs (eg., http://nces.ed.gov/nationsreportcard/guide/2000456.shtml), I have found LWP to be much more likely to time out than to get a correct response, whereas Netscape generally gets a prompt response. I have tried ua->agent("Netscape/4.7"), but that doesn't seem to be the issue. In the ndu.edu case, the first machine is a Solaris box on a LAN/ethernet connection. When I access it from a linux box via PPP, I get the 404. The Solaris setup runs perl 5.005_3 and LWP 5.48; the linux box has perl 5.6.0 and (I think) LWP 5.49. Note that I'm checking 1000's of other urls, and the script generally runs fine from either box. Note that my GETs are simple_requests, b/c I need to handle redirects manually.
