LWP reports a server error when the server is fine, b/c of the following obscure problem: There is an interaction between Solaris and certain web servers (notably Microsoft-IIS/4.0) where the response to a GET comes back improperly terminated. You can observe this independently of LWP by doing a "telnet www.ndu.edu 80", and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html" You will see telnet receive the response (it's a 404), but then (usually but not always) fail to terminate. As far as I know, you will only see this on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried same under Linux and haven't seen this problem. I have seen this on a number of other sites. The problem seems to be that LWP reports a server error (500) b/c it times out while waiting for the missing EOF terminator. I've turned on LWP::Debug and see it hang, apparently in the LWP::Protocol::http->request subroutine, or possible in collect(), though I'm not sure. You can see that LWP received the response, so it ought to do more than just report a timeout. Here is the output from a typical run: LWP::UserAgent::new: () HTTP::Cookies::add_cookie_header: Checking www.ndu.edu for cookies HTTP::Cookies::add_cookie_header: Checking .ndu.edu for cookies LWP::UserAgent::simple_request: GET http://www.ndu.edu/ndu/inss/strforum/forum83.html LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::http::request: GET /ndu/inss/strforum/forum83.html HTTP/1.0 Host: www.ndu.edu Range: bytes=0-524288 User-Agent: libwww-perl/5.48 LWP::Protocol::http::request: reading response LWP::Protocol::http::request: HTTP/1.1 404 Object Not Found Server: Microsoft-IIS/4.0 Date: Thu, 25 Jan 2001 16:38:46 GMT Content-Length: 461 Content-Type: text/html <html><head><title>Error 404</title> <meta name="robots" content="noindex"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"></head> <body> <h2>HTTP Error 404</h2> <p><strong>404 Not Found</strong></p> <p>The Web server cannot find the file or script you asked for. Please check the URL to ensure that the path is correct.</p> <p>Please contact the server's administrator if this problem persists.</p> </body></html> LWP::Protocol::http::request: HTTP/1.1 404 Object Not Found LWP::Protocol::collect: read 461 bytes [This is where LWP hangs...]
