I been using LWP to retrieve selective web pages and it seems not to do a
very good job with it. Many URL's it just seems to hang and become
non-responsive till I step in and kill it :( Here is my sample code below
with one of the many URL's that LWP just does not respond with in hopes maybe
someone can figure out a solution by observing it in action:
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0") ;
$req = HTTP::Request->new(GET => 'http://www.stinky.com/alex/resume.html');
$req->header('Accept' => 'text/html');
# send request
$res = $ua->request($req);
# check the outcome
if ($res->is_success) {
print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}