Thanks to Doug Monroe, I am able to retrieve documents both from intra and
internet. Here is my script.
Most organizations have a .pac (proxy configuration file). This determines
how requests are routed. But for LWP you need to use an IP or a regular dns
entry.
I figured out IP used for internet access in our case by sending a GET
request to the proxy itself. That returned me the document how it was
determining where it routes a request based on the URL. Picked up the IP for
internet request from there and plugged into the $ua->proxy.
#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;
my $ua = new LWP::UserAgent;
#some servers do not respond if they cannot identify the agent.
$ua->agent("Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)");
my $arg = shift || 'http://localhost/';
$ua->proxy(['http', 'ftp'] => 'http://141.207.58.168:80'); #note the port
$ua->no_proxy('localhost','bam.com');
my $req = new HTTP::Request 'GET', $arg;
$req->proxy_authorization_basic('alladin','opensesame');
my $res = $ua->request($req);
if ($res->is_success) {
print $res->as_string; #remove the noise
} else {
print "Failed: ", $res->status_line, "\n";
}
