On Mon, 23 Nov 2009, Ken Irwin wrote:
Hi all,
I'm moving to a new web server and struggling to get it configured properly. The problem of the
moment: having a Perl CGI script call another web page in the background and make decisions
based on its content. On the old server I used an antique Perl script called "hcat"
(from the Pelican book<http://oreilly.com/openbook/webclient/ch04.html>); I've also tried
curl and LWP::Simple.
In all three cases, I get the same behavior: it works just fine on the command
line, but when called by the web server through a CGI script, the LWP (or other
socket connection) gets no results. It sounds like a permissions thing, but I
don't know what kind of permissions setting to tinker with. In the test script
below, my command line outputs:
Content-type: text/plain
Getting URL: http://www.npr.org
885 lines
Whereas the web output just says "Getting URL: http://www.npr.org"; - and doesn't even get
to the "Couldn't get" error message.
Any clue how I can make use of a web page's contents from w/in a CGI script?
(The actual application has to do with exporting data from our catalog, but I
need to work out the basic mechanism first.)
Here's the script I'm using.
#!/bin/perl
use LWP::Simple;
print "Content-type: text/plain\n\n";
my $url = "http://www.npr.org";;
print "Getting URL: $url\n";
my $content = get $url;
die "Couldn't get $url" unless defined $content;
@lines = split (/\n/, $content);
foreach (@lines) { $i++; }
print "\n\n$i lines\n\n";
Any ideas?
I'd suggest testing the results of the call, rather than just looking for
content, as an empty response could be a result of the server you're
connecting to. (unlikely in this case, but it happens once in a while,
particularly if you turn off redirection, or support caching).
Unfortunately, you might have to use LWP::UserAgent, rather than
LWP::Simple:
#!/bin/perl --
use strict; use warnings;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new( timeout => 60 );
my $response = $ua->get('http://www.npr.org/');
if ( $response->is_success() ) {
my $content = $response->decoded_content();
...
} else {
print "HTTP Error : ",$response->status_line(),"\n";
}
__END__
(and changing the shebang line for my location of perl, your version
worked via both CGI and command line)
oh ... and you don't need the foreach loop:
my $i = @lines;
-Joe
