I am trying to use LWP::UserAgent and cookie_jar, obviously according to the subject
of this email.
Anyhow, I cannot get my code to work, can someone look at it and tell me if I am just
using it wrong?
I'll show my code and the response I get...
I would really appreciate any help someone can offer.
Code: (Of course I changed all real domains and user/membership info...)
my $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/8.0"); # pretend we are very capable browser
$ua->requests_redirectable;
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1));
my $req = HTTP::Request->new(POST =>
'http://www1.somedomain.com/weborder/xt_shopper_lookup.asp');
$req->content_type('application/x-www-form-urlencoded');
$req->content('shopper_memno=12345124&shopper_password=somepass&submit1=Continue');
$req->referer('http://www1.somedomain.com/weborder/xt_shopper_lookup.asp');
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
# Check the outcome of the response
if ($res->is_success) {
my $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/8.0"); # pretend we are very capable browser
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1));
$req = HTTP::Request->new(GET =>
"http://ftp.thedomainsftpsite.com/default.asp?memno=12345124";);
$req->referer('http://www1.somedomain.com/weborder/xt_shopper_lookup.asp');
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
if($res->is_success) { # Check to see if I can access the FTP site...
# Yep, it appears to have let me login... This is where I'm getting the error
that, "Login Required"...
# When I go to the website and login manually, I can just click on the FTP site to
go download files from there, like txt descriptions and stuff, one at a time.
# So this SHOULD work with the cookie jar...
$sth = $dbh->prepare (qq{ SELECT * FROM `my_items` ORDER BY `id` });
$sth->execute();
my $_cntr = 0;
while($rs = $sth->fetchrow_hashref()) {
$_pid = $rs->{pid};
$file_name = "/home/userid/files/Descriptions/" . $_pid . ".txt";
if(!-e "/$file_name/i") {# Ok it does not already exist, so go get it...
$_sub_folder = Return_Sub_Folder($_pid); # Get the folder it would be in,
based upon the ID number of it...
my $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/8.0"); # pretend we are very capable browser
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1));
$req = HTTP::Request->new(GET =>
"http://ftp.thedomainsftpsite.com/items/descriptions/$_sub_folder/$_pid.txt";);
# $req->header('Accept' => 'text/plain'); # Don't want to use this
right now...
$req->referer("http://ftp.thedomainsftpsite.com/items/descriptions/$_sub_folder/";);
# send request
$res = $ua->request($req);
if ($res->is_success) { # Check if it can get the .txt file description to
save it to my account for later use...
# COOL IT WORKED!!!
$ctypeb = $res->content;
$_output_filename = "/home/userid/files/Descriptions/" . $_pid .
".txt";
open (OUTFILE,">$_output_filename");
print OUTFILE $ctypeb; # SAVE IT!
close(OUTFILE);
} else {
#Nope still did not work. Maybe it's .TXT instead of .txt lets try
again with the .TXT extension
my $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/8.0"); # pretend we are very capable browser
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1));
$req = HTTP::Request->new(GET =>
"http://ftp.thedomainsftpsite.com/items/descriptions/$_sub_folder/$_pid.TXT";);
# $req->header('Accept' => 'text/plain'); # Don't want to use this
right now...
$req->referer("http://ftp.thedomainsftpsite.com/items/descriptions/$_sub_folder/";);
# send request
$res1 = $ua->request($req);
if ($res1->is_success) {
$ctypeb = $res1->content;
$_output_filename = "/home/userid/files/Descriptions/" . $_pid .
".txt";
open (OUTFILE,">$_output_filename");
print OUTFILE $ctypeb;
close(OUTFILE);
} else {# Nope still did not work, please send me the error via email:
$_mail_message .= qq~I could not find $_pid.txt on
ourmemebersite.com so I could not add it to the directory.\n\nRun number 3: Content
says:\n$res1->content\n\nStatus Line says:\n$res->status_line\n\n~;
}
}
}
....
Response:
bash$ perl text_desc_dl.cgi
Content-type: text/html/
Login Failed....
Thank you.
Richard
