Hi everyone,
I am trying to use Mechanize to scrape information of my account page at www.attwireless.com. am having problems signing on.
The commands to fill in the form all work without complaint:
$mech->form_name($formname); $mech->field($usernameField,$loginID); $mech->field($passwordField,$pswd);
However, when I submit the form, I don't get logged in, I come back to the Login page (which is the target of the POST). There is no error message on the login page, but it doesn't let me in either.
When I sign on with Firefox, everything works fine. The only difference I have been able to determine, compared to the trace from LWP::Debug, is that a cookie named phonenumberflag is set to "false" in LWP/Mechanize and is "true" in Firefox. (Although I admit I don't know how to create/interpret the dumps described in Colin Magee's Thread Viewing exchange between browser and website. <http://www.mail-archive.com/libwww@perl.org/msg05155.html>)
Anyone have any ideas on what the problem could be, how to find it or how to fix it? Thanks in advance!
Peter Stevens
Sample Code:
#!/usr/local/bin/perl -w ######### ######### ######### ######### ######### ######### ######### ######### # bug.pl - log into AT_and_T user account # use strict; use LWP::Debug qw(+); use WWW::Mechanize;
############ start
sub load{my ($loginID, $pswd) = @_ ;
my ($mech) ;
$mech = WWW::Mechanize->new( autocheck => 0 );
my $homepage="http://www.attwireless.com"; ;
my $formname="LoginForm"; # from mech_dump/source my $usernameField="/attws/ocx/userprofiling/OCXProfileFormHandler.value.login" ; my $passwordField="/attws/ocx/userprofiling/OCXProfileFormHandler.value.password" ;
$mech->agent_alias ( "Windows IE 6" ); print STDERR "Get $homepage \n"; $mech->get( $homepage ); die "Can't even get $homepage: " . $mech->response->status_line unless ($mech->success) ; print STDERR "Got $homepage as ". $mech->uri() . "\n";
print STDERR "fill in form...\n"; $mech->form_name($formname); $mech->field($usernameField,$loginID); $mech->field($passwordField,$pswd); # print STDERR "Login ID is $loginID , Pass is $pswd\n"; print STDERR "submit form...\n"; $mech->submit();
die "Could not submit $homepage: ". $mech->response->status_line unless
($mech->success) ;
print STDERR "Login brought us to ". $mech->uri() . "\n";
if ( $mech->uri() =~ /Login/i ) {
print STDERR "Wrong answer\n";
}# print $mech->content; }
my ($parms, $i) ; $parms = {} ;
foreach $i ( qw (loginID pswd) ) {
print "$i: " ;
$parms->{$i} = readline STDIN ;
chomp $parms->{$i};
} load($parms->{"loginID"}, $parms->{"pswd"});
# end of bug.pl
######### ######### ######### ######### ######### ######### ######### ####################Dump from LWP::Debug (indented, prints are left flush):
LWP::UserAgent::new: () Get http://www.attwireless.com LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.attwireless.com for cookies HTTP::Cookies::add_cookie_header: Checking .attwireless.com for cookies HTTP::Cookies::add_cookie_header: Checking attwireless.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: GET http://www.attwireless.com LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 475 bytes [ snip ] ... LWP::Protocol::collect: read 1010 bytes HTTP::Cookies::extract_cookies: Set cookie DSESSIONID => H0HZKOGKXE4AVB4R0HYSFEY HTTP::Cookies::extract_cookies: Set cookie phonenumberflag => false HTTP::Cookies::extract_cookies: Set cookie browserid => 1100175229559H0HZKOGKXE4AVB4R0HYSFEY LWP::UserAgent::request: Simple response: OK Got http://www.attwireless.com as http://www.attwireless.com fill in form... submit form... LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.attwireless.com for cookies HTTP::Cookies::add_cookie_header: - checking cookie path=/ HTTP::Cookies::add_cookie_header: - checking cookie DSESSIONID=H0HZKOGKXE4AVB4R0HYSFEY HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie phonenumberflag=false HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie browserid=1100175229559H0HZKOGKXE4AVB4R0HYSFEY HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: Checking .attwireless.com for cookies HTTP::Cookies::add_cookie_header: Checking attwireless.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST https://www.attwireless.com/ocs/authenticate/Login.jhtml;dsessionid=H0HZKOGKXE4AVB4R0HYSFEY?_DARGS=/home/left_frag_b2c.jhtml LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 658 bytes LWP::Protocol::collect: read 1 bytes [ snip ] LWP::Protocol::collect: read 182 bytes HTTP::Cookies::extract_cookies: Set cookie browserid => 1100175229559H0HZKOGKXE4AVB4R0HYSFEY LWP::UserAgent::request: Simple response: OK Login brought us to https://www.attwireless.com/ocs/authenticate/Login.jhtml;dsessionid=H0HZKOGKXE4AVB4R0HYSFEY?_DARGS=/hom e/left_frag_b2c.jhtml Wrong answer
###########End Dump from LWP::Debug:
