Nevermind, I got it. Below is the code. Thanks to all for everyone's help.
What a GREAT list. Never belonged to one with people who responded so
quickly. Thanks again,
Kenny
se strict;
use HTTP::Request::Common;
use HTML::Form;
use LWP::UserAgent;
#use LWP::Debug qw(+);
use HTTP::Cookies;
use FileHandle;
#BEGIN
#{
# local $^W = 0;
# *LWP::UserAgent::redirect_ok = sub {1}
#}
my $ua = LWP::UserAgent->new();
$ua->agent("Automation/0.1");
$ua->cookie_jar(
HTTP::Cookies->new( autosave => 1 ));
my $request = $ua->request( POST
'http://online.wsj.com/login?URI=%2Fdocuments%2Fdatabank.htm');
my $form = HTML::Form->parse( $request->content, $request->base());
#print $form->dump;
$form->value( 'Login', "kcmria" );
$form->value( 'Password', "money" );
my $response = $ua->request( $form->click() );
$response = $ua->request(GET
'http://online.wsj.com/documents/databank.htm');
my @outputLines = split(/\n/, $response->content);
my $outputFileHandle = new FileHandle;
$outputFileHandle->open(">d:\\etl\\temp\\temp.html");
foreach my $line (@outputLines)
{
print $outputFileHandle $line;
}
$outputFileHandle->close;
#print $response ->as_string;
----- Original Message -----
From: "Kenny G. Dubuisson, Jr." <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, July 09, 2002 3:01 PM
Subject: Fw: Logging in to form based log in page
> When I manually do this process, after I click the log in button, I get a
> dialog box warning stating that I'm about to transfer to a non-secure
page.
> I have to click "OK" and then the browser (IE) goes on to the desired
page.
> Do you think that the dialog box could be hangin it up? Do you know of a
> way to clear that dialog box from the code?
>
> Thanks again,
> Kenny
>
> ----- Original Message -----
> From: "Hill, Ronald" <[EMAIL PROTECTED]>
> To: "'Kenny G. Dubuisson, Jr.'" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> Sent: Tuesday, July 09, 2002 2:53 PM
> Subject: RE: Logging in to form based log in page
>
>
> >
> > [snipped]
> > >
> > > Ron:
> > >
> > > Does it have to select the second one...i.e. would the first
> > > (default) one
> > > work? Thanks,
> > > Kenny
> > I believe it is selecting the first one, and it will just hang. Here is
> the
> > output from the debug
> >
> > LWP::UserAgent::new: ()
> > LWP::UserAgent::request: ()
> > HTTP::Cookies::add_cookie_header: Checking online.wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .com for cookies
> > LWP::UserAgent::send_request: POST http://online.wsj.com/login
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > LWP::Protocol::collect: read 695 bytes
> > LWP::Protocol::collect: read 1896 bytes
> > LWP::Protocol::collect: read 1505 bytes
> > LWP::Protocol::collect: read 1017 bytes
> > LWP::Protocol::collect: read 1207 bytes
> > LWP::Protocol::collect: read 208 bytes
> > HTTP::Cookies::extract_cookies: Set cookie UBID =>
> > Tue%20Jul%20%209%2015:36:37
> > 0EDT%202002%200.610083999
> > LWP::UserAgent::request: Simple response: OK
> >
> > This is the form Dump:
> >
> > POST https://online.wsj.com/wsjra/servlet/WSJAuthReg
> > URI= (hidden)
> > From=wsjauth (hidden)
> > Login=<UNDEF>
> > Password=<UNDEF> (password)
> > <NONAME>=Log In (submit)
> > <NONAME>=Log In (submit)
> > save_wsjie_login=<UNDEF> (checkbox) [*<UNDEF>|yes]
> >
> > end Dump
> >
> > LWP::UserAgent::request: ()
> > HTTP::Cookies::add_cookie_header: Checking online.wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: - checking cookie path=/
> > HTTP::Cookies::add_cookie_header: - checking cookie
> > UBID=Tue%20Jul%20%209%201
> > 36:37%20EDT%202002%200.610083999
> > HTTP::Cookies::add_cookie_header: it's a match
> > HTTP::Cookies::add_cookie_header: Checking wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .com for cookies
> > LWP::UserAgent::send_request: POST
> > https://online.wsj.com/wsjra/servlet/WSJAut
> > eg
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > HTTP::Cookies::extract_cookies: Set cookie fastlogin =>
> > QZWfEwhiRuuCViHu1ZqiUl
> > MPPcGSxzZPqmHeadfYK9Y%3D
> > HTTP::Cookies::extract_cookies: Set cookie user_type => subscribed
> > HTTP::Cookies::extract_cookies: Set cookie REMOTE_USER => kcmria
> > HTTP::Cookies::extract_cookies: Set cookie wsjperm => 631/51
> > LWP::UserAgent::request: Simple response: Found
> > LWP::UserAgent::request: ()
> > HTTP::Cookies::add_cookie_header: Checking online.wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: - checking cookie path=/
> > HTTP::Cookies::add_cookie_header: - checking cookie
> > UBID=Tue%20Jul%20%209%201
> > 36:37%20EDT%202002%200.610083999
> > HTTP::Cookies::add_cookie_header: it's a match
> > HTTP::Cookies::add_cookie_header: - checking cookie wsjperm=631/51
> > HTTP::Cookies::add_cookie_header: it's a match
> > HTTP::Cookies::add_cookie_header: - checking cookie REMOTE_USER=kcmria
> > HTTP::Cookies::add_cookie_header: it's a match
> > HTTP::Cookies::add_cookie_header: - checking cookie
user_type=subscribed
> > HTTP::Cookies::add_cookie_header: it's a match
> > HTTP::Cookies::add_cookie_header: - checking cookie
> > fastlogin=QZWfEwhiRuuCViH
> > ZqiUlBsMPPcGSxzZPqmHeadfYK9Y%3D
> > HTTP::Cookies::add_cookie_header: it's a match
> > HTTP::Cookies::add_cookie_header: Checking wsj.com for cookies
> > HTTP::Cookies::add_cookie_header: Checking .com for cookies
> > LWP::UserAgent::send_request: POST http://online.wsj.com/home
> > LWP::UserAgent::_need_proxy: Not proxied
> > LWP::Protocol::http::request: ()
> > ^C
> > F:\scripts>
> > If I am reading this correctly it is going to https (a secure web page)
> then
> > the last steep
> > is going to http (non secure) So that would tell me the click function
is
> > selecting the
> > secure login button not the standard one.
> > I think now is the time to get Sean's new book!! Like I said, if someone
> > knows a way
> > of selecting the second select button that is not named I would like to
> > know.
> > Anyway I hope this helps!!
> >
> > Ron Hill
