Ok,
I am trying a fairly simple perl web client using LWP::UserAgent . What
I am trying to do is to simply login to Yahoo. Yahoo requires cookies
which at face value should not be a problem. However, the response I get
from Yahoo's server is an immediate redirect to a "please enable
cookies" page. Apparently I am unable to convey that I am accepting
cookies yet by all accounts(i.e. extensive UseNet and www searches) the
code below should work. Code is below as is debug output. My apologies
for such a long post. Does anyone have any ideas they care to share. Any
help is appreciated.
Best Regards,
Adam Russell
#!perl
use HTTP::Request::Common;
use HTTP::Cookies;
use LWP::UserAgent;
BEGIN { *LWP::UserAgent::redirect_ok = sub { 1 } }#used to allow
redirects
use LWP::Debug qw(+);
$cookie_monster=HTTP::Cookies->new();
$ua=new LWP::UserAgent;
$ua->cookie_jar($cookie_monster);
$ua->agent('LWPAgent)');
$ua->from('[EMAIL PROTECTED]');
$request=POST 'http://login.yahoo.com/config/login',
[".tries"=>'1',
".done"=>'http://clubs.yahoo.com/clubs/',
".src"=>'grp',
".intl"=>'us',
"login"=>'username',
"passwd"=>'password',
];
$request->content_type('application/x-www-form-urlencoded');
$request->header('Accept' => '*/*');
$request->header('Allowed' => 'GET HEAD PUT');
$response=$ua->request($request);
$cookie_monster->extract_cookies($response);
$cookie_monster->save();
if ($response->is_success) #This always FAILS !!
{
$request->content_type('application/x-www-form-urlencoded');
$cookie_monster->load();
$cookie_monster->add_cookie_header($request);
$response=$ua->request($request);
print $response->content;
}
else
{
print $response->as_string;
}
DEBUG OUPUT
The document has moved <A
HREF="http://login.yahoo.com/config/verify?.done=http%3a//clubs.yahoo.com/clubs/">here</A>.<P>
LWP::Protocol::http::request: HTTP/1.0 302 Found
LWP::Socket::pushback: (121 bytes)
LWP::Protocol::http::__ANON__: Collecting
LWP::Socket::read: (...)
LWP::Protocol::collect: read 121 bytes
LWP::Protocol::http::__ANON__: Collecting
LWP::Socket::read: (...)
LWP::IO::read: Read 0 bytes
HTTP::Cookies::extract_cookies: Set cookie B => eotfuv8tc5hn4&b=2
HTTP::Cookies::extract_cookies: Set cookie Y =>
v=1&n=as5pcop3nl63s&l=ec8ii8edz/o&p=m2a02qq4110002&r=5d&lg=us&intl=us
HTTP::Cookies::extract_cookies: Set cookie T =>
z=kbsw6AkhBx6AcnmFL07Dv9iNk8GMU40MjA3NU9O&a=YAE&sk=DAA/2CQ4bk5dmR&d=c2wBTVRnQk5qa3pOVGN3TWpnNQFhAVlBRQFvawFaVzAtAXp6AWtic3c2QWdXQQ--
HTTP::Cookies::extract_cookies: Set cookie I =>
i1=1u26494g7a7d7gb3bdblbpbrc0csd3&ir=7i
LWP::UserAgent::request: Simple result: Moved Temporarily
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: POST
http://login.yahoo.com/config/verify?.done=http%3a//clubs.yahoo.com/clubs/
LWP::UserAgent::_need_proxy:
(http://login.yahoo.com/config/verify?.done=http%3a//clubs.yahoo.com/clubs/)
LWP::UserAgent::_need_proxy: Not proxied
HTTP::Cookies::add_cookie_header: Checking login.yahoo.com for cookies
HTTP::Cookies::add_cookie_header: Checking .yahoo.com for cookies
HTTP::Cookies::add_cookie_header: - checking cookie path=/
HTTP::Cookies::add_cookie_header: - checking cookie
I=i1=1u26494g7a7d7gb3bdblbpbrc0csd3&ir=7i
HTTP::Cookies::add_cookie_header: it's a match
HTTP::Cookies::add_cookie_header: - checking cookie
Y=v=1&n=as5pcop3nl63s&l=ec8ii8edz/o&p=m2a02qq4110002&r=5d&lg=us&intl=us
HTTP::Cookies::add_cookie_header: it's a match
HTTP::Cookies::add_cookie_header: - checking cookie B=eotfuv8tc5hn4&b=2
HTTP::Cookies::add_cookie_header: it's a match
HTTP::Cookies::add_cookie_header: - checking cookie
T=z=kbsw6AkhBx6AcnmFL07Dv9iNk8GMU40MjA3NU9O&a=YAE&sk=DAA/2CQ4bk5dmR&d=c2wBTVRnQk5qa3pOVGN3TWpnNQFhAVlBRQFvawFaVzAtAXp6AWtic3c2QWdXQQ--
HTTP::Cookies::add_cookie_header: it's a match
LWP::Protocol::http::request: ()
LWP::Socket::new: Socket GLOB(0x7691560)
LWP::Socket::connect: (login.yahoo.com, 80)
LWP::Socket::_getaddress: resolving host 'login.yahoo.com'...
LWP::Socket::_getaddress: ...64.58.76.98
LWP::Socket::_getaddress: ...64.58.76.99
LWP::Socket::connect: Connecting to host 'login.yahoo.com' on port
'80'...
LWP::Socket::write: ()
LWP::IO::write: Write 615 bytes: 'POST
/config/verify?.done=http%3a//clubs.yahoo.com/clubs/ HTTP/1.0
Accept: */*
From: [EMAIL PROTECTED]
Host: login.yahoo.com
User-Agent: Mozilla/4.73C-CCK-MCD {C-UDP; EBM-APPLE} (Macintosh; U; PPC)
Content-Length: 102
Content-Type: application/x-www-form-urlencoded
Allowed: GET HEAD PUT
Cookie: I="i1=1u26494g7a7d7gb3bdblbpbrc0csd3&ir=7i";
Y="v=1&n=as5pcop3nl63s&l=ec8ii8edz/o&p=m2a02qq4110002&r=5d&lg=us&intl=us";
B="eotfuv8tc5hn4&b=2";
T="z=kbsw6AkhBx6AcnmFL07Dv9iNk8GMU40MjA3NU9O&a=YAE&sk=DAA/2CQ4bk5dmR&d=c2wBTVRnQk5qa3pOVGN3TWpnNQFhAVlBRQFvawFaVzAtAXp6AWtic3c2QWdXQQ--"
Cookie2: $Version=1
'
LWP::Socket::write: ()
LWP::IO::write: Write 102 bytes:
'.tries=1&.done=http%3A%2F%2Fclubs.yahoo.com%2Fclubs%2F&.src=grp&.intl=us&login=username&passwd=pass'
LWP::Protocol::http::request: reading response
LWP::Socket::read: (...)
LWP::IO::read: Read 4096 bytes
LWP::IO::read: HTTP/1.0 200 OK
Date: Thu, 29 Mar 2001 05:23:49 GMT
Refresh: 900; URL=http://www.yahoo.com/
Cache-Control: private
Pragma: no-cache
Expires: Thu, 05 Jan 1995 22:00:00 GMT
Connection: close
Content-Type: text/html
Set-Cookie: B=b8jr16otc5hn5&b=2; expires=Thu, 15 Apr 2010 20:00:00 GMT;
path=/; domain=.yahoo.com
<html>
<head>
<title>Cookie Error</title>
</head>
<body>
<img height=43 width=392
src="http://us.i1.yimg.com/us.yimg.com/i/yahooidwelcome.gif"
alt="Welcome to Yahoo!">
<h2>An Error Occurred Setting Your User Cookie</h2>
Many Yahoo! services use cookies to remember who you are logged in as.
If the cookie cannot be set correctly, then Yahoo! cannot determine if
you
are logged in.
<p>
If you wish to know more about cookies, and how we use them click <b><a
href="#descrip">here</a></b>.
<a name="wrong">
<h2>So What Went Wrong?</h2>
There are many reasons why a cookie could not be set correctly, here
are the most common reasons:
<ul>
<li>Your browser does not support cookies. For those who are technically
inclined and particularly interested,
the full and gory details as to which particular browsers support
cookies can be found
<a
href="http://www.research.digital.com/nsl/formtest/stats-by-test/NetscapeCookie.html">here</a>.
<p>
<li>You have cookie notification switched on in your browser and you did
not accept the accept the cookie set request. To remedy this, hit your
browser's <b>back</b>
button and try again, except this time accepting the cookie.
<p>
<li>You have installed an application that monitors/blocks cookies
being set. To remedy this, please disable the application while logging
in.
</ul>
<hr>
<a name="descrip">
<h2>What is a Cookie?</h2>
A cookie is a small amount of data that is sent to your web browser from
a
web server. This data is never "executed" as code, so it can't contain
programs or viruses, and can't be longer than 4,000 characters. Cookies
are
generally used to help web sites determine the state of your "client."
<p>
Let's say, for instance, that you go to a shopping web site and purchase
something. Instead of going off to pay for that item right away, you
want
to continue browsing. A cookie can be "set" to indicate that you put
something in your "shopping cart", so that when you go to pay for all
your
items, the server has a way of knowing all the items you selected by
looking
at your cookie.
<p>
Yahoo! uses a cookie simply to record your Yahoo! ID. We do
this so that wherever you go in personalized Yahoo! properties,
we know exactly what to display
for you: your stock quotes, your sports, weather, news, sites, your
personal e-mail, chat rooms, etc.
<h2>Why does Yahoo! use Cookies</h2>
First,
we should make it clear that we only use cookies to help identity you to
our
system. When you first register with Yahoo!, the feedback that you
provide us helps us figure out what kind of news and information we
should
send you. We "set a cookie" on your browser with your UserID so that
when
you visit any Yahoo! properties, we know it's you and not your neighbor.
<P>
We also use
cookies so that you don't have to bother logging in everytime when you
visit
Yahoo!. We should also make clear that we do _not_ distribute your
cookie
information to other organizations. No other sites outside yahoo.com
can
access this cookie.
<H2>What about Security?</h2>
A cookie cannot but used to extract data from your system. When a
cookie is
set, your browser checks it for length, an expiration date, and the
domain
from which it came. This domain checking means that only Yahoo can
access
your My Yahoo cookie. We do _not_ distribute your cookie or account
information to other companies and do _not_ include your password in
your cookie.
<h2>Can other people see my Yahoo preferences?</h2>
Our Yahoo! ID/password scheme is used primarily to identify you to our
system.
It provides some measure of security, but it is certainly not foolproof.
If you select <a href="/help/rememberme.html">Remember my ID &
Password</a>
anyone with access to your computer can use your Yahoo account, while
you ar
LWP::Protocol::http::request: HTTP/1.0 200 OK
LWP::Socket::pushback: (3774 bytes)
LWP::Protocol::http::__ANON__: Collecting
LWP::Socket::read: (...)
LWP::Protocol::collect: read 3774 bytes
LWP::Protocol::http::__ANON__: Collecting
LWP::Socket::read: (...)
LWP::IO::read: Read 1286 bytes
LWP::IO::read: e "logged in".
You can "log out" at any time by clicking "Change User" or "Sign Out".
If you do not select <a href="/help/rememberme.html">Remember my ID
& Password</a>
the browser you are using will forget your Yahoo! ID and cookie when you
exit it.
<p>
We hope this information clarifies the role of cookies on Yahoo!. If
you're curious, you can check out our main site directory to find other
sources of information on cookies:
<p>
<li> <a
href="http://dir.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/HTTP/Information_and_Documentation/Persistent_Cookies/">More
Cookie Links</a>
<P>
<a href="#wrong">Return to troubleshooting checklist</a>.
<p><center><hr width="80%" size=1 noshade><table border=0 cellpadding=0
cellspacing=0><tr><td align=center valign=bottom width="100%"><font
size=-2 face=arial>Copyright © 2001 <a href="http://www.yahoo.com"
target=_top>Yahoo! Inc.</a> All rights reserved.<br>
<a href="http://privacy.yahoo.com/" target=_top>Privacy Policy</a> - <a
href="http://docs.yahoo.com/info/terms/" target=_top>Terms of
Service</a> - <a
href="http://docs.yahoo.com/info/guidelines/community.html"
target=_top>Guidelines</a> - <a href="http://help.yahoo.com/help/clubs/"
target="_top">Help</a></font></td></tr></table></center></body>
</html>
LWP::Protocol::collect: read 1286 bytes
LWP::Protocol::http::__ANON__: Collecting
LWP::Socket::read: (...)
LWP::IO::read: Read 0 bytes
HTTP::Cookies::extract_cookies: Set cookie B => b8jr16otc5hn5&b=2
LWP::UserAgent::request: Simple result: OK
<html>
<head>
<title>Cookie Error</title>
</head>
<body>
<img height=43 width=392
src="http://us.i1.yimg.com/us.yimg.com/i/yahooidwelcome.gif"
alt="Welcome to Yahoo!">
<h2>An Error Occurred Setting Your User Cookie</h2>
Many Yahoo! services use cookies to remember who you are logged in as.
If the cookie cannot be set correctly, then Yahoo! cannot determine if
you
are logged in.
<p>
If you wish to know more about cookies, and how we use them click <b><a
href="#descrip">here</a></b>.
<a name="wrong">
<h2>So What Went Wrong?</h2>
There are many reasons why a cookie could not be set correctly, here
are the most common reasons:
<ul>
<li>Your browser does not support cookies. For those who are technically
inclined and particularly interested,
the full and gory details as to which particular browsers support
cookies can be found
<a
href="http://www.research.digital.com/nsl/formtest/stats-by-test/NetscapeCookie.html">here</a>.
<p>
<li>You have cookie notification switched on in your browser and you did
not accept the accept the cookie set request. To remedy this, hit your
browser's <b>back</b>
button and try again, except this time accepting the cookie.
<p>
<li>You have installed an application that monitors/blocks cookies
being set. To remedy this, please disable the application while logging
in.
</ul>
<hr>
<a name="descrip">
<h2>What is a Cookie?</h2>
A cookie is a small amount of data that is sent to your web browser from
a
web server. This data is never "executed" as code, so it can't contain
programs or viruses, and can't be longer than 4,000 characters. Cookies
are
generally used to help web sites determine the state of your "client."
<p>
Let's say, for instance, that you go to a shopping web site and purchase
something. Instead of going off to pay for that item right away, you
want
to continue browsing. A cookie can be "set" to indicate that you put
something in your "shopping cart", so that when you go to pay for all
your
items, the server has a way of knowing all the items you selected by
looking
at your cookie.
<p>
Yahoo! uses a cookie simply to record your Yahoo! ID. We do
this so that wherever you go in personalized Yahoo! properties,
we know exactly what to display
for you: your stock quotes, your sports, weather, news, sites, your
personal e-mail, chat rooms, etc.
<h2>Why does Yahoo! use Cookies</h2>
First,
we should make it clear that we only use cookies to help identity you to
our
system. When you first register with Yahoo!, the feedback that you
provide us helps us figure out what kind of news and information we
should
send you. We "set a cookie" on your browser with your UserID so that
when
you visit any Yahoo! properties, we know it's you and not your neighbor.
<P>
We also use
cookies so that you don't have to bother logging in everytime when you
visit
Yahoo!. We should also make clear that we do _not_ distribute your
cookie
information to other organizations. No other sites outside yahoo.com
can
access this cookie.
<H2>What about Security?</h2>
A cookie cannot but used to extract data from your system. When a
cookie is
set, your browser checks it for length, an expiration date, and the
domain
from which it came. This domain checking means that only Yahoo can
access
your My Yahoo cookie. We do _not_ distribute your cookie or account
information to other companies and do _not_ include your password in
your cookie.
<h2>Can other people see my Yahoo preferences?</h2>
Our Yahoo! ID/password scheme is used primarily to identify you to our
system.
It provides some measure of security, but it is certainly not foolproof.
If you select <a href="/help/rememberme.html">Remember my ID &
Password</a>
anyone with access to your computer can use your Yahoo account, while
you are "logged in".
You can "log out" at any time by clicking "Change User" or "Sign Out".
If you do not select <a href="/help/rememberme.html">Remember my ID
& Password</a>
the browser you are using will forget your Yahoo! ID and cookie when you
exit it.
<p>
We hope this information clarifies the role of cookies on Yahoo!. If
you're curious, you can check out our main site directory to find other
sources of information on cookies:
<p>
<li> <a
href="http://dir.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/HTTP/Information_and_Documentation/Persistent_Cookies/">More
Cookie Links</a>
<P>
<a href="#wrong">Return to troubleshooting checklist</a>.
<p><center><hr width="80%" size=1 noshade><table border=0 cellpadding=0
cellspacing=0><tr><td align=center valign=bottom width="100%"><font
size=-2 face=arial>Copyright © 2001 <a href="http://www.yahoo.com"
target=_top>Yahoo! Inc.</a> All rights reserved.<br>
<a href="http://privacy.yahoo.com/" target=_top>Privacy Policy</a> - <a
href="http://docs.yahoo.com/info/terms/" target=_top>Terms of
Service</a> - <a
href="http://docs.yahoo.com/info/guidelines/community.html"
target=_top>Guidelines</a> - <a href="http://help.yahoo.com/help/clubs/"
target="_top">Help</a></font></td></tr></table></center></body>
</html>
--- HTTP::Response=HASH(0x76bde6c) ---
RC: 200 (OK)
Message: OK
Cache-Control: private
Connection: close
Date: Thu, 29 Mar 2001 05:23:49 GMT
Pragma: no-cache
Content-Type: text/html
Expires: Thu, 05 Jan 1995 22:00:00 GMT
Client-Date: Thu, 29 Mar 2001 05:36:58 GMT
Refresh: 900; URL=http://www.yahoo.com/
Set-Cookie: B=b8jr16otc5hn5&b=2; expires=Thu, 15 Apr 2010 20:00:00 GMT;
path=/; domain=.yahoo.com
Title: Cookie Error
<html>
<head>
<title>Cookie Error</title>
</head>
<body>
<img height=43 width=392
src="http://us.i1.yimg.com/us.yimg.com/i/yahooidwelcome.gif"
alt="Welcome to Yahoo!">
<h2>An Error Occurred Setting Your User Cookie</h2>
Many Yahoo! services use cookies to remember who you are logged in as.
If the cookie cannot be set correctly, then Yahoo! cannot determine if
you
are logged in.
<p>
If you wish to know more about cookies, and how we use them click <b><a
href="#descrip">here</a></b>.
<a name="wrong">
<h2>So What Went Wrong?</h2>
There are many reasons why a cookie could not be set correctly, here
are the most common reasons:
<ul>
<li>Your browser does not support cookies. For those who are technically
inclined and particularly interested,
the full and gory details as to which particular browsers support
cookies can be found
<a
href="http://www.research.digital.com/nsl/formtest/stats-by-test/NetscapeCookie.html">here</a>.
<p>
<li>You have cookie notification switched on in your browser and you did
not accept the accept the cookie set request. To remedy this, hit your
browser's <b>back</b>
button and try again, except this time accepting the cookie.
<p>
<li>You have installed an application that monitors/blocks cookies
being set. To remedy this, please disable the application while logging
in.
</ul>
<hr>
<a name="descrip">
<h2>What is a Cookie?</h2>
A cookie is a small amount of data that is sent to your web browser from
a
web server. This data is never "executed" as code, so it can't contain
programs or viruses, and can't be longer than 4,000 characters. Cookies
are
generally used to help web sites determine the state of your "client."
<p>
Let's say, for instance, that you go to a shopping web site and purchase
something. Instead of going off to pay for that item right away, you
want
to continue browsing. A cookie can be "set" to indicate that you put
something in your "shopping cart", so that when you go to pay for all
your
items, the server has a way of knowing all the items you selected by
looking
at your cookie.
<p>
Yahoo! uses a cookie simply to record your Yahoo! ID. We do
this so that wherever you go in personalized Yahoo! properties,
we know exactly what to display
for you: your stock quotes, your sports, weather, news, sites, your
personal e-mail, chat rooms, etc.
<h2>Why does Yahoo! use Cookies</h2>
First,
we should make it clear that we only use cookies to help identity you to
our
system. When you first register with Yahoo!, the feedback that you
provide us helps us figure out what kind of news and information we
should
send you. We "set a cookie" on your browser with your UserID so that
when
you visit any Yahoo! properties, we know it's you and not your neighbor.
<P>
We also use
cookies so that you don't have to bother logging in everytime when you
visit
Yahoo!. We should also make clear that we do _not_ distribute your
cookie
information to other organizations. No other sites outside yahoo.com
can
access this cookie.
<H2>What about Security?</h2>
A cookie cannot but used to extract data from your system. When a
cookie is
set, your browser checks it for length, an expiration date, and the
domain
from which it came. This domain checking means that only Yahoo can
access
your My Yahoo cookie. We do _not_ distribute your cookie or account
information to other companies and do _not_ include your password in
your cookie.
<h2>Can other people see my Yahoo preferences?</h2>
Our Yahoo! ID/password scheme is used primarily to identify you to our
system.
It provides some measure of security, but it is certainly not foolproof.
If you select <a href="/help/rememberme.html">Remember my ID &
Password</a>
anyone with access to your computer can use your Yahoo account, while
you are "logged in".
You can "log out" at any time by clicking "Change User" or "Sign Out".
If you do not select <a href="/help/rememberme.html">Remember my ID
& Password</a>
the browser you are using will forget your Yahoo! ID and cookie when you
exit it.
<p>
We hope this information clarifies the role of cookies on Yahoo!. If
you're curious, you can check out our main site directory to find other
sources of information on cookies:
<p>
<li> <a
href="http://dir.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/HTTP/Information_and_Documentation/Persistent_Cookies/">More
Cookie Links</a>
<P>
<a href="#wrong">Return to troubleshooting checklist</a>.
<p><center><hr width="80%" size=1 noshade><table border=0 cellpadding=0
cellspacing=0><tr><td align=center valign=bottom width="100%"><font
size=-2 face=arial>Copyright © 2001 <a href="http://www.yahoo.com"
target=_top>Yahoo! Inc.</a> All rights reserved.<br>
<a href="http://privacy.yahoo.com/" target=_top>Privacy Policy</a> - <a
href="http://docs.yahoo.com/info/terms/" target=_top>Terms of
Service</a> - <a
href="http://docs.yahoo.com/info/guidelines/community.html"
target=_top>Guidelines</a> - <a href="http://help.yahoo.com/help/clubs/"
target="_top">Help</a></font></td></tr></table></center></body>
</html>
-----------------------------------
