If you still have this redirect looping trouble after picking up the proper
session ID, there may be another problem...
IIS programmers seem to have the habit of putting a lot of redirects in their
sites. Sometimes a form submit will redirect 3 or 4 times before it gets where
it is going. I am not aware of the advantages of this stragety vs. going
directly where you want to end up, but I see it all the time.
I think a hint may be in this line:
Client-Warning: Redirect loop detected
I think LWP may be keeping track of the URLs you've visited, and when you get
redirected back to the same URL again, LWP figures there's no point in going
where we've already been. Problem is, LWP may make this decision without
considering all the factors. Portions of the request, such as cookies or form
variables, may have changed since the prior request to the same URL. So even
though you visit the same URL the results could be different the second time.
Or even if every component of the request is the same, the server's internal
state information (so called "session" table etc.) may have changed. So
perhaps the "loop" is exactly what the site designer (whether well advised or
not) intended. In fact, every browser that visits this site may be going
through the same loop, but visitors are not aware of anything more than an
excessive delay. And the site may wonder why their hit counts are so high and
their servers so bogged down but yet not much really happens.
Anyway, if LWP has a way to shut off the "Redirect loop detected" feature you
may find things eventually work themselves out. But I'm not enough of an LWP
expert to tell you if or how this could be done. I know its been discussed
before. Anyone else want to chime in here?
Hope this helps.
From: kalita%pikespeak.uccs.edu@Internet on 2000-05-25 05:55 PM
To: libwww%perl.org@Internet
cc: kalita%pikespeak.uccs.edu@Internet (bcc: Marvin Simkin)
Subject: A question about submitting forms to an IIS/ASP sitee
Hi everyone!
I would love to hear from anyone regarding how to solve the problem
I am facing.
I am trying to make a call to a form at a site varsityBooks.com
using the libwww package.
The downloaded the source for the form that I want to submit is given below.
It's simple except it hs a "sessid" field with a huge hex value.
I kept the value as it is.
---------form source-----------
<TABLE width="592" cellpadding=0 cellspacing=0 border="0">
<TR>
<TD width="1" bgcolor="#003366"><IMG SRC="Images/dot_clear.gif"
WIDTH=1
HEIGHT=1 BORDER=0 ALT=" "></TD>
<TD align="center" width="270" bgcolor="#E5E5E5" valign="top">
<form name="search" method="get" action="
http://www.varsitybooks.com/Search.as
p?dqs=4">
<input type="hidden" name="sessid"
value="F132182C060850C518C735257A026C4A">
<input TYPE="HIDDEN" NAME="SearchBy" VALUE="advanced">
<table border="0" cellpadding="2" cellspacing="0">
<tr>
<td align=right><b>Title:</b></td>
<td><input TYPE="TEXT" NAME="Title" size="20" MAXLENGTH="100"
VALUE=""></td>
</tr>
<tr>
<td align=right><b>Author:</B></td>
<td><input TYPE="TEXT" NAME="Author" MAXLENGTH="100" VALUE=""
size="20"></td>
</tr>
<tr>
<td align=right><b>ISBN:</b></font></td>
<td><input TYPE="TEXT" NAME="ISBN" MAXLENGTH="100" VALUE=""
size="20"></td>
</tr>
<tr>
<td align=right><b>Keyword:</b></td>
<td><input TYPE="TEXT" NAME="Keyword" MAXLENGTH="100" VALUE=""
size="20">
</td>
</tr>
<tr>
<td align=right><b>Publisher:</b></td>
<td><input TYPE="TEXT" NAME="Publisher" MAXLENGTH="100"
VALUE="" size="20">
</td>
</tr>
<TR>
<TD> </TD>
<TD align="center"><input type="image"
src="images/button_searchforbooks.gif"
width=137 height=29 border=0 alt="Search for Books" name="goBook"></TD>
</TR>
</TABLE></form>
</table>
------------------form ends------------------------------
I wrote the following program to submit the form.
----------Perl code starts-------------------
#!/usr/bin/perl
use LWP::Debug qw(+);
#use LWP::Simple;
use URI::URL;
use strict;
use HTTP::Cookies;
use LWP::UserAgent;
use HTTP::Request::Common;
#Set up the cookie jar
my $cookie = new HTTP::Cookies (ignore_discard => 1);
#Creat the user agent
my $ua = new LWP::UserAgent;
#Attach the cookie jar to the user agent
$ua -> cookie_jar ($cookie);
#Signature for our User Agent
$ua->agent ("Mozilla/5.0");
my $url = new URI::URL
("http://www.varsitybooks.com/search.asp");
#$url -> query_form (searchBy => "title",
# search => "Applied Cryptography");
my $req = GET $url,
["searchBy" => "title",
"sessid" => "F132182C060850C518C735257A026C4A",
"dqs" => 4,
"search" => "Applied Cryptography"];
my $content = $ua -> request ($req) -> as_string ();
print $content;
----------------------Perl code ends--------------------
When I run the program, this the response I get.
It always complains saying the page has moved. I am running the program at
a machine at school. There is no proxy server here. It's a Linux machine using
Red Hat 6.1.
--------------------runtime debug trace begins---------------
LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: GET http://www.varsitybooks.com/search.asp
LWP::UserAgent::_need_proxy: Not proxied
HTTP::Cookies::add_cookie_header: Checking www.varsitybooks.com for cookies
HTTP::Cookies::add_cookie_header: Checking .varsitybooks.com for cookies
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET /search.asp HTTP/1.0
Host: www.varsitybooks.com
User-Agent: Mozilla/5.0
LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
Server: Microsoft-IIS/4.0
Date: Fri, 26 May 2000 00:33:00 GMT
Set-Cookie: SITESERVER=ID=e9551cfbf537b05bf85ab028b0c2480d; expires=Monday,
01-Jan-2035 00:00:00 GMT; path=/; domain=.varsitybooks.com
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Connection: close
Location: /search.asp?wfx2=lh2hj34l8s
LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
HTTP::Cookies::extract_cookies: Set cookie SITESERVER =>
ID=e9551cfbf537b05bf85ab028b0c2480d
LWP::UserAgent::request: Simple response: Found
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: GET
http://www.varsitybooks.com/search.asp?wfx2
=lh2hj34l8s
LWP::UserAgent::_need_proxy: Not proxied
HTTP::Cookies::add_cookie_header: Checking www.varsitybooks.com for cookies
HTTP::Cookies::add_cookie_header: Checking .varsitybooks.com for cookies
HTTP::Cookies::add_cookie_header: - checking cookie path=/
HTTP::Cookies::add_cookie_header: - checking cookie
SITESERVER=ID=e9551cfbf537b05bf85ab028b0c2480d
HTTP::Cookies::add_cookie_header: it's a match
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET /search.asp?wfx2=lh2hj34l8s HTTP/1.0
Host: www.varsitybooks.com
User-Agent: Mozilla/5.0
Cookie: SITESERVER=ID=e9551cfbf537b05bf85ab028b0c2480d
Cookie2: $Version=1
LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
Server: Microsoft-IIS/4.0
Date: Fri, 26 May 2000 00:33:01 GMT
Connection: close
Location: /search.asp
Set-Cookie:SITESERVER=GUID=e9551cfbf537b05bf85ab028b0c2480d;path=/;expires=Sat,
29-Oct-2030 23:12:40 GMT; domain=.varsitybooks.com
Expires: Thu, 01 Dec 1994 16:00:00 GMT
LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
HTTP::Cookies::extract_cookies: Set cookie SITESERVER =>
GUID=e9551cfbf537b05bf85ab028b0c2480d
LWP::UserAgent::request: Simple response: Found
HTTP/1.1 302 (Found) Moved Temporarily
Connection: close
Date: Fri, 26 May 2000 00:33:01 GMT
Location: /search.asp
Server: Microsoft-IIS/4.0
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Client-Date: Thu, 25 May 2000 18:35:48 GMT
Client-Peer: 206.132.46.66:80
Client-Warning: Redirect loop detected
Set-Cookie:
SITESERVER=GUID=e9551cfbf537b05bf85ab028b0c2480d;path=/;expires=Sat
, 29-Oct-2030 23:12:40 GMT; domain=.varsitybooks.com
--------------------runtime trace ends--------------------
Will someone be kind enough to tell me what the problem is?
Is it the sessid field? Can someone tell me such a field's value
is created? Is there a way to simulate it? Or, is there something
else wrong? I am searching for books using "title" words. When
I bring up the exact same form in the browser and fill it up and
click on the submit button, it goes to varsityBooks.com and gives
me the results I want. The result is a Web page with a list of books.
Any pointer you can give me will be very much appreciated.
I am new to using LIBWWW. I have used it with several Unix based sites
and it works. I am having problems with IIS/ASP based sites.
The problems are all similar.
Thanks a lot!
Jugal Kalita
Associate Professor
Computer Science
University of Colorado at Colorado Springs
