RE: :mechanize issues/mechanize.pm dies!!

[Nicholas S Zambo]

Bruce,

I don't think that this is a problem with mechanize or your code.  Looking 
at the html of the site in question I see this:

<select name="u_input" size="1">
                <option selected>-Choose a Department-</option>
                <option value="ADPIBA">Adult Degree Program</option>
                <option value="AMS">American Studies</option>
                <option value="ANT" 
Name="Anthropology">Anthropology</option>
                <option value="ARHART">Art &amp; Art History</option>
                <option value="BIO">Biological Sciences</option>
                <option value="BLS">Black Studies</option>
                <option value="CHE">Chemistry</option>
                ......

The option tag for anthro is malformed, in that there is no "name" 
attribute for options.  This is probably confusing the mechanize engine. 
If you take out this line:
$b->field("u_input", $curdept);

and put in 
$b->field("Anthropology", $curdept);

there are no more errors, but I seriously doubt that the code will do what 
you need (sorry I didn't test it).

An option is to get the page, scrub out the bad syntax and pass the 
resulting page to mechanize.  Then do what you have to do.. or call up the 
doofus writing the web page and ask them to fix it. :)

Hope that helps.

Nick






"bruce" <[EMAIL PROTECTED]>

06/02/2004 11:01 AM
Please respond to bedouglas
 
        To:     <[EMAIL PROTECTED]>, 
<[EMAIL PROTECTED]>
        cc: 
        Subject:        [BULK] - RE: :mechanize issues/mechanize.pm dies!!


hi..

i'm havin an issue with the following script. it uses www::mechanize and
appears to die when i use the word/var "ANT" for one of the form inputs. 
the
test script demonstrates getting information from the test site using both
LWP and mechanize. the LWP approach works, the mechanize fails.

any ideas/comments/criticisms as to why the mechanize fails would be
appreciated..

you can see the machanize works by using "CHE" instead of "ANT"...


###########################################################

#!/usr/bin/perl -w


use WWW::Mechanize;
use HTML::TreeBuilder;
use LWP::UserAgent;
   $ua = new LWP::UserAgent;
   $ua->timeout(30);
   $ua->agent("AgentName/0.1 " . $ua->agent);

   my $req = new HTTP::Request POST =>
'http://lca.lehman.cuny.edu/dept/registrar/schedule/coursefinder.asp';

   my $query;
   my $cstr;
   $cstr =
"?term=FAL04CRS&division=D&u_input=ANT&sortby=Disc%2C+Coursenum%2C+Section&o
rder=+ASC";

   $req->content_type('application/x-www-form-urlencoded');
   $req->content($cstr);

   my $res = $ua->request($req);
   my $q1c = $res->content;
   print $q1c;

#
# using the above LWP method works ok...
# implementing with www::mechanize causes issues
#
# it appears that using "ANT" for "u_input" causes the
# mechanize function to run into issues. other information
# for "u_input" works ok... but "ANT" causes the app to die...
#

   my (@_semester, @_dept, @_sort, @_order1, @_order, @_div);
   my ($default_semester, $default_div, $default_order);

$default_semester="FAL04CRS";
$default_div="D";

$curdept="ANT";  ## <<<<<<< major issue!!!!!!!

$default_sort="Disc Coursenum Section";
$default_order="+ASC";

my $b = WWW::Mechanize->new();


$b->get("
http://lca.lehman.cuny.edu/dept/registrar/schedule/coursefinder.asp
");
         $b->form_number(1);
            #print $b->current_form()->dump();
         $b->field("term", $default_semester);
         $b->field("division", $default_div);
         $b->field("u_input", $curdept);
         $b->field("sortby", $default_sort);
         $b->field("order", $default_order);
         #open(F, ">out.html");
         #print F $b->submit()->content();
          #close(F);

   #      &parse_file();
   print "uuu ".$default_semester."  ".$default_div."  ".$curdept."  ".
$default_sort."  ".$default_order."\n";

die;
###########################################################


thanks...

bruce




==========
The preceding e-mail message (including any attachments) contains 
information that may be confidential, be protected by the attorney-client 
or other applicable privileges, or constitute non-public information.  It 
is intended to be conveyed only to the designated recipient(s).  If you 
are not an intended recipient of this message, please notify the sender by 
replying to this message and then delete it from your system.  Use, 
dissemination, distribution, or reproduction of this message by unintended 
recipients is not authorized and may be unlawful.
==========