Re: LWP post cannot get result page. LWP post did not send correct form data.

[Rob Dixon]

On 14/06/2011 09:11, Gary Yang wrote:

I have been trying to learn LWP Post, by going through an examples on the web. Perl&  
LWP. I try to search AbeBooks for the book "Codex Seraphinianus" from the main page 
(AbeBooks). Below is the code that I am using. But this just sends me back the content of the 
search page not the result page. Anyone knows what the problems are? Can someone show me a 
real working code of LWP post?

Here is the code that I am using:

#!/usr/bin/perl -w

use strict;
use LWP;

my $browser = LWP::UserAgent->new;

my $response = $browser->post('http://www.abebooks.com',
    ["sts" =>  "t",
     "an" =>  "",
     "tn" =>  "Codex Seraphinianus",
     "kn" =>  "",
     "isbn" =>  "",
    ]
);

die "Error: ", $response->status_line, "\n" unless $response->is_success;

my $out_file = "result_seraph.html"; # where to save it
open(OUT, ">$out_file") || die "Can't write-open $out_file: $!";

binmode(OUT);
print OUT $response->content;
close(OUT);
print "Bytes saved: ", -s $out_file, " in $out_file\n";


  Below are the related html code from abebooks.com.


<form action="/servlet/SearchResults" method="post" name="homepageSearch">
<table cellpadding="3" cellspacing="0" width="100%" border="0">
             <tr>
               <td class="attributeHeader" align="right">Author:</td>
               <td width="74%" align="left">
               <input type="hidden" name="sts" value="t" />
               <input type="text" name="an" size="40" tabindex="10" title="Author Field: 
Enter the name of the author you are searching for here"></td>
               </tr>
             <tr>
               <td class="attributeHeader" align="right">Title:</td>
               <td align="left"><input type="text" name="tn" size="40" tabindex="11" 
title="Title Field: Enter the title of the book you are searching for here"></td>
             </tr>
             <tr>
               <td class="attributeHeader" align="right">Keyword:</td>
               <td align="left"><input type="text" name="kn" size="40" tabindex="12" 
title="Keyword Field: Enter keywords related to the book you are searching for here (eg. Horror, Spanish Language)"></td>
             </tr>
             <tr>
               <td class="attributeHeader" align="right"><span class="tip"><a href="/docs/what-is-an-isbn/" 
onclick="window.open(this.href,'ISBN','height=400,width=380,location=no,toolbar=no,scrollbars=yes,resizable=yes'); return false">what's 
this?</a></span>  ISBN:</td>
               <td align="left"><input type="text" name="isbn" size="40" tabindex="13" 
title="ISBN Field: Enter the ISBN of the book you are searching for here"></td>
             </tr>
                         <tr>
                             <td>&nbsp;</td>
               <td class="button" align="left">
                 <input type="image" src="/images/Shared/findBook-large.gif" alt="FIND BOOK" 
tabindex="14">
                 &nbsp;
                 <a href="/servlet/SearchEntry" tabindex="15" title="More Search 
Options">More Search Options</a>               </td>
                         </tr>
                 </table></form>

You must send the query to the URL in the 'action' attribute of the
form element, not simply to the site root directory.

Rob