Richard,
Oh yes this fix works nicely. You the man!! Thank you very much.
I will mention however, that this function: read_header_lines()
has been moved to Net/HTTP/Methods.pm in libwww-perl 5.62
As to Erik's question.. I am using RedHat 7.2 (newly installed)
with libwww-perl 5.62.
This bug is most definitely related to eBay's system, and
they probably dont even know the problem exists because
all of the browsers continue to work without a hitch.
I suspect the eBay problem is a recent problem (and perhaps
it was intentional) because "eBayAgent" which is a publically
available (written in perl) auction sniper stopped working
as well. And somewhere I read a thread that the eBay programmers
where trying to defeat "wget" to stop folks from recursively
slurping their web space.
Although this eBay header doesn't strictly conform to HTTP specs,
I hope that this particular fix makes it into the next version of
libwww-perl.
Another problem, may or may not be related to this deformed header...
I tried to setup a cookie_jar to save this eBay nonsession cookie, but it
didn't work as described in: perldoc lwpcook. I get only 1 line:
#LWP-Cookies-1.0
in lwpcookies.txt and nothing else. Not a big deal, but I thought I
would mention it anyway.
Thanks again for your help,
Chris Stuber (mapsurfer)
Silicon Mapping Solutions, Inc.
EMail: [EMAIL PROTECTED]
Richard Chen wrote:
>
> The problem can be seen from a network sniffer tool:
>
> GET /aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736 HTTP/1.1
> Connection: close
> Host: cgi.ebay.com:80
> Accept: image/gif,image/x-xbitmap,image/jpeg,image/pjpeg,image/png,*/*
> Accept-Charset: iso-8859-1,*,utf-8
> Accept-Encoding: gzip
> Accept-Language: en
> User-Agent: Mozilla/4.78 [en] (X11; U; Linux 2.4.7-10 i686)
>
> HTTP/1.0 200 OK
> Server: Microsoft-IIS/4.0
> Date: Sun, 02 Dec 2001 15:03:28 GMT
> Set-Cookie:
>nonsession=AQAAAAEAAAAIAAAAWAAAAMBCCjzAzzE8MDEwMDczMDU0MDh4NjVodHRwOi8vY2dpLmViYXkuY29tL2F3LWNnaS9lQmF5SVNBUEkuZGxsP1ZpZXdJdGVtJml0ZW09MTMwNDEzODczNk4*i;
> path=/; domain=.ebay.com; Expires=Mon, 02-Dec-2002 15:03:28 GMT
> HTTP/1.0 200 OK
> Server: Microsoft-IIS/4.0
> Date: Sun, 02 Dec 2001 15:03:28 GMT
> ...
>
> Notice that the line
> HTTP/1.0 200 OK
> is sent back twice. This does not follow the http protocol.
> LWP considers this as a bad header in sub read_header_lines of
> Net/HTTP.pm and dies from it:
>
> sub read_header_lines {
> my $self = shift;
> my @headers;
> while (my $line = my_readline($self)) {
> if ($line =~ /^(\S+)\s*:\s*(.*)/s) {
> push(@headers, $1, $2);
> }
> elsif (@headers && $line =~ s/^\s+//) {
> $headers[-1] .= " " . $line;
> }
> else {
> die "Bad header: $line\n";
> }
> }
> return @headers;
> }
>
> To fix this, juse add the line
> next if ( $line =~ /^HTTP/ );
> immediately before the die call line.
>
> Hope this helps.
>
> Richard
>
> On Sat, Dec 01, 2001 at 11:12:32PM -0500, [EMAIL PROTECTED] wrote:
> >
> > I am having extreme problems trying to fetch eBay items
> > using LWP. Try to fetch an auction item like:
> >
> > http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736
> >
> > Note: This item number expires 12/9/2001
> >
> > perl -MLWP::Simple -e 'getprint
> > "http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736";'
> > 500 Bad header: HTTP/1.0 200 OK
> > <URL:http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736>
> >
> > The weird thing is that it works fine with all versions of IE, Netscape,
> > lynx, etc, but fails for LWP and WGET. I poke around and found a document
> > that describes the bug for wget
> > http://www.mail-archive.com/[email protected]/msg01562.html
> >
> > To summarize the report, it says that some of eBay's page (like the one
> > mentioned above) dont output a "Content-type" header, it uses the
> > "<META HTTP-EQUIV="Content-Type"" tag instead. Why? I dunno...
> > Wayne Schlitt developed a patch for wget-1.7.1-pre1 to assume "Content-type:
> > text/html"
> > if no header is found. I am hoping that the author of LWP will follow suit.
> >
> > Like I say, most all browsers make this assumption and work fine. Apparently
> > this is a fairly recent issue with these eBay pages. Does anyone know of a
> > quickfix
> > or patch for LWP to deal with this?
> >
> > Thanks,
> >
> > Chris Stuber (mapsurfer)
> > [EMAIL PROTECTED]
