The problem can be seen from a network sniffer tool:
GET /aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736 HTTP/1.1
Connection: close
Host: cgi.ebay.com:80
Accept: image/gif,image/x-xbitmap,image/jpeg,image/pjpeg,image/png,*/*
Accept-Charset: iso-8859-1,*,utf-8
Accept-Encoding: gzip
Accept-Language: en
User-Agent: Mozilla/4.78 [en] (X11; U; Linux 2.4.7-10 i686)
HTTP/1.0 200 OK
Server: Microsoft-IIS/4.0
Date: Sun, 02 Dec 2001 15:03:28 GMT
Set-Cookie:
nonsession=AQAAAAEAAAAIAAAAWAAAAMBCCjzAzzE8MDEwMDczMDU0MDh4NjVodHRwOi8vY2dpLmViYXkuY29tL2F3LWNnaS9lQmF5SVNBUEkuZGxsP1ZpZXdJdGVtJml0ZW09MTMwNDEzODczNk4*i;
path=/; domain=.ebay.com; Expires=Mon, 02-Dec-2002 15:03:28 GMT
HTTP/1.0 200 OK
Server: Microsoft-IIS/4.0
Date: Sun, 02 Dec 2001 15:03:28 GMT
....
Notice that the line
HTTP/1.0 200 OK
is sent back twice. This does not follow the http protocol.
LWP considers this as a bad header in sub read_header_lines of
Net/HTTP.pm and dies from it:
sub read_header_lines {
my $self = shift;
my @headers;
while (my $line = my_readline($self)) {
if ($line =~ /^(\S+)\s*:\s*(.*)/s) {
push(@headers, $1, $2);
}
elsif (@headers && $line =~ s/^\s+//) {
$headers[-1] .= " " . $line;
}
else {
die "Bad header: $line\n";
}
}
return @headers;
}
To fix this, juse add the line
next if ( $line =~ /^HTTP/ );
immediately before the die call line.
Hope this helps.
Richard
On Sat, Dec 01, 2001 at 11:12:32PM -0500, [EMAIL PROTECTED] wrote:
>
> I am having extreme problems trying to fetch eBay items
> using LWP. Try to fetch an auction item like:
>
> http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736
>
> Note: This item number expires 12/9/2001
>
> perl -MLWP::Simple -e 'getprint
> "http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736";'
> 500 Bad header: HTTP/1.0 200 OK
> <URL:http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item=1304138736>
>
> The weird thing is that it works fine with all versions of IE, Netscape,
> lynx, etc, but fails for LWP and WGET. I poke around and found a document
> that describes the bug for wget
> http://www.mail-archive.com/[email protected]/msg01562.html
>
> To summarize the report, it says that some of eBay's page (like the one
> mentioned above) dont output a "Content-type" header, it uses the
> "<META HTTP-EQUIV="Content-Type"" tag instead. Why? I dunno...
> Wayne Schlitt developed a patch for wget-1.7.1-pre1 to assume "Content-type:
> text/html"
> if no header is found. I am hoping that the author of LWP will follow suit.
>
> Like I say, most all browsers make this assumption and work fine. Apparently
> this is a fairly recent issue with these eBay pages. Does anyone know of a
> quickfix
> or patch for LWP to deal with this?
>
> Thanks,
>
> Chris Stuber (mapsurfer)
> [EMAIL PROTECTED]
