Hi, Roy:
Thank you for your quick reply. I downloaded and installed your lastest
version of Momspider and libwww-perl, and also made changes to wwwhttp.pl
as you suggested. Now the problems I reported yesterday were gone.
However, there are still other problems. Please try the following URLs:
http://courageous.library.cmu.edu:8088/www/busi.html
There are two "Internal Errors" in the "Broken Links" Momspider reported,
but these sites are actually reachable.
http://courageous.library.cmu.edu:8088/www/physics.html
There are two "404 Not Found" (http://www.uspto.gov/ and
http://www.lbl.gov/LBL-Documents/CJOs/), three "Internal Errors" and one
"Forbidden" in the "Broken Links" Mompsider reported. Again, these links
are actually working.
Any ideas about what's going on here?
Thanks,
Jianqun (Jane) Wang
On Wed, 5 Apr 2000, Roy T. Fielding wrote:
> >Anybody used Momspider before? I found some problems when I tried it.
> >Some of the "broken" links Momspider reports are not borken at all.
> >The most strange thing is that if I used "get" under libwww-perl-0.40 (the
> >Perl WWW library Momspider requires) to try the "broken" page Mompsider
> >incorrectly reported, it gave "404 Not Found" error as well. Try this
> >page:
> >
> >http://www.ctheory.com/r45.html
>
> As mentioned, that would be the lack of a Host header, since the
> last release of MOMspider was before Host existed and uses an ancient
> version of libwww-perl.
>
> You can grab the full (unreleased) distribution from one of
>
> http://www.ics.uci.edu/pub/fielding/mom_lwp_97.tar.gz
> ftp://ftp.ics.uci.edu/pub/fielding/mom_lwp_97.tar.gz
>
> and add the following patch:
> ========
> --- wwwhttp.pl Thu Jun 19 03:44:16 1997
> +++ wwwhttp.pl Tue Oct 6 12:56:18 1998
> @@ -83,6 +83,7 @@
> # but is required for HTTP server root.
>
> $reqstr = "$method $object HTTP/1.0\015\012";
> + $reqstr .= "Host: $host\015\012";
> foreach $hd (keys(%headers))
> {
> if ($val = $headers{$hd})
> ========
>
> ....Roy
>
