Hi, On 8/5/06, Chris Schneider <[EMAIL PROTECTED]> wrote: > Given this, shouldn't the default URL normalizer just add a slash to > the end of a URL that doesn't have a file extension?
Section 6.2.4 of RFC 3986 suggests that a crawler could do such a normalization if it detects that http://mail.python.org/mailman/listinfo redirects to http://mail.python.org/mailman/listinfo/. I think just blindly adding the slash without knowing about the redirection is incorrect. BR, Jukka Zitting -- Yukatan - http://yukatan.fi/ - [EMAIL PROTECTED] Software craftsmanship, JCR consulting, and Java development ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
