The 'bot blocker' image server at blogspot is broken so it's impossible to reply to this blog!
-----Original Message----- From: Matt Kangas [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 14, 2006 10:38 AM To: nutch-dev@lucene.apache.org Subject: Re: IncrediBILL's Random Rants: How Much Nutch is TOO MUCH Nutch? Heh. Perhaps we should eliminate the default user-agent string? Then he'd have less of a target to aim at... :) On a more serious note, it seems reasonable to require a customized "bot" URL at least. But publishing an email contact is questionable these days. Neither Y! nor G do it, precisely because it will just get spammed. (Wait until a spam-bot crawls this blogspot page and starts hammering nutch-agent...) On Jun 14, 2006, at 1:03 PM, Doug Cutting wrote: > http://incredibill.blogspot.com/2006/06/how-much-nutch-is-too-much- > nutch.html -- Matt Kangas / [EMAIL PROTECTED]
