On Mon, Jan 12, 2009 at 5:35 AM, Nicolas Dumazet <[email protected]> wrote: > It seems to be a set of tools: crawler, parsers, indexers... To allow > a search. In short, an experimental search engine. Or a wannabe > commercial engine maybe, given that a .net domain is registered: > http://www.paxle.net/ > > ... > > This tool seem to have a blacklist: > "org.paxle.filter.blacklist.impl.BlacklistFilter". If you're able to > reach the author, you can probably ask him to blacklist your tools. > Question is _how_ : I haven't been able to find an email or any > information on this.
I don't think contacting the author should be necessary. If the bot is obeying robots.txt and other relevant directives, use those to block it. If it's not obeying robots exclusion standards, it should be blocked site-wide with an informative error message. _______________________________________________ Toolserver-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/toolserver-l
