On Tue, 16 Nov 1999, Frank Martini wrote: > > and potentially using spam-crawler poison on the > > website. > Could you expand on this? What is it... and how would HtDig handle it? Ah. So the idea of poison is that you have a CGI that generates an infinite tree of fake HTML pages with fake mailing addresses. Spambots fall-in, get lost, and end up with a large list of bogus addresses. As for how ht://Dig and other robots would handle this, you could easily post META robot headers and/or robots.txt restrictions keeping well-behaved robots out. I'm not sure how well it works--I've never tried it. But I'm looking for possible solutions. -Geoff Hutchison Williams Students Online http://wso.williams.edu/ ------------------------------------ To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] containing the single word unsubscribe in the SUBJECT of the message.
