On Monday, December 01, 2003, Peter Hillier-Brook wrote... JA>> The archives are publicly available via a website...
> I have no problem with the archives being available via the Web, > however I don't think a "robots.txt" file would be too difficult to > implement (I could offer some help if necessary). There is no > obvious reason why spiders should be encouraged to crawl the archive > site! a) Most spiders ignore robots.txt now, so there isn't really any point. b) We (moderators, not me personally) don't manage the archive services. -- Jonathan Angliss ([EMAIL PROTECTED]) sophisticated, adj. deprived of its native simplicity.
________________________________________________________ http://www.silverstones.com/thebat/TBUDLInfo.html
