Hello Jonathan, On Monday, December 1, 2003 you wrote:
JA> On Monday, December 01, 2003, Peter Hillier-Brook wrote... >> Would you care to explain why messages in a subscription list are >> published to the known universe via the Web? JA> The archives are publicly available via a website... All it takes is a JA> 'spider' or web crawler to hit the list archive page, and it'll index JA> everything on it. I don't believe the list moderators manage the JA> archives, it's done by another bunch of people. There is little the JA> moderators can do about it, and you're likely to find, even the JA> changes they could make are often ignored by most indexing spiders. I have no problem with the archives being available via the Web, however I don't think a "robots.txt" file would be too difficult to implement (I could offer some help if necessary). There is no obvious reason why spiders should be encouraged to crawl the archive site! -- Regards, Peter HB ________________________________________________________ http://www.silverstones.com/thebat/TBUDLInfo.html
