Re: [Robots] Robots.txt Evolution?

[Nick Arnett]

I wonder if the best way to see an evolved standard take hold would be 
to involve httpd developers, especially Apache, from the start.  That 
would seem particularly important if it's to be anything that needs to 
be updated automatically.  Their incorporation of anything new would 
probably lead to others following.

Somewhat idiosyncratically, I'd like to see evolution in terms of 
gathering discussion-type data, such as message boards, mailing list 
archives and blogs, though the latter certainly makes things somewhat 
easier.  Of course, for those kinds of things to be palatable, they 
probably need to include stroppng e-mail obfuscation, lest they become 
address harvesting tools.

Nick

--
Nick Arnett
Phone/fax: (408) 904-7198
[EMAIL PROTECTED]
_______________________________________________
Robots mailing list
[EMAIL PROTECTED]
http://www.mccmedia.com/mailman/listinfo/robots