A long time ago, I proposed a robots processing instruction that could be used in any XML format. I can find that again.
An in-document robots directive is useful because it is controlled by the document author, rather than by the webmaster. "nofollow" is not particularly useful, because there is almost always another path to the document. Still, it can be a polite hint to the robot that all the links in the doc are junk. I would use exactly the model in the HTML robots meta tag, because: a) that is what robots already know how to deal with, and b) it has proven good enough. wunder --On April 19, 2005 11:15:16 PM -0400 Nikolas 'Atrus' Coukouma <[EMAIL PROTECTED]> wrote: > > Hi, > I've recently ended up in argument about what to do with feeds that > don't want to be reproduced. I e-mailed Dave Winer in the hope of > getting some information about RSS end of things. That resulted in a > blog entry with interesting comments [1], and I now know that Creative > Commons has an RDF schema for describing licensing [2]. > > The only common feature I want to include, and haven't found, is the > "noindex" type of behavior (do not include in search engines). I > searched the archives of this list and found an old thread discussing > this very issue [3]. It seems to have fizzled out and I haven't found > anything more recent documents or discussions. > > Was the issue simply forgotten or purposfully dropped? > > In the RSS discussion, it was suggested by Roger Benningfield that > search eninges and syndication sites use atom:summary instead of > atom:content to avoid the noarchive issue. The rationale is that > summaries are meant to be reproduced, much like an abstract for a paper. > > I'm not sure about nofollow, I think noindex is definitely needed. The > latter could be used to opt-out of services such as Feedster, > Technorati, and PubSub. > > Thoughts and comments? > > [1] http://www.reallysimplesyndication.com/2005/04/19#a445 > [2] http://web.resource.org/cc/ > [3] http://www.imc.org/atom-syntax/mail-archive/msg00183.html > > Regards, > -Nikolas 'Atrus' Coukouma > > -- Walter Underwood Principal Architect, Verity
