According to [EMAIL PROTECTED]: > At 10:51 AM -0600 2/21/02, Evaldas Imbrasas wrote: > >... > >Also, what about <noindex, follow> tag? Is it in 3.1.6? I couldn't find > >any info about that, either. > > I'm curious whether there is a case where people want the search > engine to not index text and not follow links in that text block as > well? Why should you need a <noindex follow> tag? Why does the > indexer ignore links in the <noindex> block by default? > > Just wondering,
Good question, Avi. However, the way htdig handled <noindex> since at least 3.0.8b2, if not earlier, was to turn off both indexing and following of links. Indeed, it wasn't until 3.1.0b1, when Geoff added support for meta robots tags, that the HTML parser made any distinction between turning off indexing and turning off following of links. So, rather than change the default behaviour of <noindex> many versions later in 3.1.6, I opted to make it a backwards compatible extension to an already non-standard tag. Do you know what tags other search engines support for turning off indexing and/or following links in selected sections of documents? -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
