Insurance Squared Inc. wrote:
The funny thing about that wiki page (and some others in that area) is that they apparently use the nofollow tags. Given the topic of that wiki, isn't that a bit odd? I personally dislike the nofollow tag and think it should be used only in extreme circumstances (i.e. here's a link to a site you absolutely don't want to visit). I believe in this case however it's simply being used so that sites that are listed don't get any pagerank/weight/whatever passed to them from an authority site. A really bizarre policy for a search related site IMO.
I think it's a default setting for the Wiki, which nobody bothered to change...
Swinging back on topic, does nutch obey the nofollow tags?
Yes. Please see HtmlParser and HTMLMetaTags classes for details. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
