Well so much for knee-jerk suspicions as to intent. No need to look for
conspiracy theories when default settings are more likely to be the
cause. That should probably a corollary to occam's razor or something :).
Andrzej Bialecki wrote:
Insurance Squared Inc. wrote:
The funny thing about that wiki page (and some others in that area)
is that they apparently use the nofollow tags. Given the topic of
that wiki, isn't that a bit odd? I personally dislike the nofollow
tag and think it should be used only in extreme circumstances (i.e.
here's a link to a site you absolutely don't want to visit). I
believe in this case however it's simply being used so that sites
that are listed don't get any pagerank/weight/whatever passed to them
from an authority site. A really bizarre policy for a search related
site IMO.
I think it's a default setting for the Wiki, which nobody bothered to
change...
Swinging back on topic, does nutch obey the nofollow tags?
Yes. Please see HtmlParser and HTMLMetaTags classes for details.
