[
https://issues.apache.org/jira/browse/NUTCH-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586117#comment-14586117
]
Sebastian Nagel commented on NUTCH-2039:
----------------------------------------
Thanks, [~sujenshah]! A few comments (I haven't tried to run the plugin):
* if SimilarityScoringFilter is inherited from AbstractScoringFilter it's not
necessary to implement all the "stub" methods (cf. NUTCH-1653)
* the gold-standard file needs to be parsed only once, right? This could be
done from SimilarityScoringFilter.setConf(conf) the resulting DocumentVector is
then cached.
* If Nutch is run in Hadoop's distributed mode loading the gold-standard and
stopwords file should be done either from hdfs or from class path as for config
files, e.g.
{code}
Reader reader = conf.getConfResourceAsReader(goldStandardFileName);
{code}
> Relevance based scoring filter
> ------------------------------
>
> Key: NUTCH-2039
> URL: https://issues.apache.org/jira/browse/NUTCH-2039
> Project: Nutch
> Issue Type: New Feature
> Reporter: Sujen Shah
> Labels: memex, nutch
> Fix For: 1.11
>
>
> A ScoringFilter plugin that uses a similarity measure to calculate the
> similarity between a given page(gold standard) and the currently parsed page.
> The score obtained from this similarity is then distributed to its outlinks.
> This filter aims to focus the crawler to crawl/explore relevant pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
