[
https://issues.apache.org/jira/browse/NUTCH-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605716#comment-14605716
]
Sebastian Nagel commented on NUTCH-2038:
----------------------------------------
(#1) The unit tests fail if build and tests are run from a clean source tree:
* 2 files from pull request #39 are missing in the SVN commit. Ideally, they
should be added as
{noformat}
conf/naivebayes-train.txt.template
conf/naivebayes-wordlist.txt.template
{noformat}
As templates they get only instantiated/copied once, and are not overwritten if
a user changes the *.txt in-place.
* anyway, it would be nice to catch all IO errors related to an incomplete
configuration and show a nice error message, e.g., "Failed to load
naivebayes-train.txt configured in parsefilter.naivebayes.trainfile: ..."
(#2) the commons-cli is properly "installed" in
{{build/plugins/parsefilter-naivebayes/}} resp. in the corresponding runtime
folder or job file: It should be enough to add all indirect dependencies to the
plugin.xml
> Naive Bayes classifier based html Parse filter (for filtering outlinks)
> -----------------------------------------------------------------------
>
> Key: NUTCH-2038
> URL: https://issues.apache.org/jira/browse/NUTCH-2038
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher, injector, parser
> Reporter: Asitang Mishra
> Assignee: Chris A. Mattmann
> Labels: memex, nutch
> Fix For: 1.11
>
>
> A html parse filter that will filter out the outlinks in two stages.
> Classify the parse text and decide if the parent page is relevant. If
> relevant then don't filter the outlinks. If irrelevant then go thru each
> outlink and see if the url contains any of the important words from a list.
> If it does then let it pass.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
