[
https://issues.apache.org/jira/browse/NUTCH-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris A. Mattmann resolved NUTCH-2038.
--------------------------------------
Resolution: Fixed
alright [~asitang] all committed! Fixed the ParserFactoryTest error. Thanks!
{noformat}
[chipotle:~/tmp/nutch-trunk] mattmann% svn commit -m "fix for NUTCH-2038: Naive
Bayes classifier based html Parse filter (for filtering outlinks) contributed
by Asitang Mishra <[email protected]> this closes #39"
Sending .gitignore
Sending build.xml
Sending conf/nutch-default.xml
Sending ivy/ivy.xml
Sending src/plugin/build.xml
Adding src/plugin/parsefilter-naivebayes
Adding src/plugin/parsefilter-naivebayes/build.xml
Adding src/plugin/parsefilter-naivebayes/ivy.xml
Adding src/plugin/parsefilter-naivebayes/plugin.xml
Adding src/plugin/parsefilter-naivebayes/src
Adding src/plugin/parsefilter-naivebayes/src/java
Adding src/plugin/parsefilter-naivebayes/src/java/org
Adding src/plugin/parsefilter-naivebayes/src/java/org/apache
Adding src/plugin/parsefilter-naivebayes/src/java/org/apache/nutch
Adding
src/plugin/parsefilter-naivebayes/src/java/org/apache/nutch/parsefilter
Adding
src/plugin/parsefilter-naivebayes/src/java/org/apache/nutch/parsefilter/naivebayes
Adding
src/plugin/parsefilter-naivebayes/src/java/org/apache/nutch/parsefilter/naivebayes/NaiveBayesClassifier.java
Adding
src/plugin/parsefilter-naivebayes/src/java/org/apache/nutch/parsefilter/naivebayes/NaiveBayesParseFilter.java
Adding
src/plugin/parsefilter-naivebayes/src/java/org/apache/nutch/parsefilter/naivebayes/package-info.java
Transmitting file data ...........
Committed revision 1688084.
[chipotle:~/tmp/nutch-trunk] mattmann%
{noformat}
Great work and thanks to you and Seb and others!
> Naive Bayes classifier based html Parse filter (for filtering outlinks)
> -----------------------------------------------------------------------
>
> Key: NUTCH-2038
> URL: https://issues.apache.org/jira/browse/NUTCH-2038
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher, injector, parser
> Reporter: Asitang Mishra
> Assignee: Chris A. Mattmann
> Labels: memex, nutch
> Fix For: 1.11
>
>
> A html parse filter that will filter out the outlinks in two stages.
> Classify the parse text and decide if the parent page is relevant. If
> relevant then don't filter the outlinks. If irrelevant then go thru each
> outlink and see if the url contains any of the important words from a list.
> If it does then let it pass.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
