Re: Custom Plugin Resources Files

2017-06-29 Thread SJC Multimedia
ava/org/apache/nutch/parsefilter/ >> naivebayes/NaiveBayesParseFilter.java#L132-L137 >> >> On Thu, Jun 29, 2017 at 8:29 AM, <user-digest-h...@nutch.apache.org> >> wrote: >> >> > >> > >> > From: SJC Multimedia <sjcmultime...@gmail.com> >&

Re: Custom Plugin Resources Files

2017-06-29 Thread SJC Multimedia
> > From: SJC Multimedia <sjcmultime...@gmail.com> > > To: user@nutch.apache.org > > Cc: > > Bcc: > > Date: Thu, 29 Jun 2017 08:28:54 -0700 > > Subject: Custom Plugin Resources Files > > I am building a custom plugin in Nutch 2.3.1 on Hadoop/HBase. In

Re: Custom Plugin Resources Files

2017-06-29 Thread lewis john mcgibbney
t-h...@nutch.apache.org> wrote: > > > From: SJC Multimedia <sjcmultime...@gmail.com> > To: user@nutch.apache.org > Cc: > Bcc: > Date: Thu, 29 Jun 2017 08:28:54 -0700 > Subject: Custom Plugin Resources Files > I am building a custom plugin in Nutch 2.3.1 on Hadoop/HBas

Re: Custom Plugin Resources Files

2017-06-29 Thread Jorge Betancourt
Sure, no problem, Is not specifically for Nutch 2.x but on master you can take a look at the scoring-similarity [1] plugin, this is just a text file but it's used internally by the plugin. Usually, a lot of plugins define their own additional conf files this way, so it's not uncommon or very

Re: Custom Plugin Resources Files

2017-06-29 Thread SJC Multimedia
Okay makes sense. If you dont mind can you point me to a specific plugin that does something similar? On Thu, Jun 29, 2017 at 8:39 AM, Jorge Betancourt < betancourt.jo...@gmail.com> wrote: > Hi Dave, > > My advice would be to leave your resources out of the plugins, if there is > a

Re: Custom Plugin Resources Files

2017-06-29 Thread Jorge Betancourt
Hi Dave, My advice would be to leave your resources out of the plugins, if there is a configuration file (or additional files), just load what you need from the conf directory if the files dictionary can change just make it configurable on the nutch-site.xml. Best Regards, Jorge PS: You can

Re: Custom Plugin Resources Files

2017-06-29 Thread SJC Multimedia
Thing I have already tried is to bundle these resources in the job jar and load them from the classpath but that didn't work. I also tried copying them to HDFS and loading them from there but that too failed. What is the best way to bundle such static resources and reference them in the custom

Custom Plugin Resources Files

2017-06-29 Thread SJC Multimedia
I am building a custom plugin in Nutch 2.3.1 on Hadoop/HBase. In the plugin code, I need to pull in a dictionary of files and run some comparisons while parsing the document. Is there a way to include directory of files through the custom plugin ant build framework that will work on both local