Hi, Can I open an issue in Jira system about FetchSchedules getFields() method? This method is never used. But it is necessary. I extend AbstractFetchSchedule class for my specific schedule requirements. But I can not use required columns data in my class.(Ex: metadata column)
On Tue, Apr 9, 2013 at 8:48 AM, Canan GİRGİN <[email protected]> wrote: > Hi Lewis, > > My custom CustomDefaultFetchSchedule class "getFields" method is never > called. Have you got any idea? > > In extension points, filters getField method is calling before start > operation. > For Example: ParseFilters getField methods are called by ParserJob. > > CustomDefaultFetchSchedule.getfields(): > > @Override > public Set<WebPage.Field> getFields() { > > FIELDS.addAll(super.getFields()); > FIELDS.add(WebPage.Field.METADATA); > return FIELDS; > } > > When I add metadata field in GeneratorJob class , eveything is okey and > metadata field is not empty: > static { > FIELDS.add(WebPage.Field.FETCH_TIME); > FIELDS.add(WebPage.Field.SCORE); > FIELDS.add(WebPage.Field.STATUS); > FIELDS.add(WebPage.Field.METADATA); > } > > > Nutch 2.1 / HBASE > > > On Mon, Apr 8, 2013 at 9:26 AM, Canan GİRGİN <[email protected]>wrote: > >> Hi Lewis, >> >> Yes, I *added *language-identifier. In DB metada column I can see >> "language=en" >> >> >> >> On Mon, Apr 8, 2013 at 12:06 AM, Lewis John Mcgibbney < >> [email protected]> wrote: >> >>> Hi Canan, >>> >>> >>> On Sun, Apr 7, 2013 at 1:41 AM, <[email protected]>wrote: >>> >>>> Than I try to use Metada Field. But this field is always null: >>>> *ByteBuffer blang = page.getFromMetadata(new Utf8(Metadata.LANGUAGE)); >>>> >>>> >>>> Did you add the language-identifier plugin to Nutch plugin.includes >>> property in nutch-site.xml? >>> >> >> >
