Hi,
I am mostly interested in fetcher job. In this job I see this code
StorageUtils.initMapperJob(currentJob, fields, IntWritable.class,
FetchEntry.class, FetcherMapper.class, FetchEntryPartitioner.class, false);
In storage utils this function has
DataStore<String, WebPage> store = createWebStore(job.getConfiguration(),
String.class, WebPage.class);
if (store==null) throw new RuntimeException("Could not create datastore");
Query<String, WebPage> query = store.newQuery();
query.setFields(toStringArray(fields));
GoraMapper.initMapperJob(job, query, store, outKeyClass, outValueClass,
mapperClass, partitionerClass, reuseObjects);
I followed all these functions but did not find actual code that sends query to
hbase table.
I believe it is somewhere in gora-hbase.
Thanks.
Alex.
-----Original Message-----
From: Renato Marroquín Mog
rovejo <[email protected]>
To: Gora Dev <[email protected]>
Sent: Tue, Feb 26, 2013 8:01 pm
Subject: Re: gora-hbase query
Hi Alex,
The Gora-HBase module is only in charge of querying and persisting
data from any where, not only Nutch. That being said, you want the
part where Nutch populates a map used in different Nutch jobs? Which
jobs are you talking about? Generator? Fetcher? You can probably get
some more lights over in NutchLand.
I am happy to go over the code with you anyways, just please be a
little bit more specific.
Renato M.
2013/2/26 <[email protected]>:
>
> Hello,
>
> Can someone point me the code in gora-hbase that queries hbase and populates
nutch map key values for varies nutch jobs?
> I plan to use SingleColumnValueFilter to see if it selects only subset of
records.
>
> Thanks.
> Alex.
>
>
