Jim Twensky wrote:
Hello Jonathan,
Thanks for the fast response. Yes, my question is on other methods to put
the same data layout into HBase from my map reduce jobs. I've seen the
TableOutputFormat but I couldn't find any example usages of it.
Try src/example/mapred. See SampleUploader. It does like you want with
a mapper prepping data and then a reducer to insert into hbase table.
For how the reduce is configured, see the initTableReduceJob in
TableMapReduceUtil. It looks like this:
job.setOutputFormat(TableOutputFormat.class);
job.setReducerClass(reducer);
job.set(TableOutputFormat.OUTPUT_TABLE, table);
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(BatchUpdate.class);
if (partitioner != null) {
job.setPartitionerClass(HRegionPartitioner.class);
HTable outputTable = new HTable(new HBaseConfiguration(job), table);
int regions = outputTable.getRegionsInfo().size();
if (job.getNumReduceTasks() > regions){
job.setNumReduceTasks(outputTable.getRegionsInfo().size());
}
}
... sets the output format, reducer, output keys, table name, and tries
to set reduce count.
If any issue with the above, let us know.
St.Ack
