System.out.println( " Running with on tables "+args[1]+ " and "+args[2]+"
with zk "+args[3]);
What was the output from the above ?
I would expect a call similar to the following in your run() method - this
comes from TestTableMapReduce.java:
TableMapReduceUtil.initTableReducerJob(
Bytes.toString(table.getTableName()),
IdentityTableReducer.class, job);
On Mon, Jan 6, 2014 at 7:12 PM, AnilKumar B <[email protected]> wrote:
> Hi,
>
> In my MR job, I need to write output into multiple tables, So I am
> using MultiTableOutputFormat as below. But I am getting
> TableNotFoundException.
>
> I am attaching code snippet below, Is this the correct way to use
> MultiTableOutputFormat ?
>
>
> Job class:
> public int run(String[] args) throws Exception {
> System.out.println( " Running with on tables "+args[1]+ " and "+args[2]+"
> with zk "+args[3]);
> Configuration hbaseConf = HBaseConfiguration.create(getConf());
> // hbaseConf.set(Constants.HBASE_ZOOKEEPER_QUORUM_PROP,
> Constants.HBASE_OS_CL1_QUORUM);
> hbaseConf.set(Constants.HBASE_ZOOKEEPER_QUORUM_PROP, args[3]);
> Job job = new Job(hbaseConf);
> job.setJarByClass(MultiTableTestJob.class);
> job.setInputFormatClass(TextInputFormat.class);
> job.setMapperClass(MultiTableTestMapper.class);
> job.setMapOutputKeyClass(Text.class);
> job.setMapOutputValueClass(Text.class);
> job.setReducerClass(MultiTableTestReducer.class);
> job.setOutputKeyClass(Text.class);
> job.setOutputValueClass(Text.class);
> FileInputFormat.setInputPaths(job, new Path(args[0]));
> job.setOutputFormatClass(MultiTableOutputFormat.class);
> TableMapReduceUtil.addDependencyJars(job);
> TableMapReduceUtil.addDependencyJars(job.getConfiguration());
> return job.waitForCompletion(true) == true ? 0 : -1;
> }
> public static void main(String[] args) throws Exception{
> Configuration configuration = new Configuration();
> configuration.set("HBASE_DEST_TABLE", args[1]);
> configuration.set("HBASE_LOOKUP_TABLE", args[2]);
> ToolRunner.run(configuration, new CISuperSessionJob(), args);
> }
>
> Reducer Class:
>
> private ImmutableBytesWritable tbl1;
> private ImmutableBytesWritable tbl2;
>
> protected void setup(Context context) throws IOException
> ,InterruptedException {
> Configuration c = context.getConfiguration();
> tbl1 = new
>
> ImmutableBytesWritable(Bytes.toBytes(context.getConfiguration().get("HBASE_DEST_TABLE")));
> tbl2 = new
>
> ImmutableBytesWritable(Bytes.toBytes(context.getConfiguration().get("HBASE_LOOKUP_TABLE")));
> };
>
> protected void reduce(Text key, java.lang.Iterable<Text>
> values, Context context) throws IOException ,InterruptedException {
> //
> if (some condition) {
> Put put = getSessionPut(key, vc);
> if (put != null) {
> context.write(tbl1, put);
> }
> } else {
> //
> Put put = getEventPut(key, vc);
> context.write(tbl2, put);
> }
> }
> }
>
>
> Exception:
> org.apache.hadoop.hbase.TableNotFoundException: mapred.reduce.tasks=100
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:999)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:864)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821)
> at
> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
> at
>
> org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat$MultiTableRecordWriter.getTable(MultiTableOutputFormat.java:101)
> at
>
> org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat$MultiTableRecordWriter.write(MultiTableOutputFormat.java:127)
> at
>
> org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat$MultiTableRecordWriter.write(MultiTableOutputFormat.java:68)
> at
>
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586)
> at
>
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
>
>
>
> Thanks & Regards,
> B Anil Kumar.
>
