Hi All.
I am trying to generate Hfiles for bulk import. but i am getting the
following exception. The hbase table has only 1 column family
attempt_201109091959_0002_r_000000_0: log4j:WARN No appenders could be found
for logger (org.apache.hadoop.hdfs.DFSClient).
attempt_201109091959_0002_r_000000_0: log4j:WARN Please initialize the log4j
system properly.
11/09/09 20:29:09 INFO mapred.JobClient: map 100% reduce 0%
11/09/09 20:29:16 INFO mapred.JobClient: map 100% reduce 33%
11/09/09 20:29:18 INFO mapred.JobClient: Task Id :
attempt_201109091959_0002_r_000000_1, Status : FAILED
java.io.IOException: One family only
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat$1.write(HFileOutputFormat.java:102)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat$1.write(HFileOutputFormat.java:82)
at
org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:513)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.hadoop.hbase.mapreduce.KeyValueSortReducer.reduce(KeyValueSortReducer.java:46)
at
org.apache.hadoop.hbase.mapreduce.KeyValueSortReducer.reduce(KeyValueSortReducer.java:35)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Here is configuration
Configuration conf = HBaseConfiguration.create();
conf.set("fs.default.name",FS_DEFAULT_NAME);
conf.setBoolean("mapred.speculative.execution", false);
conf.set("hbase.zookeeper.quorum", HBASE_ZOOKEEPER_QUORUM);
conf.set("hbase.zookeeper.property.clientPort",
HBASE_ZOOKEEPER_PROPERTY_CLIENTPORT);
Job job = new Job(conf);
job.setJarByClass(BulkImport.class);
Path p1 = new Path("/user/root/URI/InputFiles/");
FileInputFormat.setInputPaths(job, p1);
job.setInputFormatClass(TextInputFormat.class);
job.setMapperClass(Map.class);
HTable htable = new HTable(conf, "mytab1");
job.setReducerClass(KeyValueSortReducer.class);
Path p2 = new Path("/user/root/URI/OutputFiles5/");
FileOutputFormat.setOutputPath(job, p2);
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(KeyValue.class);
HFileOutputFormat.configureIncrementalLoad(job, htable);
System.exit(job.waitForCompletion(true) ? 0 : 1);
Any Clue
Thanks
--
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>
