Re: Hbase Bulk Load Java Sample Code

[Ioakim Perros]

On 08/27/2012 04:18 PM, o brbrs wrote:
Hi,
I'm new at hase and i want to make bulk load from hdfs to hbase with java.
Is there any sample code which includes importtsv and completebulkload
libraries on java?

Thanks.

Hi,

Here is a sample configuration of a bulk loading job consisting only of 
map tasks:

        Configuration config = HBaseConfiguration.create();

        config.set(TableOutputFormat.OUTPUT_TABLE, tableNAME);
        Path inputPath = new Path(inputStringPath);

        Job job = new Job(config, "Sample job" );

        job.setMapOutputKeyClass(mapperKey);
        job.setMapOutputValueClass(mapperValue);

        FileInputFormat.setInputPaths(job, inputPath);
        job.setInputFormatClass(inputFormat);
        FileOutputFormat.setOutputPath(job,new Path (HFileoutputPath)); 
//directory at HDFS where HFiles will be placed
        //before bulk loading

        job.setOutputFormatClass(HFileOutputFormat.class);

        job.setJarByClass(caller);
        job.setMapperClass(mapper);

        HTable hTable = new HTable(config, tableNAME); //tableNAME is a 
String representing a table which has to
        //already exist in HBase
        HFileOutputFormat.configureIncrementalLoad(job, hTable); 
//check respective API for the complete functionality of this         
//function

        job.waitForCompletion(true);

        /* after the job's completion, we have to write the HFiles
         * into HBase's specified table */
        LoadIncrementalHFiles lihf = new LoadIncrementalHFiles(config);
        lihf.doBulkLoad(new Path(HFileoutputPath), hTable);

Create a map task which produces key,value pairs just as you expect them 
to exist in your HBase's table (e.g.: key: ImmutableBytesWritable, Put) 
and you re done.

Regards,
Ioakim