That's the kind of solution I'm looking for.
Here is what I have:
String jobName = "Seq2HFile";
Job job = new Job(getConf(), jobName);
job.setJarByClass(Seq2HFile.class);
job.setMapperClass(*MyIdentityMapper.class*);
job.setMapOutputKeyClass(BytesWritable.class);
job.setMapOutputValueClass(BytesWritable.class);
job.setPartitionerClass(TotalOrderPartitioner.class);
job.setReducerClass(KeyValueSortReducer.class);
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(KeyValue.class);
job.setNumReduceTasks(1);
job.setInputFormatClass(SequenceFileInputFormat.class);
SequenceFileInputFormat.addInputPaths(job, inputPath);
job.setOutputFormatClass(HFileOutputFormat.class);
HFileOutputFormat.setOutputPath(job, new Path(outputPath));
job.submit();
job.waitForCompletion(true);
The bit I'm stuck is MyIdentityMapper. My input is a
SequenceFile<BytesWritable, BytesWritable>. According to HFileOutputFormat
signature, output key is ImmutableBytesWritable and value is KeyValue.
I guess BytesWritable -> ImmutableBytesWritable is straightforward. But
I've got no clue how to fill KeyValue.
public static class MyIdentityMapper
extends Mapper<BytesWritable, BytesWritable, ImmutableBytesWritable,
KeyValue> {
public void map(BytesWritable key, BytesWritable value, Context
context) throws IOException,
InterruptedException {
* // What do I write here?*
}
}
On Fri, Dec 6, 2013 at 12:31 PM, Jean-Marc Spaggiari <
[email protected]> wrote:
> Hi Igor,
>
> I will say, MapReduce.
>
> SequenceFileInputFormat
> HFileOutputFormat
>
> JM
>
>
> 2013/12/5 Igor Gatis <[email protected]>
>
> > I have SequenceFiles I'd like to convert to HFile. How do I that?
> >
>
