Sounds like hbase's HFileOutputFormat depends on KeyValue's "family" field. I don't want that.
All I want is to keep keys and values in an indexed filed. TFile would work as well. But it seems there is no TFileOutputFormat available. On Fri, Dec 6, 2013 at 4:47 PM, Igor Gatis <[email protected]> wrote: > That's the kind of solution I'm looking for. > > Here is what I have: > > String jobName = "Seq2HFile"; > Job job = new Job(getConf(), jobName); > job.setJarByClass(Seq2HFile.class); > > job.setMapperClass(*MyIdentityMapper.class*); > job.setMapOutputKeyClass(BytesWritable.class); > job.setMapOutputValueClass(BytesWritable.class); > > job.setPartitionerClass(TotalOrderPartitioner.class); > > job.setReducerClass(KeyValueSortReducer.class); > job.setOutputKeyClass(ImmutableBytesWritable.class); > job.setOutputValueClass(KeyValue.class); > job.setNumReduceTasks(1); > > job.setInputFormatClass(SequenceFileInputFormat.class); > SequenceFileInputFormat.addInputPaths(job, inputPath); > > job.setOutputFormatClass(HFileOutputFormat.class); > HFileOutputFormat.setOutputPath(job, new Path(outputPath)); > > job.submit(); > job.waitForCompletion(true); > > The bit I'm stuck is MyIdentityMapper. My input is a > SequenceFile<BytesWritable, BytesWritable>. According to HFileOutputFormat > signature, output key is ImmutableBytesWritable and value is KeyValue. > > I guess BytesWritable -> ImmutableBytesWritable is straightforward. But > I've got no clue how to fill KeyValue. > > public static class MyIdentityMapper > extends Mapper<BytesWritable, BytesWritable, ImmutableBytesWritable, > KeyValue> { > public void map(BytesWritable key, BytesWritable value, Context > context) throws IOException, > InterruptedException { > * // What do I write here?* > } > } > > > > On Fri, Dec 6, 2013 at 12:31 PM, Jean-Marc Spaggiari < > [email protected]> wrote: > >> Hi Igor, >> >> I will say, MapReduce. >> >> SequenceFileInputFormat >> HFileOutputFormat >> >> JM >> >> >> 2013/12/5 Igor Gatis <[email protected]> >> >> > I have SequenceFiles I'd like to convert to HFile. How do I that? >> > >> > >
