You need to implement a custom Writable (the serialization interface supported by Hadoop). If you want to use your own custom types as keys, they must implement WritableComparable. You could implement a "box" custom Writable to hold HashMap or any other type, but you'd have to find a way of encoding the type you want to send over the wire (probably as a byte array). You could use Java serialization to turn the HashMap into a byte buffer and then box that up in a BytesWritable (which holds byte[]). It will probably be slow, though.
On Thu, Apr 15, 2010 at 3:04 PM, M B <[email protected]> wrote: > How can I have my mapper output a HashMap as the value in the > OutputCollector (so my reducer can work directly on the HashMap key/value > pairs)? I tried just setting things up as HashMap in > conf.setOutputValueClass(HashMap.class), but that didn't work. What do I > need to change to allow another type (hashmap or arraylist) to be the output > of the map method? > > public static class Map extends MapReduceBase implements > Mapper<LongWritable, Text, Text, HashMap> { > ... > public void map(LongWritable key, Text value, OutputCollector<Text, > HashMap> output, Reporter reporter) throws IOException { > ... > output.collect(new Text("hello"), myHash); > ... > -- Eric Sammer phone: +1-917-287-2675 twitter: esammer data: www.cloudera.com
