I think your first approach at serializing is correct, except for the use of ObjectWritable. From the docs, ObjectWritable only handles Strings, Arrays, and primitives. You are trying to use it to serialize your ArrayList. Try converting the ArrayList to an array of Strings first.
As for the second problem, I’d have a look at Cascading <http://www.cascading.org/> Hope these help… Chris From: Bhaskar Ghosh [mailto:[email protected]] Sent: Sunday, September 19, 2010 6:22 AM To: [email protected] Subject: How to create a composite value object for output from Map method Hi All, What would be the right approach to solve this problem: 1. I need to output an object as the value from my map method. The object's class should have two mambers: an ArrayList<String> and another, an integer. I used following two ways, but they are not working: * I wrote a class MyCompositeValueWritable that implements Writable interface. * Inside the overridden readFields and write methods, I try to read/write using the ObjectWritable class. * [see attached file MyWordCount_ObjVal1_2.java] * * The custom class is a plain class 'MyCompositeValue' not implementing or inheriting anything. * The Map and Reduce methods try to output the <key, value=<object of MyCompositeValue> > using the ObjectWritable class. * [see attached file Case2.txt] * * * Am I going wrong somewhere? Appreciate any help. 2. I have another problem, in which I need two types of mappers and reducer, and I want to execute them in this order: * Mapper1 -> Reducer1 -> Mapper2 -> Reducer2 * Is it possible through ChainMapper and/or ChainReducer classes? It yes, then how? Can anybody provide some starting working example, or point me to some good url for the same? * Currently, I am doing it as a work-around: * The first set of Mapper-Reducer write to HDFS. Then the second set of Mapper-Reducer pick up that output file from HDFS and writes further processed output to another HDFS directory. * An example would be really really helpful. Thanks Bhaskar Ghosh "Ignorance is Bliss... Knowledge never brings Peace!!!"
