Your choice is as Edward says, write a wrapper class to hold all of the objects you wish to write, or to write multiple files by manually opening additional output files and writing the particular objects to their own file.
On Thu, Dec 10, 2009 at 11:31 AM, Edward Capriolo <[email protected]>wrote: > On Thu, Dec 10, 2009 at 4:07 AM, Mehul Sutariya > <[email protected]> wrote: > > Hello everyone, > > > > I am trying to write different output types from mapper. For example, say > I > > have 3 classes A,B and C such that: > > > > - C extends A and > > - B extends A > > > > and my mapper is: > > Mapper<K1, V1, K2, A> > > Reducer<K2, A, K3, V3> > > > > Now, in my map function, when I try to write a record, where the value is > an > > instance of B or C, Hadoop framework throws an exception because the > > framework does some sort of interface checking. I understand the problem > > with doing this is that there would be no way for the reducer to know > what > > specialized type of instance it is, when it is reading the records that > map > > stage wrote using the readFields() method and hence I get a Type mismatch > > error. > > > > Has anyone felt the need of doing that, or is there a workaround for such > > type of operations? I would like to know possible alternatives if any. > > > > Right now, as a workaround, I am writing converting my records and > writing > > it as text as the output of map phase and then parsing the record again > in > > reduce phase to generate the objects and then finally write the > appropriate > > objects. > > > > Thanks, > > Mehul. > > > > You may want to take a look at how Nutch works. Nutch wrapps different > objects with a NutchWritable this way the Object Class that gets > collected is always the same. > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
