What I ended up doing in this case, IIRC, is to use another phase to convert inputs 1 and 2 into some contrived new single Writable format. Then both sets of input are merely fed into one mapper. So I'd literally have Writable classes that contained, inside, either a FooWritable or BarWritable. A little ugly but not bad.
On Mon, Aug 2, 2010 at 3:24 PM, Shannon Quinn <[email protected]> wrote: > CompositeInputFormat implements a hadoop.mapred.join interface, whereas > job.setInputFormatClass() is expecting a class that extends a > hadoop.ioclass. Also, TupleWritable is in the deprecated hadoop.mapred > package, too. > > Still hunting around the API for the newer equivalent; there has to be a way > of doing this? > > On Mon, Aug 2, 2010 at 6:20 PM, Jake Mannix <[email protected]> wrote: > >> On Mon, Aug 2, 2010 at 3:13 PM, Shannon Quinn <[email protected]> wrote: >> > >> > Excellent. Any idea what the Hadoop 0.20.2 equivalent for >> > CompositeInputFormat is? :) >> > >> >> Ah, there is that part. Hmm... it's really really annoying to not have >> that >> in 0.20.2. >> >> This is actually why I haven't migrated the distributed matrix stuff to the >> newest >> Hadoop API - map-side join is pretty seriously useful sometimes. >> >> Does the old CompositeInputFormat work with the new API, does anyone know? >> >> -jake >> >
