Amogh, I don't have experience with patches for hadoop.
I take it that I apply this patch using the linux patch utility. I further assume, I need only apply the latest patch, which is 5. Am I correct. On Wed, Dec 9, 2009 at 7:30 AM, Amogh Vasekar <am...@yahoo-inc.com> wrote: > http://issues.apache.org/jira/browse/MAPREDUCE-370 > > You’ll have to work around for now / try to apply patch. > > Amogh > > > > On 12/9/09 8:54 PM, "Geoffry Roberts" <geoffry.robe...@gmail.com> wrote: > > Aaron, > > I am using 0.20.1 and I'm not finding org.apache.hadoop.mapreduce. > > lib.output.MultipleOutputs. I'm using the download page where the tar ball > is dated from Sep.09. > > > > Sounds like I need to look at the code repository. > > > > On Tue, Dec 8, 2009 at 1:39 PM, Aaron Kimball <aa...@cloudera.com> wrote: > > Geoffry, > > There are two MultipleOutputs implementations; one for the new API, one for > the old one. > > The new API (org.apache.hadoop.mapreduce.lib.output.MultipleOutputs) does > not have a getCollector() method. This is intended to work with > org.apache.hadoop.mapreduce.Mapper and its associated Context object. > > The old API implementation of MO > (org.apache.hadoop.mapred.lib.MultipleOutputs) is intended to work with > org.apache.hadoop.mapred.Mapper, Reporter, and friends. > > If you're going to use the new org.apache.hadoop.mapreduce-based code, you > should not need to import anything in the mapred package. That having been > said -- I just realized that the new-API-compatible MultipleOutputs > implementation is not in Hadoop 0.20. It's only in the unreleased 0.21. If > you're using 0.20, you should probably stick with the old API for your > process. > > Cheers, > - Aaron > > > On Tue, Dec 8, 2009 at 12:40 PM, Geoffry Roberts < > geoffry.robe...@gmail.com> wrote: > > All, > > This one has me stumped. > > What I want to do is output from my reducer multiple files, one for each > key value. I also want to avoid any deprecated parts of the API. > > As suggested, I switched from using MultipleTextOutputFormat to > MultipleOutputs but have run into an impasse. MultipleOutputs' getCollector > method requires a Reporter as a parameter, but as far as I can tell, the API > doesn't support this. The only reporter I can find is in the context > object, but is declared protected. > > Am I stuck? or just missing something? > > My code: > > @Override > public void reduce(Text key, Iterable<Text> values, Context context) > throws IOException { > String fileName = key.toString(); > MultipleOutputs.addNamedOutput((JobConf) > context.getConfiguration(), fileName, OutputFormat.class, Text.class, > Text.class); > mos = new MultipleOutputs((JobConf) > context.getConfiguration()); > for (Text line : values) { > > // This is the problem line: > mos.getCollector(fileName, <reporter goes here>).collect( > key, line); > } > > mos.close(); > > } > > On Mon, Oct 5, 2009 at 11:17 AM, Aaron Kimball <aa...@cloudera.com> wrote: > > Geoffry, > > The new API comes with a related OF, called MultipleOutputs > (o.a.h.mapreduce.lib.output.MultipleOutputs). You may want to look into > using this instead. > > - Aaron > > > On Tue, Sep 29, 2009 at 4:44 PM, Geoffry Roberts < > geoffry.robe...@gmail.com> wrote: > > All, > > What I want to do is output from my reducer multiple files one for each key > value. > > Can this still be done in the current API? > > It seems that using MultipleTextOutputFormat requires one to use deprecated > parts of API. > > It this correct? > > I would like to use the class or its equivalent and stay off anything > deprecated. > > Is there a work around? > > In the current API one uses Job and a class derived from the > classorg.apache.hadoop.mapreduce.OutputFormat. > MultipleTextOutputFormat does not derive from this class. > > Job.setOutputFormatClass(Class<? extends org.apache.hadoop.mapreduce. > OutputFormat>); > > > In the Old, deprecated API, one uses JobConf and an implementation of the > interface org.apache.hadoop.mapred.OutputFormat. MultipleTextOutputFormat > is just such an implementation. > > JobConf.setOutputFormat(Class<? extends org.apache.hadoop.mapred . > OutputFormat); > > > > > > >
