It's possible to write your own class to take better encapsulate writing of side-effect files, but as people have said, you can run into unanticipated issues if the number of files you try to write at once becomes high.
On Sat, Feb 6, 2010 at 3:47 AM, Udaya Lakshmi <[email protected]> wrote: > Hi Amareshwari, > > But this feature is not available in Hadoop 0.18.3. Is there any work > around for this version. > Thanks, > Udaya. > > On Fri, Feb 5, 2010 at 10:49 AM, Amareshwari Sri Ramadasu < > [email protected]> wrote: > > > See MultipleOutputs at > > > http://hadoop.apache.org/common/docs/r0.20.0/api/org/apache/hadoop/mapred/lib/MultipleOutputs.html > > > > -Amareshwari > > > > On 2/5/10 10:41 AM, "Udaya Lakshmi" <[email protected]> wrote: > > > > Hi, > > I was wondering if it is possible to write each key-value pair produced > by > > the reduce function to a different file. How could I open a new file in > the > > reduce function of the reducer? I know its possible in configure function > > but it will write all the output that reducer to that file. > > Thanks, > > Udaya. > > > > >
