Hi Alan, You can use MultipleOutputFormat. You can override the generateFileName...methods to get the functionality you want.
A partitioner controls how data moves from the mapper to the reducer, so if you take that approach, you will have to specify the number of reducers as the number of files you want, which is not the best option if some days have more data than the others. You also dont have control over the file name. See Tom White's Hadoop The Definitive Guide for an excellent example and usage. Thanks and Regards, Sonal www.meghsoft.com On Mon, May 10, 2010 at 5:38 PM, Some Body <[email protected]> wrote: > Hi, > > I'm trying to understand how to generate multiple outputs in my reducer > (using 0.20.2+228). > Do I need MultipleOutput or should I partition my output in the mapper? > > My reducer currently gets key/val input pairs like this which all end up in > my part_r_0000 file. > > hostA_VarX_2010-05-01_morning <FLOATVAL> > hostA_VarY_2010-05-01_morning <FLOATVAL> > hostA_VarX_2010-05-01_afternoon <FLOATVAL> > hostA_VarY_2010-05-01_afternoon <FLOATVAL> > ..... > hostB_VarX_2010-05-01_morning <FLOATVAL> > hostB_VarY_2010-05-01_morning <FLOATVAL> > hostB_VarX_2010-05-01_afternoon <FLOATVAL> > hostB_VarY_2010-05-01_afternoon <FLOATVAL> > ..... > hostA_VarX_2010-05-02_morning <FLOATVAL> > hostA_VarY_2010-05-02_morning <FLOATVAL> > hostA_VarX_2010-05-02_afternoon <FLOATVAL> > hostA_VarY_2010-05-02_afternoon <FLOATVAL> > ..... > hostB_VarX_2010-05-02_morning <FLOATVAL> > hostB_VarY_2010-05-02_morning <FLOATVAL> > hostB_VarX_2010-05-02_afternoon <FLOATVAL> > hostB_VarY_2010-05-02_afternoon <FLOATVAL> > ..... > > But instead of 1 output file I want one output file per day/group. e.g. > 2010-05-01_morning.txt > 2010-05-01_afternoon.txt > > Each <date>_<time>.txt file would contain all keys/vals for all hosts & > VarNames > > Thanks, > Alan
