Reducers run independently and without knowledge of one another, so you can't get one reducer to depend on the output of another. I think having two jobs is the simplest way to achieve what you're trying to do.
Tom On Thu, Jan 22, 2009 at 3:48 PM, Brian MacKay <[email protected]> wrote: > Hello Tom, > > Would like to apply some rules To CAT_A, then use the output of CAT_A to > reduce CAT_B. I'd rather not run two JOBS, so perhaps I need two > reducers? > > > First Reducer processes CAT_A, then when complete second reducer does > CAT_B? > > I suppose this would accomplish the same thing? > > > > -----Original Message----- > From: Tom White [mailto:[email protected]] > Sent: Thursday, January 22, 2009 10:41 AM > To: [email protected] > Subject: Re: Set the Order of the Keys in Reduce > > Hi Brian, > > The CAT_A and CAT_B keys will be processed by different reducer > instances, so they run independently and may run in any order. What's > the output that you're trying to get? > > Cheers, > Tom > > On Thu, Jan 22, 2009 at 3:25 PM, Brian MacKay > <[email protected]> wrote: >> Hello, >> >> >> >> Any tips would be greatly appreciated. >> >> >> >> Is there a way to set the order of the keys in reduce as shown below, > no >> matter what order the collection in MAP occurs in. >> >> >> >> Thanks, Brian >> >> >> >> >> >> public void map(WritableComparable key, Text values, >> >> OutputCollector<Text, Text> output, Reporter reporter) >> throws IOException { >> >> >> >> //collect many CAT_A and CAT_B in random order >> >> output.collect(CAT_A, details); >> >> output.collect(CAT_B, details); >> >> >> >> } >> >> >> >> >> >> >> >> public void reduce(Text key, Iterator<Text> values, >> >> OutputCollector<Text, Text> output, Reporter >> reporter) throws IOException { >> >> >> >> //always reduce CAT_A first, then reduce CAT_B >> >> >> >> } >> >> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ >> >> The information transmitted is intended only for the person or entity > to >> which it is addressed and may contain confidential and/or privileged >> material. Any review, retransmission, dissemination or other use of, > or >> taking of any action in reliance upon, this information by persons or >> entities other than the intended recipient is prohibited. If you > received >> this message in error, please contact the sender and delete the > material >> from any computer. >> >> > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. Any review, retransmission, dissemination or other use of, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this message in error, please contact the sender and delete the material > from any computer. > > >
