Dino, Ya that was another method we are using now. thanks
-Ross -- Ross Nordeen Computer Networking And Systems Administration Michigan Technological University http://www.linkedin.com/in/rjnordee ----- Original Message ----- From: "Dino Kečo" <dino.k...@gmail.com> To: mapreduce-user@hadoop.apache.org, "Ross" <rjnor...@mtu.edu> Sent: Saturday, August 13, 2011 1:29:14 AM GMT -08:00 US/Canada Pacific Subject: Re: Possible to override the context.write() method in ReduceContext? Hi Ross, The first thing that I should do is to create one main reducer and create writeToContext method in it. This method should handle your logic with if(...){...} and context.write(). All other reducers would extend this main reducer and call writeToContext instead context.write. Thanks, dino On Fri, Aug 12, 2011 at 11:06 PM, Ross Nordeen < rjnor...@mtu.edu > wrote: Using 0.20.2... Is it possible to override the context.write() method in ReduceContext? I have an entire set of Reducers that I would like to all use a specific function just before every context.write() but I don't want them to worry about that logic, just to have it handled transparently. For instance: Iterator<Text> vit = values.iterator(); if (trans2 != null) { key = (Text) trans2.transform(key); } while (vit.hasNext()) { Text item = vit.next(); if (trans1 != null) { item = (Text) trans1.transform(item); } context.write(key, item); } The logic before/after the write is often different, and there may be writes that happen in different cases. I want to move the if->transform functionality out to the write() function so I can just call context.write(a,b) instead of if (trans1 !=.... My end-goal is something like this: protected void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException { Iterator<Text> vit = values.iterator(); while (vit.hasNext()) context.write(key, vit.next()); } With the write() method override: public void write(Text key, Text val) { if (trans1 != null) val = trans1.transform(val); if (trans2 != null) key = trans2.transform(key); super.write(key,val); } or something similar. Thanks in advance, -Ross -- Ross Nordeen Computer Networking And Systems Administration Michigan Technological University http://www.linkedin.com/in/rjnordee
