Implement a custom OutputFormat (http://hadoop.apache.org/core/docs/
current/mapred_tutorial.html#Job+Output) and a custom RecordWriter
(http://hadoop.apache.org/core/docs/current/
mapred_tutorial.html#RecordWriter). In the write() method of the your
RecordWriter you can do the filtering based on keys.
Arun
On Apr 15, 2008, at 3:20 PM, Aayush Garg wrote:
HI,
Could you please suggest what classes and another better way to
achieve
this:-
I am getting outputcollector in my reduce function as:
void reduce(....)
{
output.collect(key,value);
}
Here key is Text,
and value is Custom class type that I generated from rcc.
1. After all calls are complete to reduce function, I need to
eliminate
certain rows in this outputformat based on keys. I guess I need to
store
this outputformat in some static Map(declared in Reduce class) and
need to
do required operations from the Main function. Is this right approach?
2. This stored outputformat I want to use for another Map Reduce
job. What
classes and format should I use in the previous step so that I can
easily
use this as input in another program invoking MR job.
Regards,
Garg
