Using 0.20.2...

Is it possible to override the context.write() method in ReduceContext? I have 
an entire set of Reducers that I would like to all use a specific function just 
before every context.write() but I don't want them to worry about that logic, 
just to have it handled transparently.

For instance:

Iterator<Text> vit = values.iterator();

if (trans2 != null) {
    key = (Text) trans2.transform(key);
}

while (vit.hasNext()) {
    Text item = vit.next();
    if (trans1 != null) {
        item = (Text) trans1.transform(item);
    }
    context.write(key, item);
}

The logic before/after the write is often different, and there may be writes 
that happen in different cases. I want to move the if->transform functionality 
out to the write() function so I can just call context.write(a,b) instead of if 
(trans1 !=....

My end-goal is something like this:

protected void reduce(Text key, Iterable<Text> values, Context context) 
        throws IOException, InterruptedException {
    Iterator<Text> vit = values.iterator();

    while (vit.hasNext())
        context.write(key, vit.next());
}

With the write() method override:

public void write(Text key, Text val) {
    if (trans1 != null)
        val = trans1.transform(val);
    if (trans2 != null)
        key = trans2.transform(key);
    super.write(key,val);
}

or something similar.

Thanks in advance,

-Ross

--
Ross Nordeen
Computer Networking And Systems Administration
Michigan Technological University
http://www.linkedin.com/in/rjnordee

Reply via email to