I want to know if I can "wire" mapper and reducer classes. By "wire" I mean
to use inversion of control (or dependency injection). For example, I see in
a lot of books and on the Map/Reduce tutorial page (
http://hadoop.apache.org/common/docs/r0.20.0/mapred_tutorial.html),
something like this:
conf.setMapperClass(MyMapper.class);
conf.setReducerClass(MyReducer.class);
However, I want do something like this:
MyMapper mapper = new MyMapper();
mapper.setSomeProperty("something");
MyReducer reducer = new MyReducer();
reducer.setSomeProperty("somethingElse");
JobConf conf = new JobConf();
conf.setMapper(mapper);
conf.setReducer(reducer);
Is this possible? I'm still learning MapReduce so I don't know if this is
possible. But all examples that I have come across follow the pattern on the
tutorial page. There is a work around that I've been forced to. In my mapper
and reducer class, I can grab some properties file and then fill in the
values that I need (for example for JDBC connection). I see the problem that
when I run this on a distributed environment, I'll need the properties files
there as well. Does the mapper and reducer classes have a reference back to
the JobConf object? If so, I suppose I can get the properties there.
Thanks.
