[
https://issues.apache.org/jira/browse/MRUNIT-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224378#comment-13224378
]
Jarek Jarcec Cecho commented on MRUNIT-69:
------------------------------------------
I quite like Jim's idea. I'm looking forward to simplify usage even more. I
could imagine even more simplification to save any necessary typing. For
example something like:
class CoolMapperTest extends MapperTest<CoolMapper.class> {
public testProperArguments() {
withInput(new LongWritable(1), new Text("hi"));
withOutput(new Text("hi"), new LongWritable(1));
withCounter("MyGroup", "MyCounter", 1);
}
}
I'm expecting here that user will have one test class per one mapper (reducer,
map reduce pair) which is something that I'm already doing in my tests to keep
them clean. I'm not sure whether others are doing it as well. Another important
note here is that I'm expecting that MapperTest will execute all methods
starting with name "test" to set up testing environment (inputs, outputs,
counters, ...). However test itself would be executed by MapperTest and user
wouldn't have any control over it which might be limiting for some use cases.
However those problems might be address by exposing API in MapperTest that can
override default behavior.
> longterm plan for MRUNIT development?
> -------------------------------------
>
> Key: MRUNIT-69
> URL: https://issues.apache.org/jira/browse/MRUNIT-69
> Project: MRUnit
> Issue Type: Brainstorming
> Reporter: Jim Donofrio
> Priority: Minor
>
> So I am curious what the plan is for the longterm future of MRUNIT?
> I think currently MRUNIT is useful for just unit testing a single mapper or
> reducer but currently there is a void for testing more complicated features
> such as MultipleInputs, MultipleOutputs, a driver class, counters, among
> other things. I wonder if instead of adding support to the current MRUNIT
> framework for these extra features it would more useful to add in hooks to
> the existing LocalJobRunner and MiniMRCluster classes to provide methods to
> more easily verify file output from text files, sequence files, etc. This
> would allow MRUNIT to test driver classes, MultipleInputs, MultipleOutputs,
> etc. MRUNIT would also then test against the real hadoop code instead of an
> implementation that mimics hadoop which can miss some bugs such as the
> ReduceDriver that did not reuse the same object until 0.8.0. MRUNIT would
> also keep up with new map reduce features instead of us having to implement
> fake versions of them
> I understand that performance would be an issue due to the file I/O but I
> wonder how fast the LocalJobRunner would be if we wrote a new class that
> extending FileSystem to allow users to write out fake files to memory and
> make the LocalJobRunner read from them
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
