All the junit tests in hadoop are system type tests (as you've
noted). But, since Mapper and Reducer method parameters are all
interfaces, it should be easy to use something like EasyMock or jMock
to mock the input parameters and test your classes.
Cheers,
Nige
On Sep 21, 2007, at 8:25 AM, kate rhodes wrote:
I've figured out the details of how to do it. I'll post a generalized
tutorial to the wiki this evening.
--
- kate = masukomi
http://weblog.masukomi.org/
On 9/21/07, kate rhodes <[EMAIL PROTECTED]> wrote:
On 9/21/07, Enis Soztutar <[EMAIL PROTECTED]> wrote:
Hi,
You can develop your unit tests by using classes under src/test.
Infact
you may find MiniMRCluster and MiniMRDFS very useful.
Alternatively you
may invoke the program in local mode, with a smaller input to
check the
outcomes.
Anything that involves communicating with another class is NOT a unit
test. It's an integration test because problems in that other class
could give false failures. Also, bugs you're unknowingly relying
on in
the other class could provide false passes. If, for example, someone
introduced a bug in MiniMRCluster and I was relying on it in my test
for my Mapper then in would cause my test to fail. And that's
bull. It
means I wasn't testing my Mapper, I was testing the integration of my
Mapper with the MiniMRCluster with probably 40 or 50 other classes
that get tied in.
A unit test *has* to be totally isolated. The class under test should
be allowed no communication with any other object that isn't a
mock-object otherwise the problems noted above crop up. I need to
know
that the class I'm testing is written correctly regardless of what
other bugs may or may not have been introduced in other classes.
Otherwise I see my class fail and start wondering what I've
screwed up
in it, and what needs debugging even if my class is 100% correct. I'm
not willing to waste my time hunting the causes of bogus failures
when
i could have a test suite that actually told me the correct class
that
had the bug in it.
There are good reasons to write integration tests but you have to
understand that they are dramatically different beasts than unit
tests.
I had already found those classes you mentioned, but, like I said, I
haven't been able to find any examples of a UNIT test of a Mapper or
Reducer (integration tests don't count).
--
- kate = masukomi
http://weblog.masukomi.org/
