On Sep 22, 2007, at 2:59 PM, kate rhodes wrote:
On 9/21/07, Nigel Daley <[EMAIL PROTECTED]> wrote:
All the junit tests in hadoop are system type tests (as you've
noted). But, since Mapper and Reducer method parameters are all
interfaces, it should be easy to use something like EasyMock or jMock
to mock the input parameters and test your classes.
You are correct... in part. There are two problems that I
discovered with this.
1) JobConf does not implement an interface it extends Configuration
and the only thing Configuration implements is Iterable.
2) and this is the scary one... There are NO mock object libraries OR
objects in Hadoop. OMFGWTFBBQ. Seriously guys. This means there's
*probably* not a single UNIT test in Hadoop, just lots of integration
and system tests. Not to diss the integration tests you guys have
written but they're NOT unit tests and that fact means that you're
just hurting yourselves when it comes time to debug because a bug in
one class has the potential to show up as a failure in any of the
other classes that use it in their tests, which can mislead you into
thinking that you've got a bug in a class that's perfectly fine.
Welcome to Open Source :-) Definitions on what unit tests are/aren't
w.r.t. to OOP have only recently started to converge around those
popularized by Martin Fowler, Kent Beck, et al. ('converge' is likely
too strong a word). It's possible that many developers aren't test
driven, don't keep up on the test philosophy du jour, and/or don't
agree with these definitions. Most days I'm just grateful that our
agreed upon code review guidelines include checks for tests:
http://wiki.apache.org/lucene-hadoop/CodeReviewChecklist
Now, I've got no problems using EasyMock and / or jMock for testing my
personal apps I do it all the time, but it means that I can't add any
to Hadoop without convincing someone to add in one of those two jars
(actually in jMock's case it looks like a whole pile-o-jars).
You can always submit it as part of a patch if the library is license
compatible:
http://people.apache.org/~cliffs/3party.html
Nige
