On 9/21/07, Nigel Daley <[EMAIL PROTECTED]> wrote: > All the junit tests in hadoop are system type tests (as you've > noted). But, since Mapper and Reducer method parameters are all > interfaces, it should be easy to use something like EasyMock or jMock > to mock the input parameters and test your classes.
You are correct... in part. There are two problems that I discovered with this. 1) JobConf does not implement an interface it extends Configuration and the only thing Configuration implements is Iterable. 2) and this is the scary one... There are NO mock object libraries OR objects in Hadoop. OMFGWTFBBQ. Seriously guys. This means there's *probably* not a single UNIT test in Hadoop, just lots of integration and system tests. Not to diss the integration tests you guys have written but they're NOT unit tests and that fact means that you're just hurting yourselves when it comes time to debug because a bug in one class has the potential to show up as a failure in any of the other classes that use it in their tests, which can mislead you into thinking that you've got a bug in a class that's perfectly fine. Now, I've got no problems using EasyMock and / or jMock for testing my personal apps I do it all the time, but it means that I can't add any to Hadoop without convincing someone to add in one of those two jars (actually in jMock's case it looks like a whole pile-o-jars). If someone will add EasyMock or jMock to Hadoop I'd be willing to help refactor some of the existing tests that were intended to be unit tests. Or, if there's a potential licencing issue with distributing it with Hadoop at least declare one of them to be a defacto part of Hadoop (like the missing ant.jar) so that people can start writing tests that use them. Speaking of which WHY is the ant.jar not included with Hadoop? It depends on it, and there's obviously not a licensing issue, but you can't do half the things in it without copying it into the lib dir. -- - kate = masukomi http://weblog.masukomi.org/
