I am looking for some guidance that will help me write better tests for our map reduce jobs. My current jobs are tested using MRUnit, which covers most of the "logic" but I feel like I am missing good "end-to-end" developer tests.
I took a look at the tests for mapred classes but I am not sure that it achieves my goal of an end-to-end test because of the use of MockInstance. https://github.com/apache/accumulo/blob/master/mapreduce/src/test/java/org/apache/accumulo/core/client/mapred{,reduce}/ For me the characteristic of a end-to-end test that I would find valuable is a suite that one could execute using mini-{accumulo,yarn,et.al.} but I don't see any examples of how one would go about making those components work in concert with each other. Does anyone have any guidance when it comes to writing automated developer end-to-end tests? What kinds of testing strategies are people out there using for MR jobs? Thanks, Ariel Valentin e-mail: [email protected] website: http://blog.arielvalentin.com skype: ariel.s.valentin twitter: arielvalentin linkedin: http://www.linkedin.com/profile/view?id=8996534 --------------------------------------- *simplicity *communication *feedback *courage *respect
