I will start off by saying that myself, as well as most of the committers, are no familiar with EasyMock, PowerMock, or really any Mocking tools. But I've done some reading to determine what the scope of mocking can get us. So pardon me if I bore you because I'm just going to lay out what I know in hopes of A. making sure my understanding is correct and B. to see if what I think we should use it for is within the scope of Mocking as well as feasible.
1. With exception for PowerMock, mocking tools are nothing more than tools which give the ability to define a pre-set set of responses to an interface. That is, if we wanted to implement a low level iterator to replace the Reader we use to read items off disk/out of the in memory map, we could, but we could only give it a set of responses to give when queried. We could not (through just the mocking tools) set it up to give programmatic responses, such as returning the result of another method, unless we 'pre-load' the method result. That is, we couldn't have one mock method which put an item in a map and another mock method pull it out again, unless we define the pull method to return that specific value for the Xth call to the pull method to which that correlated. 2. PowerMock lets us essentially do the same as things like EasyMock, except we're not bound to interfaces. This way we can use a defined class and only overwrite the methods we want with predefined results. 3. What I want to see us doing, at a very high level, is to have the ability to mock an entire TServer to the extent where we will use something to replace Zookeeper (We should probably turn our ZK work with an interface) with a MockZookeeper (not generated through a Mock util) which is nothing more than a Map. Same thing with the FileReader, except a SortedMap, the loggers, and the master. This way we could fully implement a whole TServer without worry about HDFS and Zookeeper. To a similar extent I would like to see this done for all core components, but mocking the various connectors we use to get done what we need to. I see a few sets of Mock class we will have to create. But with less chance of divergence in behavior then we currently experience with our MockAccumulo setup. 4. My principal concern about 3 above is divergence with the code (less so than our current setup but with emulating thrift interfaces we could get problems). But also the feasibility of mocking thrift connections. But if all this works out, I think we'll be able to optimize our code coverage better than our current setup without the requirement to do a full Accumulo setup for junit tests. Cheers John ----- Original Message ----- | From: "Jesse Yates" <[email protected]> | To: [email protected] | Sent: Thursday, October 27, 2011 2:21:33 PM | Subject: Re: Mocking framework | On Thu, Oct 27, 2011 at 10:55 AM, Drew Farris <[email protected]> wrote: | | > I agree in principle without having reviewed the tests accompanying | > ACCUMULO-53. | > | > I've used EasyMock in the past and have been very happy with the | > ability to programatically mock implementations of interfaces, | > define | > test input and assert a proper sequence of calls, and output. In | > general I think mock object testing is a useful tool, especially in | > cases where the existing interface implementations have heavyweight | > external dependencies, such as for mocking JDBC result sets or | > portions of the Hadoop framework. I've used mock objects to write | > unit | > tests for mappers and reducers by mocking out the hadoop context, | > counters, etc. Arguably this would be have been better achieved with | > MRUnit, which wasn't on my radar at the time. | > | > I found that I didn't have to jump through hoops to make code | > testable. Private methods could really be private instead of | > protected | > or public in order to facilitate testing. | | | The comment about testing private methods was only a comment on the | power of | the framework, not on what should be done. IMHO you shouldn't be | testing | private methods w/o a very good reason. | | | > I also appreciate the | > ability to not only assert return values, but assert a series of | > method calls and side effects. | > | > The danger with this is only testing in isolation using mocks -- If | > the mocks don't follow the contract or conventions the | > implementations | > the tests are useless or deceptive. Implementation class behavior | > can | > change in ways that are invisible from an api perspective, so it's | > important that these sorts of changes be reflected adequately by the | > mocks themselves. | | | Of course there is also the case where you end up just checking to | make sure | the object under test calls a series of methods on your mock - not a | really | informative test and bring of the correct interface issue you | mentioned. | | | > Coupled with adequate integration tests this danger | > is minimized. Generally I think the tradeoff in increased | > testability | > makes up for these drawbacks. | > | | +1 for good integration tests; they tend to mitigate that issue. | | | > | > Drew | > | > On Thu, Oct 27, 2011 at 12:49 PM, Jesse Yates | > <[email protected]> | > wrote: | > > Hi all, | > > | > > Currently Accumulo doesn't have any mocking framework. However, in | > > some | > > recent work (ACCUMULO-53) doing good testing was greatly | > > facilitated by | > > adding in a mocking framework. They make it easy to only test the | > > piece | > of | > > the architecture that we are interested in testing, helping to | > > isolate | > and | > > ensure that we are really fixing problems in the code. | > > | > > So, I would like to propose adding the following test-time | > > dependencies | > for | > > testing: | > > | > > EasyMock (http://easymock.org/) | > > This is a really popular mocking framework for java and is really | > > easy to | > > use. I've rarely run into a situation that wasn't covered with | > > what | > EasyMock | > > does. | > > | > > PowerMock (http://code.google.com/p/powermock/) | > > This is the utility to cover what other mocking frameworks | > > (including | > > Mockito) can't do - capturing creation of objects (new | > > Whatever(...)) and | > > returning mocks instead, mocking static calls, mocking standard | > > java lib | > > calls, and accessing private methods/fields. | > > | > > This combination has proven to be incredibly effective and easy to | > > use in | > > the past, allowing for a really comprehensive testing suite. | > > | > > If you want to see an example of this stuff in action, check out | > > the | > patch | > > 'java_ACCUMULO-53_v2.patch' on ACCUMULO-53. | > > | > > What does everyone think? | > > | > > Thanks! | > > --Jesse | > > | > > ------------------- | > > Jesse Yates | > > 240-888-2200 | > > @jesse_yates | > > | > | | | | -- | ------------------- | Jesse Yates | 240-888-2200 | @jesse_yates
