This all looks reasonable to me. I wonder if most mocking tools don't support programmatic definition of return values because it makes the declaration of expectations less explicit.
Drew On Fri, Oct 28, 2011 at 11:19 AM, John W Vines <[email protected]> wrote: > I will start off by saying that myself, as well as most of the committers, > are no familiar with EasyMock, PowerMock, or really any Mocking tools. But > I've done some reading to determine what the scope of mocking can get us. So > pardon me if I bore you because I'm just going to lay out what I know in > hopes of A. making sure my understanding is correct and B. to see if what I > think we should use it for is within the scope of Mocking as well as feasible. > > 1. With exception for PowerMock, mocking tools are nothing more than tools > which give the ability to define a pre-set set of responses to an interface. > That is, if we wanted to implement a low level iterator to replace the Reader > we use to read items off disk/out of the in memory map, we could, but we > could only give it a set of responses to give when queried. We could not > (through just the mocking tools) set it up to give programmatic responses, > such as returning the result of another method, unless we 'pre-load' the > method result. That is, we couldn't have one mock method which put an item in > a map and another mock method pull it out again, unless we define the pull > method to return that specific value for the Xth call to the pull method to > which that correlated. > > 2. PowerMock lets us essentially do the same as things like EasyMock, except > we're not bound to interfaces. This way we can use a defined class and only > overwrite the methods we want with predefined results. > > 3. What I want to see us doing, at a very high level, is to have the ability > to mock an entire TServer to the extent where we will use something to > replace Zookeeper (We should probably turn our ZK work with an interface) > with a MockZookeeper (not generated through a Mock util) which is nothing > more than a Map. Same thing with the FileReader, except a SortedMap, the > loggers, and the master. This way we could fully implement a whole TServer > without worry about HDFS and Zookeeper. To a similar extent I would like to > see this done for all core components, but mocking the various connectors we > use to get done what we need to. I see a few sets of Mock class we will have > to create. But with less chance of divergence in behavior then we currently > experience with our MockAccumulo setup. > > 4. My principal concern about 3 above is divergence with the code (less so > than our current setup but with emulating thrift interfaces we could get > problems). But also the feasibility of mocking thrift connections. But if all > this works out, I think we'll be able to optimize our code coverage better > than our current setup without the requirement to do a full Accumulo setup > for junit tests. > > Cheers > John > > ----- Original Message ----- > | From: "Jesse Yates" <[email protected]> > | To: [email protected] > | Sent: Thursday, October 27, 2011 2:21:33 PM > | Subject: Re: Mocking framework > | On Thu, Oct 27, 2011 at 10:55 AM, Drew Farris <[email protected]> wrote: > | > | > I agree in principle without having reviewed the tests accompanying > | > ACCUMULO-53. > | > > | > I've used EasyMock in the past and have been very happy with the > | > ability to programatically mock implementations of interfaces, > | > define > | > test input and assert a proper sequence of calls, and output. In > | > general I think mock object testing is a useful tool, especially in > | > cases where the existing interface implementations have heavyweight > | > external dependencies, such as for mocking JDBC result sets or > | > portions of the Hadoop framework. I've used mock objects to write > | > unit > | > tests for mappers and reducers by mocking out the hadoop context, > | > counters, etc. Arguably this would be have been better achieved with > | > MRUnit, which wasn't on my radar at the time. > | > > | > I found that I didn't have to jump through hoops to make code > | > testable. Private methods could really be private instead of > | > protected > | > or public in order to facilitate testing. > | > | > | The comment about testing private methods was only a comment on the > | power of > | the framework, not on what should be done. IMHO you shouldn't be > | testing > | private methods w/o a very good reason. > | > | > | > I also appreciate the > | > ability to not only assert return values, but assert a series of > | > method calls and side effects. > | > > | > The danger with this is only testing in isolation using mocks -- If > | > the mocks don't follow the contract or conventions the > | > implementations > | > the tests are useless or deceptive. Implementation class behavior > | > can > | > change in ways that are invisible from an api perspective, so it's > | > important that these sorts of changes be reflected adequately by the > | > mocks themselves. > | > | > | Of course there is also the case where you end up just checking to > | make sure > | the object under test calls a series of methods on your mock - not a > | really > | informative test and bring of the correct interface issue you > | mentioned. > | > | > | > Coupled with adequate integration tests this danger > | > is minimized. Generally I think the tradeoff in increased > | > testability > | > makes up for these drawbacks. > | > > | > | +1 for good integration tests; they tend to mitigate that issue. > | > | > | > > | > Drew > | > > | > On Thu, Oct 27, 2011 at 12:49 PM, Jesse Yates > | > <[email protected]> > | > wrote: > | > > Hi all, > | > > > | > > Currently Accumulo doesn't have any mocking framework. However, in > | > > some > | > > recent work (ACCUMULO-53) doing good testing was greatly > | > > facilitated by > | > > adding in a mocking framework. They make it easy to only test the > | > > piece > | > of > | > > the architecture that we are interested in testing, helping to > | > > isolate > | > and > | > > ensure that we are really fixing problems in the code. > | > > > | > > So, I would like to propose adding the following test-time > | > > dependencies > | > for > | > > testing: > | > > > | > > EasyMock (http://easymock.org/) > | > > This is a really popular mocking framework for java and is really > | > > easy to > | > > use. I've rarely run into a situation that wasn't covered with > | > > what > | > EasyMock > | > > does. > | > > > | > > PowerMock (http://code.google.com/p/powermock/) > | > > This is the utility to cover what other mocking frameworks > | > > (including > | > > Mockito) can't do - capturing creation of objects (new > | > > Whatever(...)) and > | > > returning mocks instead, mocking static calls, mocking standard > | > > java lib > | > > calls, and accessing private methods/fields. > | > > > | > > This combination has proven to be incredibly effective and easy to > | > > use in > | > > the past, allowing for a really comprehensive testing suite. > | > > > | > > If you want to see an example of this stuff in action, check out > | > > the > | > patch > | > > 'java_ACCUMULO-53_v2.patch' on ACCUMULO-53. > | > > > | > > What does everyone think? > | > > > | > > Thanks! > | > > --Jesse > | > > > | > > ------------------- > | > > Jesse Yates > | > > 240-888-2200 > | > > @jesse_yates > | > > > | > > | > | > | > | -- > | ------------------- > | Jesse Yates > | 240-888-2200 > | @jesse_yates >
