The testing pain that I have had most is actually the pain of testing a client library against failure modes. The scaffolding that would most help there would be a mocked Zookeeper client that I could use to trigger disconnections and expirations and such. Launching a local server is distinctly sub-optimal for several reasons. One is that I can't reliably run tests at the same time. Another is that it is hard to trigger possible race conditions with an independent server.
It isn't terribly hard to build this, but it has so far exceeded the quantum of time that I typically have available. On Tue, Nov 1, 2011 at 9:11 AM, Patrick Hunt <[email protected]> wrote: > I'm not suggesting it's an all or nothing proposition. We have a lot > of system/functional level testing, which is great. What we lack is > unit testing. Take this recent ZOOKEEPER-1271 issue as an example - in > many cases it's very hard, at the system level, to exercise all the > code paths esp the failure cases. Perhaps if we had more breadth we > could have caught this earlier. > > I'm open to all types of testing, these were just some ideas I had for > "big bang for the buck" type scenario. I encourage you all to add > additional testing, or new types of testing as you see fit. > > For example, another area where we are weak is enabling ppl to test > their client code, Camille if you contribute that back I think it > would be awesome! We could layer other things on top of that, for > example some way to trigger different behaviors on the servers which > would cause client failover and such. (the problem there is validating > correct client behavior). > > Patrick > > On Tue, Nov 1, 2011 at 9:01 AM, Fournier, Camille F. > <[email protected]> wrote: > > I also like mockito. It has some limitations but generally can be very > useful. However, I'm also very much a fan of fixing our code base to be > more testable against real servers. I actually like the fact that we use a > lot of "integration" testing in our automated build. You do catch things > that you would miss in mockito unless you were very good at thinking > through all possible orderings of behavior. > > > > My internal ZK java client testing goes almost entirely against mockito > and in retrospect I think I would've been possibly better off using mostly > a real server started by the unit tests. Not that I missed a lot of bugs > using mockito, but what I gained in speed of tests I lost in readability of > the test cases for people that aren't fluent in mocking. > > > > Aspects are still a deep mystery to me. I know they can be terribly > useful but wow are they hard to read. > > > > I can probably throw together a few quick mockito sample tests to get > the ball rolling on this. After I look at some of the patch backlog. > > > > C > > > > -----Original Message----- > > From: Mahadev Konar [mailto:[email protected]] > > Sent: Tuesday, November 01, 2011 12:52 AM > > To: [email protected] > > Subject: Re: Better testing for client code... > > > > I am all for using mockito. I have found it very handy in writing good > > unit tests. Our codebase needs little changes to be able to unit test > > our codebase. > > > > Writing a "real" unit test for the java client would be a good use > > case to see what all changes need to happen on the client. > > > > thanks > > mahadev > > > > On Mon, Oct 31, 2011 at 1:20 PM, Patrick Hunt <[email protected]> wrote: > >> I've been thinking that we need better testing for the client code. In > >> particular it's currently difficult to simulate all the different > >> types of failures a client might see. I'm wondering how we might do > >> this - two ideas come to mind; 1) aspectj and 2) mocks (mockito). > >> Anyone have other suggestions? > >> > >> I've used 1 before on an ad-hoc basis to find issues in the server > >> quorum code (aspects which would randomly cause a connect, bind, > >> write, read, etc.... to fail). 2 seems better from a deterministic > >> basis (ie repeatable tests that are easier to verify pass/fail). The > >> only issue I see with 2 (other than refactoring some of the client > >> code) is the need to mock networking level code. Perhaps both > >> approaches could be applied together or separately. > >> > >> Thoughts? > >> > >> Patrick > >> > > >
