I have been trying to digest the following series of talks between Martin Fowler, Kent Beck, and David Heinemeier, called "Is TDD Dead? <http://martinfowler.com/articles/is-tdd-dead/>". The topic is a bit inflammatory, but there's some good stuff here.
Some major points discussed: - Is there such a thing as test-induced damage to the architecture? - Should you expect your developers to practice TDD (i.e. does that workflow work for everyone) - Mocks vs. Integration (inside-out vs. outside-in testing) There's a lot to chew on here. Personally, I agree with what others have said in this thread: I think like most things in life, you need to find the middle-way and have both integration and unit tests. To know how healthy your testing ecosystem is, ask yourself these 3 questions: 1. Which do I spend more time on: my changes, or writing tests for my changes? 2. Do I need to learn an entirely orthogonal way of interacting with my code for testing? 3. How many defects are getting through? Enjoying following this thread. - Katherine On Fri, Sep 12, 2014 at 5:42 AM, Michael Foord <[email protected]> wrote: > > On 12/09/14 06:05, Ian Booth wrote: > >> >> On 12/09/14 01:59, roger peppe wrote: >> >>> On 11 September 2014 16:29, Matthew Williams >>> <[email protected]> wrote: >>> >>>> Hi Folks, >>>> >>>> There seems to be a general push in the direction of having more >>>> mocking in >>>> unit tests. Obviously this is generally a good thing but there is still >>>> value in having integration tests that test a number of packages >>>> together. >>>> That's the subject of this mail - I'd like to start discussing how we >>>> want >>>> to do this. Some ideas to get the ball rolling: >>>> >>> Personally, I don't believe this is "obviously" a good thing. >>> The less mocking, the better, in my view, because it gives >>> better assurance that the code will actually work in practice. >>> >>> Mocking also implies that you know exactly what the >>> code is doing internally - this means that tests written >>> using mocking are less useful as regression tests, as >>> they will often need to be changed when the implementation >>> changes. >>> >>> Let's assume that the term stub was meant to be used instead of >> mocking. Well >> written unit tests do not involve dependencies outside of the code being >> tested, >> and to achieve this, stubs are typically used. As others have stated >> already in >> this thread, unit tests are meant to be fast. Our Juju "unit" tests are >> in many >> cases not unit tests at all - they involve bringing up the whole stack, >> including mongo in replicaset mode for goodness sake, all to test a single >> component. This approach is flawed and goes against what would be >> considered as >> best practice by most software engineers. I hope we can all agree on that >> point. >> > > I agree. I tend to see the need for stubs (I dislike Martin Fowler's > terminology and prefer the term mock - as it really is by common parlance > just a mock object) as a failure of the code. Just sometimes a necessary > failure. > > Code, as you say, should be written as much as possible in decoupled units > that can be tested in isolation. This is why test first is helpful, because > it makes you think about "how am I going to test this unit" before your > write it - and you're less likely to code in hard to test dependencies. > > Where dependencies are impossible to avoid, typically at the boundaries of > layers, stubs can be useful to isolate units - but the need for them often > indicates excessive coupling. > > >> To bring up but one of many concrete examples - we have a set of Juju CLI >> commands which use a Juju client API layer to talk to an API service >> running on >> the state server. We "unit" test Juju commands by starting a full state >> server >> and ensuring the whole system behaves as expected, end to end. This is >> expensive, slow, and unnecessary. What we should be doing here is >> stubbing out >> the client API layer and validating that: >> 1. the command passes the correct parameters to the correct API call >> 2. the command responds the correct way when results are returned >> >> Anything more than that is unnecessary and wasteful. Yes, we do need >> end-end >> integration tests as well, but these are in addition to, not in place of, >> unit >> tests. And integration tests tend to be fewer in number, and run less >> frequently >> than, unit tests; the unit tests have already covered all the detailed >> functionality and edge cases; the integration tests conform the moving >> pieces >> mesh together as expected. >> >> As per other recent threads to juju-dev, we have already started to >> introduce >> infrastructure to allow us to start unit testing various Juju components >> the >> correct way, starting with the commands, the API client layer, and the API >> server layer. Hopefully we will also get to the point where we can unit >> test >> core business logic like adding and placing machines, deploying units etc, >> without having to have a state server and mongo. But that's a way off >> given we >> first need to unpick the persistence logic from our business logic and >> address >> cross pollination between our architectural layers. >> >> > +1 > > Being able to test business logic without having to start a state server > and mongo will make our tests soooo much faster and more reliable. The more > we can do this *without* stubs the better, but I'm sure that's not entirely > possible. > > All the best, > > Michael > > > -- > Juju-dev mailing list > [email protected] > Modify settings or unsubscribe at: https://lists.ubuntu.com/ > mailman/listinfo/juju-dev >
-- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
