On 12/09/14 06:05, Ian Booth wrote:
On 12/09/14 01:59, roger peppe wrote:
On 11 September 2014 16:29, Matthew Williams
<[email protected]> wrote:
Hi Folks,
There seems to be a general push in the direction of having more mocking in
unit tests. Obviously this is generally a good thing but there is still
value in having integration tests that test a number of packages together.
That's the subject of this mail - I'd like to start discussing how we want
to do this. Some ideas to get the ball rolling:
Personally, I don't believe this is "obviously" a good thing.
The less mocking, the better, in my view, because it gives
better assurance that the code will actually work in practice.
Mocking also implies that you know exactly what the
code is doing internally - this means that tests written
using mocking are less useful as regression tests, as
they will often need to be changed when the implementation
changes.
Let's assume that the term stub was meant to be used instead of mocking. Well
written unit tests do not involve dependencies outside of the code being tested,
and to achieve this, stubs are typically used. As others have stated already in
this thread, unit tests are meant to be fast. Our Juju "unit" tests are in many
cases not unit tests at all - they involve bringing up the whole stack,
including mongo in replicaset mode for goodness sake, all to test a single
component. This approach is flawed and goes against what would be considered as
best practice by most software engineers. I hope we can all agree on that point.
I agree. I tend to see the need for stubs (I dislike Martin Fowler's
terminology and prefer the term mock - as it really is by common
parlance just a mock object) as a failure of the code. Just sometimes a
necessary failure.
Code, as you say, should be written as much as possible in decoupled
units that can be tested in isolation. This is why test first is
helpful, because it makes you think about "how am I going to test this
unit" before your write it - and you're less likely to code in hard to
test dependencies.
Where dependencies are impossible to avoid, typically at the boundaries
of layers, stubs can be useful to isolate units - but the need for them
often indicates excessive coupling.
To bring up but one of many concrete examples - we have a set of Juju CLI
commands which use a Juju client API layer to talk to an API service running on
the state server. We "unit" test Juju commands by starting a full state server
and ensuring the whole system behaves as expected, end to end. This is
expensive, slow, and unnecessary. What we should be doing here is stubbing out
the client API layer and validating that:
1. the command passes the correct parameters to the correct API call
2. the command responds the correct way when results are returned
Anything more than that is unnecessary and wasteful. Yes, we do need end-end
integration tests as well, but these are in addition to, not in place of, unit
tests. And integration tests tend to be fewer in number, and run less frequently
than, unit tests; the unit tests have already covered all the detailed
functionality and edge cases; the integration tests conform the moving pieces
mesh together as expected.
As per other recent threads to juju-dev, we have already started to introduce
infrastructure to allow us to start unit testing various Juju components the
correct way, starting with the commands, the API client layer, and the API
server layer. Hopefully we will also get to the point where we can unit test
core business logic like adding and placing machines, deploying units etc,
without having to have a state server and mongo. But that's a way off given we
first need to unpick the persistence logic from our business logic and address
cross pollination between our architectural layers.
+1
Being able to test business logic without having to start a state server
and mongo will make our tests soooo much faster and more reliable. The
more we can do this *without* stubs the better, but I'm sure that's not
entirely possible.
All the best,
Michael
--
Juju-dev mailing list
[email protected]
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju-dev
