On Wed, Feb 11, 2015 at 4:53 PM, Eric Snow <[email protected]> wrote: > tl;dr Using fakes for testing works well so I wrote a base fake type. [1] > > While working on the GCE provider, Wayne and I started taking a > different approach to unit testing than the usual 1. expose an > external dependency as an unexported var; 2.export it in > export_test.go; 3. patch it out in tests. [2] Instead we wrote an > interface for the provider's low-level API and implemented the > provider relative to that. [3] Then in tests we used a fake > implementation of that interface instead of the concrete one. Instead > of making the actual API requests, the fake simply tracked method > calls and controlled return values.
This is a "mock object" under some well known people's terminology [1]. There are certainly benefits, but there are also very well known problems in that approach which should be kept in mind. We've experienced them very closely in the first Python-based implementation of juju, and I did many other times elsewhere. I would probably not have written [2] if I had a time machine. The most problematic aspect of this approach is that tests are pretty much always very closely tied to the implementation, in a way that you suddenly cannot touch the implementation anymore without also fixing a vast number tests to comply. Tests organized in this fashion also tend to obfuscate the important semantics being tested, and instead of that you see a sequence of apparently meaningless calls and results out of context. Understanding and working on these tests just a month after you cooked them is a nightmare. The perfect test breaks if and only if the real assumption being challenged changes. Anything walking away from this is decreasing the test quality. As a recommendation to avoid digging a hole -- one that is pretty difficult to climb out of once you're in -- instead of testing method calls and cooking fake return values in your own test, build a real fake object: one that pretends to be a real implementation of that interface, and understands the business logic of it. Then, have methods on it that allow tailoring its behavior, but in a high-level way, closer to the problem than to the code. [1] http://martinfowler.com/articles/mocksArentStubs.html [2] https://labix.org/mocker gustavo @ http://niemeyer.net -- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
