On Jul 25, 2014, at 7:46 AM, Tobias Oberstein <[email protected]> wrote:
> E.g., for integration tests, we (Autobahn|Python) have Autobahn|Testsuite, > which actually tests interaction of components over the network. > > For low-level (per function) testing, we have (synchronous) unit tests. > > But there is a gap between those latter two. > > It is this level of testing that I am missing. This is the reason that this feature still exists, despite its use being "discouraged". I'm putting "discouraged" in scare quotes because it's not really that we discourage using it, it's that we discourage using it for the wrong reasons - but almost everyone uses it for the wrong reasons, so it's easier just to say "don't do it*" "*: actually do it sometimes" than to try to always explain the cases where you might want to use it. It's not a bad feature. In fact it's a great feature! But it's a great feature for integration testing, not a great feature for unit testing. Integration testing is something you should do after you have a fully working unit test suite and the components all work as you expect in isolation, and you are ready to make sure that they fulfill their system contracts - in other words, you do it to experimentally discover gaps in your unit tests, not as part of test-driven design. The only time I've ever used Deferred-returning tests on a regular basis and not had nightmares about it was when I was testing a system that made heavy use of an asynchronous external database, and particularly made heavy use of features of PostgreSQL which could not be easily mocked within an in-process synchronous database such as SQLite. I felt OK about it because there were low-level tests that exhaustively tested the database binding to make sure there were never any application-level surprises. I knew that those low-level tests worked because none of the high-level Deferred-returning tests ever failed intermittently despite being run on a wide array of hardware :-). Anyway, to make a long story short, returning a Deferred from a test is a very advanced, somewhat risky feature that you should use only if you have no other tools at your disposal that do what you want to do. And there are probably more tools to do what you want to do than you might initially think :). Happy testing, -glyph
