Hi all. Gavin brought up a good question in a MP (https://code.edge.launchpad.net/~leonardr/lazr.authentication/initial-implementation/+merge/12989
) and I wanted to highlight the discussion to the list.
The question is whether packages we release should include their test
dependencies, even if we do not believe them to be necessary for the
packages' operation. For instance, should setup.py for
lazr.authentication specify the testing tool wsgi_intercept as an
installation requirement, even if it is only a testing tool?
Gavin felt we shouldn't, and I felt we should. Sounds like it's time
for a discussion. :-) The following is my argument, ripped from the MP.
There's a tension between wanting to reduce dependencies for your
users, and wanting to be confident that your tests actually reflect
the performance of your code. The second consideration is sometimes
called "test what you fly, fly what you test," and is associated in
the States with NASA, our space agency. I have seen well-tested code
that directly or indirectly depended on a package that was expected to
only be a test dependency, but was actually a run-time dependency.
Because the automated tests relied on the test dependencies to run,
the only way to catch these kinds of packaging errors was to do manual
QA that effectively duplicates the role of your automated tests. I
don't think that's a good way to release packages: I want to rely on
my automated tests.
Therefore, I think that the "test what you fly, fly what you test"
axiom should be followed as a rule, unless there are really compelling
reasons not to. A few additional dependencies are generally worth it
to me for the additional confidence in my releases.
I do think that there is a subjective point at which the benefit
outweighs the cost. I probably start to feel that I can break the
rule for library packages when the test dependencies are hard to build
or have a count of more than five or six. On the other hand, I pretty
much never want to break the rule for deployments of production
systems. In other words, if lazr.authentication started getting
really bloated because of test dependencies, I would be tempted to
break the rule for its release, and separate out the test
dependencies; contrariwise, if Launchpad got a lot of test
dependencies, I would not be tempted at all to break the rule for its
deployment.
Perhaps this is worth a reviewers' meeting discussion, or maybe we can
reach consensus on the list.
Gary
_______________________________________________
Mailing list: https://launchpad.net/~launchpad-dev
Post to : [email protected]
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
