On 10/30/2013 10:37 PM, Robert Collins wrote:
This is a bit of a social norms thread....
I've been consistently asking for tests in reviews for a while now,
and I get the occasional push-back. I think this falls into a few
broad camps:
A - there is no test suite at all, adding one in unreasonable
B - this thing cannot be tested in this context (e.g. functional tests
are defined in a different tree)
C - this particular thing is very hard to test
D - testing this won't offer benefit
E - other things like this in the project don't have tests
F - submitter doesn't know how to write tests
G - submitter doesn't have time to write tests
Now, of these, I think it's fine not add tests in cases A, B, C in
combination with D, and D.
I don't think E, F or G are sufficient reasons to merge something
without tests, when reviewers are asking for them. G in the special
case that the project really wants the patch landed - but then I'd
expect reviewers to not ask for tests or to volunteer that they might
be optional.
Now, if I'm wrong, and folk have different norms about when to accept
'reason X not to write tests' as a response from the submitter -
please let me know!
First - providing tests with code is a basic design tenant of OpenStack
- https://wiki.openstack.org/wiki/BasicDesignTenets (I realize we don't
highlight that as much as we used to, but it really is baked in culture
to people that have been here for a while).
This is pretty close to my thinking. I like to challenge people on C.
There was a pretty good case of it late in grizzly or havana where I was
reviewing some nova virt code and the sumbitter claimed it was too hard
to test. A week later danpb wrote a 100 line patch that provided the
entire test framework for the case. So "C" was a pretty BS answer.
Adding tests and test frameworks are a way of paying it forward in the
project. Much like doing reviews, adding tests is a way to save other
people on the project time. Good tests pay for themselves very quickly
in time lost on the project.
So I generally find "I don't want to add tests" the same kind of
selfishness as "why should I have to review code?".
A good example of the kind of savings we get, over the summer when were
were regularlly breaking on requirements mismatch (and on the update
script modifying things incorrectly), I went in and added a testing
framework to requirements repo, so we knew all the parts were doing
expected things. Without that we'd still be fighting these things on a
weekly basis, now it's a solved problem, and we can go fight the next fire.
This was actually as C class problem (which is why no one else had done
it yet), and required building new devstack job to complete. However the
benefits were really high on it.
-Sean
--
Sean Dague
http://dague.net
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
