On 9/3/15 10:11 AM, Greg DeKoenigsberg wrote:
3. Ensure that variable precedence is rigorously tested. Remember that
2.0 is still in alpha, and regressions should be temporary, so long as
you help us by reporting them. We do have some unit and integration
tests and we are working on cleaning them up, and we welcome more
tests not covered already by existing cases.
Thanks for the response, Greg. This is the thing, though — I've provided
those tests, repeatedly, and they've never been used, even recently with
2.0. Developers have even asked me on GitHub whether bugs were still
present, which is kind of absurd when I've provided simple test cases. I
shouldn't have to run my own private test cases every time I try a new
version so that I can let the developers know if bugs have reappeared.
Here's an example of the kind of simple test cases I've provided:
https://github.com/ansible/ansible/issues/9498
That never should have been able to regress without developers noticing
(and given that the fix for the regression, two weeks ago, doesn't
appear to have been accompanied by any tests, I'm not particularly
confident that it won't again).
The existing variable precedence tests are clearly inadequate, and I
don't think there's any way they can be sufficient in the existing test
layout, with one variable precedence playbook and a handful of roles
mixed in haphazardly with all the other test files. I'd appreciate if
someone could comment on my suggestion for reorganizing the test suite
into separate directories, with individual test cases and all their
related files grouped together (which is how I'm testing these locally,
after all). It would make it trivial to integrate all the test cases
I've provided — I could even provide pull requests — and would help with
keeping track of separate issues and documenting the expected behavior.
I don't really see another solution here.
- Dan
--
You received this message because you are subscribed to the Google Groups "Ansible
Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/55E9198B.4040708%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
