On 03/29/2015 09:43 PM, Thomas De Schampheleire wrote:
I don't think there is a CI server currently.
Mads: are you running one privately?
No, nothing continuous. I run the tests locally before deploying internally
or pushing upstream. That works ok. Right now I don't see a big need for CI;
I don't think it would solve a "real" problem.
The recent failures of two tests, caused by the Dulwich bump but only
visible when you actually deploy this new Dulwich, could have been
detected by a CI server (if they'd run 'python setup.py develop' each
time, that is).
But it's a cornercase, that's true.
Yes. That push was premature and unintended.
I think the right way to prevent it from happening again is to make sure
we just follow the usual and intended procedure and finish testing and
burn-in before pushing.
Still, no amount of testing can make sure that all combinations of
"supported" versions of external dependencies works correctly. (That is
why I prefer to keep constraints on the dependency versions so we at
least know we have done some amount of testing with all versions.) In
this case we had to "rush" an update of dulwich ... but it should still
have been tested more thorougly before pushing.
I can imagine some valuable milestones / deliverables:
* be able to run the existing tests with pytest instead of nosetests
* be able to run individual tests - that is currently tricky because
nosetest and/or the way our tests are structured
Can you give some examples here?
When you know the names of the test, for example from a previous run,
you can simply run something like:
nosetests
kallithea/tests/functional/test_admin_repos.py:TestAdminReposControllerGIT.test_create_non_ascii
I have not been able to reliably found a way to construct such magic
strings by looking at a test failure output ... but I assume there is a
way. Please consider updating docs/contributing.rst with some hints ...
and perhaps also move some prose away from kallithea/tests/__init__.py .
* test coverage profiling ... and eventually tests for the areas without
coverage
* pull requests is one area with bad coverage ... and it is tricky because
we want tests for many different repo scenarios with common and special
cases
I would really like more tests in this area too
Absolutely. I don't have much experience with thorough testing or TDD
but it do not seem feasible with the current test framework ... or with
my current level of familiarity with it.
/Mads
_______________________________________________
kallithea-general mailing list
[email protected]
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
