On Sun, Mar 29, 2015 at 3:51 PM, Mads Kiilerich <[email protected]> wrote: > On 03/28/2015 08:54 PM, Thomas De Schampheleire wrote: >> >> On Sat, Mar 28, 2015 at 12:13 AM, Brianna Laugher >> <[email protected]> wrote: >>> >>> Well, I'm kind of making it up as I go, but I envisage something like >>> · initial familiarisation with project, discussion of aims (which we >>> have done a bit here and now on the wiki page) >>> · basics: pytest can run all tests, tests run without errors/failures, >>> CI server set up (I haven't seen one for Kallithea yet - is there >>> one?), probably with coverage - pytest helpers submit PRs, project >>> authors review and discuss >> >> I don't think there is a CI server currently. >> Mads: are you running one privately? > > > No, nothing continuous. I run the tests locally before deploying internally > or pushing upstream. That works ok. Right now I don't see a big need for CI; > I don't think it would solve a "real" problem.
The recent failures of two tests, caused by the Dulwich bump but only visible when you actually deploy this new Dulwich, could have been detected by a CI server (if they'd run 'python setup.py develop' each time, that is). But it's a cornercase, that's true. > > Long term it would be nice for Kallithea development to have test run > integrated with PRs, as one automatic and mandatory "review" criteria. Agreed. > > I think CI is more valuable for projects where multiple people are pushing > to the same branch frequently and where it not is feasible to test > everything before pushing (perhaps because it must be tested on multiple > platforms or use special hardware or just is too slow). > >>> Then you have a choice of focusing on existing tests or exploring >>> needs for new tests. >>> Existing tests: Progressively rewrite to be more pytesthonic. You >>> mentioned improving the speed and that is another definite area to >>> look at! >>> New tests: Add unit tests for low coverage areas. Figure out what >>> fixtures might be needed, especially for functional tests. >>> >>> I would suggest the latter could be especially useful for Kallithea as >>> you already mentioned fixtures for different vcs repos - sounds >>> perfect. >>> >>> As I put 3 helpers with Kallithea they are people with a variety of >>> pytest experience, so something like converting or writing new unit >>> tests is a good task for someone with less experience, and analysing >>> the speed and writing fixtures is a good task for someone with more. >>> >>> Like that is actually heaps for a months' effort, right? I get really >>> excited and I'm really bad at keeping the scope down. :) But that's >>> pretty much what I think - then if you/Kallithea would like a >>> different focus, or have other ideas, point the way! >> >> What you describe sounds good to me, but I definitely welcome more >> feedback from other Kallithea contributors/maintainers in this >> matter... > > > I can imagine some valuable milestones / deliverables: > > * be able to run the existing tests with pytest instead of nosetests > > * be able to run individual tests - that is currently tricky because > nosetest and/or the way our tests are structured Can you give some examples here? When you know the names of the test, for example from a previous run, you can simply run something like: nosetests kallithea/tests/functional/test_admin_repos.py:TestAdminReposControllerGIT.test_create_non_ascii > > * test coverage profiling ... and eventually tests for the areas without > coverage > > * pull requests is one area with bad coverage ... and it is tricky because > we want tests for many different repo scenarios with common and special > cases I would really like more tests in this area too, as we've seen already two bugs related to our different use cases: you use pull requests in one repo with different branches, and I use it with different repos but the same branch in each repo. Best regards, Thomas _______________________________________________ kallithea-general mailing list [email protected] http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
