[Martin v. Löwis] > So what is your recipe: I don't have one. I personally always use -uall and -r, and then run the tests 8 times, w/ and w/o -O, under debug and release builds, and w/ and w/o deleting .py[co] files first. Because that last one almost never finds problems anymore, perhaps it would be good to stop bothering with it routinely (it really doesn't have potential to find a problem unless someone has been mucking with the marshaling of code objects, right?).
> Add -r to all buildbots? Sure. -r adds variety to testing at no cost (the same number of tests run, in the same pamount of time, with or without -r). > Only to those which have an 'a' in their name? Sorry, no idea what that means. > Only to every third build? Duplicating the number of builders? For -r, no. I'd always use -r (and always do anyway). > Same question for --with-pydebug. Combining this with -r would multiply > the number of builders by 4 already. I would much rather see a debug-build run than the current "with and without deleting .py[co] files first" variant. If the latter were dropped and the former were added, and -r were used all the time, the number of recipes wouldn't change. Testing time would increase, by the time to _do_ a debug build, and by the extra time a debug build test run requires. We should test with and without -O too, although that's another that rarely finds a problem. > I'm not keen on deciding this for myself. Somebody else please decide > for me. I don't know how hard it is to teach the system how to do something "not so often", and I expect that's an important unknown since I imagine that vastly increasing test time would discourage people from volunteering buildbot slaves. Since the most fruitful variations (IME) for finding code errors are using -r and running a debug build too, I'd change the current run-all-the-time recipes to: - Stop doing the second "without deleting .py[co]" run. - Do one run with a release build. - Do one run with a debug build. - Use -uall -r for both. If we know how to get something done "occasionally", then about once a week it would be prudent to also: - Try the "with and without deleting .py[co] files first" business. - Try with and without -O, Those last two choices cover 8 distinct modes, when paired with each other and with the "release versus debug build" choice. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
