On Wed, Dec 19, 2007 at 05:58:35PM -0800, Brett Cannon wrote: -> On Dec 19, 2007 4:33 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote: -> > The bots are kicking-off so many false alarms that it is becoming difficult to tell whether a check-in genuinely broke a build. -> > -> > At the root of the problem is a number of tests in the test suite that randomly blow-up. I now tend to automatically dismiss failures in test_logging and test_threading for example. -> -> Yeah, certain tests need some TLC to make them more predictable and -> less prone to throw a failure because of some touch race condition or -> something on the machine was not available to make the test work. -> -> As I have stated on my blog, once I am done with importlib and the -> stdlib reorg I plan on working on dev docs and then attack the whole -> structure of the unit tests. -> -> But who knows when that will happen. =)
OK, but this isn't really a structural problem, right? This is non-determinism in certain tests ;) How broken are these tests? Can we point a 17-yr-old at them and tell them to fix 'em (yes think "GHOP")? (If the problem is reproducible on a 1-in-10 basis then I think the answer is "yes".) And are test_threading and test_logging the two that need the most work? Hmm, perhaps a good starting task would be to run the tests 10-100 times, in random order, on a single machine, to get a statistical picture of of the problem. cheers, --titus (always on the lookout for core => GHOP tasks) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
