In article <b312f3e7-5c73-486e-925e-da8343963...@googlegroups.com>, Nick Mellor <thebalance...@gmail.com> wrote:
> Hi, > > I've got a unit test that will usually succeed but sometimes fails. An > occasional failure is expected and fine. It's failing all the time I want to > test for. > > What I want to test is "on average, there are the same number of males and > females in a sample, give or take 2%." > [...] > My question is: how would you run an identical test 5 times and pass the > group *as a whole* if only one or two iterations passed the test? Something > like: > > for n in range(5): > # self.assertAlmostEqual(...) > # if test passed: break > else: > self.fail() I would do something like: def do_test_body(): """Returns 1 if it passes, 0 if it fails""" results = [do_test() for n in range(number_of_trials)] self.assert(sum(results) > threshold) That's the simple part. The more complicated part is figuring out how many times to run the test and what an appropriate threshold is. For that, you need to talk to a statistician. -- http://mail.python.org/mailman/listinfo/python-list