On Sun, Jul 24, 2011 at 4:59 PM, Vladimir Perić <[email protected]> wrote: > On Sun, Jul 24, 2011 at 11:16 PM, Vinzent Steinberg > <[email protected]> wrote: >> On Jul 24, 4:45 pm, Tom Bachmann <[email protected]> wrote: >>> That would be one possibility; defaulting to a specific, tested seed in >>> release tests. >>> >>> On the other hand my tests have not failed in the last >300 runs on >>> jenkins (since I pushed the last fix). This does not mean too much of >>> course. >> >> I think our tests should be determined by default, there should be no >> randomness, because it makes it impossible to reproduce failures. (If >> the test runner printed the seed and it was possible to pass the seed >> to the test runner, this would be something else, but this is >> currently not the case.) > > Actually, it does print the seed now and you can pass one with > "--seed=123". This was added in commit 9210842: Implement randomised > testing. > >> >> So I suggest we make random tests optional for now. Having random >> tests by default needs to be discussed IMHO. > > A quick search comes up with this issue, maybe we should discuss it > there (or on the list, of course): > http://code.google.com/p/sympy/issues/detail?id=2281
Yes, reproducibility was a necessity, and I wouldn't have pushed in any random test code without it. I think that random tests can complement regular tests, because they can essentially increase coverage in a way that is impossible otherwise. Of course, it's very important that they are 100% reproducible, meaning that it should print a seed that can later be entered, and this seed should be universally used for all random code. Actually, I'm not really convinced that the random tests in Tom's new code are doing a good job of "complementing" regular tests. If I remember correctly, there were many cases where random tests were the only tests. Perhaps the others here who are more critical of random testing could do a quick audit of the random tests (see any test added by Tom recently in the git history) to see how well this holds. By the way, Tom, was the fix for the test failures included in the 0.7.1 branch? Aaron Meurer > >> >> Vinzent >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/sympy?hl=en. >> >> > > > > -- > Vladimir Perić > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sympy?hl=en. > > -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
