On Wed, 6 Jun 2018, Travis Scrimshaw wrote:
Heisenfailures are very difficult to debug: One run, doctests fail. You make
a small tweak (one that changes the random seed), which you think fixes the
issue, and then tests pass. However, if it is a random test, then you have
no idea if you have fixed the problem or not.
True, we should somehow save the random seed used and give it with a
failure notice.
I have done something to help random testing:
sage: from sage.tests.finite_poset import test_finite_lattice
sage: L = posets.RandomLattice(10, 0.98)
sage: test_finite_lattice(L) is None # Long time
True
We could check, for example, that matrix AB is invertible iff both A and B
are, and so on.
* * *
Another thing, should we try to classify bugs we found? I mean that could
we get insight of places to look for possible other bugs?
--
Jori Mäntysalo
