> Randomness makes it difficult to correlate a failure to the commit that made
> the test to fail (as was pointed out earlier in the discussion). If each
> execution path is different, it may very well be that a failure you
> experience is introduced several commits ago, so it may not be your fault.

This is true only to a certain degree. If you  don't randomize all you
do is essentially run a fixed scenario. This protects you against a
regression in this particular state, but it doesn't help in
discovering new corner cases or environment quirks, which would be
prohibitive to run as a full Cartesian product of all possibilities.
So there is a tradeoff here and most folks in this project have agreed
to it. If you look at how many problems randomization have helped
discover I think it's a good tradeoff.

Finally: your scenario can be actually reproduced with ease. Run the
tests with a fixed seed before you apply a patch and after you apply
it... if there is no regression you can assume your patch is fine (but
it doesn't mean it won't fail later on on a different seed, which
nobody will blame you for).


To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to