On Mon, Jan 6, 2020 at 12:05 PM Julian Foad <julianf...@apache.org> wrote: > > Nathan Hartman wrote: > > I assume that's the one added in r1872121, > > test_mergeinfo_merge_random_non_validated_inputs(). > > > > I got a fail on it. > > Thanks for the report. I made a bad assumption in the test code. > > http://svn.apache.org/r1872388 should fix it.
Thank you. I have been thinking about the random input tests... My thoughts: Random input tests are not deterministic in the sense that one never knows in advance which inputs will be tested. While that's great for finding bugs that a deterministic test won't surface, I wonder if we should try to keep the regression test suite deterministic (as much as feasible) and move any random input testing to a separate "fuzz testing" program. Meanwhile, I'm *not* suggesting to remove the test from the regression test suite. Rather, I'm suggesting to make it deterministic by giving it a fixed list of test inputs. (The random inputs generator could be utilized to generate those inputs, even thousands of them if we'd like.) Thoughts? Nathan
