My own personal machine could work although I've found in the past it can be a pain since I'm constantly installing and reconfiguring it for other tasks and it's more likely to cause other things to break. But it's certainly a possible option.
-- Michael Mior [email protected] Le mer. 12 sept. 2018 à 13:01, Julian Hyde <[email protected]> a écrit : > My intuition about fuzz testing is that since we are searching an > exponentially-sized search space in random order we will find 90% of the > bugs with the first 10% of the effort (or some similar power law). We > should burn a large amount of CPU on it when we first introduce it, and > thereafter burn a small amount each nightly test run. > > No need to ask INFRA for a VM. Just use your own personal machine > overnight. > > Julian > > > > On Sep 12, 2018, at 6:01 AM, Michael Mior <[email protected]> wrote: > > > > True, although 23 days over the lifetime of the project still isn't very > > much. Definitely better than nothing though. If we take a bit of a hit in > > CI runtime and catch some bugs, I'm for it :) > > > >> It would be great, however we need to have a fuzzer first :) > > > > My setup fuzzing the parser with afl seems to be working well, although > > it's quite slow and never found any crashes so probably not really worth > it. > > > > -- > > Michael Mior > > [email protected] > > > > > > Le mer. 12 sept. 2018 à 08:40, Vladimir Sitnikov < > > [email protected]> a écrit : > > > >>> My only concern is that may betoo short and unlikely to find any bugs > >> > >> Remember: each time it starts from a random point. > >> Apache Jenkins / Calcite-Master has 800+ builds now. > >> Travis / Calcite has 2300+ builds now. > >> > >> Just to clarify: current Travis configuration is 4 jobs (Java 8, 9, 10, > 11) > >> They take ~15 minutes to complete. > >> We can add one more job that would be dedicated to fuzz testing, and we > >> could configure it for 1..15 minutes. > >> > >> 2300 builds * 15 minutes is 23 days worth of fuzzing. > >> > >>> It seems like we could also possibly request a VM > >>> from INFRA to run fuzz testing full time > >> > >> It would be great, however we need to have a fuzzer first :) > >> > >> Vladimir > >> > >
