That's a full step ahead of my suggestion. I just want the thing in a shared place so that I can run it when doing dodgy things to code I don't understand... if there's some automated process running it, that's just gravy on top.
On Tue, Nov 10, 2015 at 1:26 AM, Dylan Baker <[email protected]> wrote: > Here's my idea. > > We add ubo-fuzzer as a profile. We generate each test file on demand, and > write it out to a temporary file before running it. Then if the test fails > we can record the test into the results file for the developer to come back > and look at later to decide if it's worth keeping. Does this sound like a > reasonable approach? > > On Mon, Nov 9, 2015 at 5:08 PM, Ian Romanick <[email protected]> wrote: >> >> On 11/09/2015 05:19 AM, Ilia Mirkin wrote: >> > On Thu, Sep 25, 2014 at 4:39 PM, Ian Romanick <[email protected]> >> > wrote: >> >> On 09/24/2014 09:47 AM, Ian Romanick wrote: >> >>> So, here it is. Finally. >> >>> >> >>> The first two patches provide the infrastructure for generating >> >>> randomized UBO tests. I think these are pretty solid, but there are >> >>> probably ways to impove the Python, etc. >> >>> >> >>> The remaining three patches are examples of ways the infrastructure >> >>> can >> >>> be used. Here is where I am not sure what we should do. I know that >> >>> we >> >>> don't want to make the "forever" test in patch 4 part of regular >> >>> piglit >> >>> runs. However, it has found a LOT of bugs in EVERY OpenGL driver that >> >>> I >> >>> have tested. >> >>> >> >>> I'm also unsure about the random tests generated by patch 3. Do we >> >>> want >> >>> actual random tests in regular piglit runs? What do we do for tests >> >>> for >> >>> GLSL 1.40? Generate the "same" tests, but use #version 140 instead of >> >>> #extension? >> >>> >> >>> In any case, I know that folks are hard at work on fp64 support, so >> >>> using the various random runners here should help that effort. Sorry >> >>> for all the delays. >> >>> >> >>> One last thing... I'm presenting a bunch of information about this >> >>> work >> >>> at XDC in a couple weeks. Maybe we want to wait to hammer out the >> >>> more >> >>> difficult details until then. Dunno. >> >> >> >> I've pushed updated version of this series to the ubo-lolz branch of my >> >> fd.o piglit repo. >> > >> > It looks like this didn't end up going anywhere... on several >> > occasions I've either used this script (like for fp64), or recommended >> > it to others (like for ssbo, and will do so for ARB_enhanced_layouts >> > when that conversation comes up). >> > >> > I think it'd benefit greatly from being in a shared and updated >> > location as features are added, bugs are fixed, etc. However running >> > it as part of piglit may not be a great idea. Perhaps we can find a >> > place in the repo where we can store it? Or maybe even a different >> > repo? >> > >> > How about tests/fuzzing in piglit? Any objections? >> >> Having it actually live somewhere is a good idea. There are definitely >> some bugs in it... and some of the tests that failed on other >> implementations may be expecting things the spec doesn't allow. I need >> to dig back through my e-mail, but some guys from NVIDIA had convinced >> me that there was something wrong... but I don't recall what. >> >> When I presented this at XDC in 2014, I think the consensus was that >> tests that actually found a bug should be added to the "right" place in >> the repo, but we don't want to run 47,000,000 random tests on regular >> piglit runs. Someone should have a system somewhere that just runs >> these (and other) random, fuzzing tests 24/7. >> >> > Cheers, >> > >> > -ilia >> >> _______________________________________________ >> Piglit mailing list >> [email protected] >> http://lists.freedesktop.org/mailman/listinfo/piglit > > _______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
