This took me a little bit to figure out what was going on... so to save someone else some time: the problem is that the summary logic branches into two -- individual results, and subtest results. For the subtest results, the "overall" test result isn't kept anywhere. So the fact that the test crashed is completely lost, since the individual subtests completed.
The reported data is accurate -- those subtests did all pass; however it's incomplete. Perhaps if the test result is a crash, and there are subtests, then create a new "fake" subtest that shows that the overall test actually crashed? -ilia On Mon, Apr 7, 2014 at 7:27 AM, Marek Olšák <[email protected]> wrote: > Hi, > > There is a serious issue with piglit that crashing tests just > disappear from the HTML summary. Let's say I apply this: > > diff --git a/tests/fbo/fbo-generatemipmap-formats.c > b/tests/fbo/fbo-generatemipmap-formats.c > index 666030f..0d4a06b 100644 > --- a/tests/fbo/fbo-generatemipmap-formats.c > +++ b/tests/fbo/fbo-generatemipmap-formats.c > @@ -431,6 +431,8 @@ test_format(const struct format_desc *format, > GLenum basetype) > GLboolean pass = GL_TRUE; > > printf("Testing %s%s\n", format->name, tex_width == 256 ? "" : > " (NPOT)"); > + if (format->internalformat == GL_RG16) > + *(int*)NULL = 0; > tex = create_tex(format->internalformat, format->base_internal_format, > basetype); > > Then, the output of the test is: > > $ bin/fbo-generatemipmap-formats GL_ARB_texture_rg -auto > Using test set: GL_ARB_texture_rg > Testing GL_R8 > PIGLIT:subtest {'GL_R8' : 'pass'} > Testing GL_R16 > PIGLIT:subtest {'GL_R16' : 'pass'} > Testing GL_RG > PIGLIT:subtest {'GL_RG' : 'pass'} > Testing GL_RG8 > PIGLIT:subtest {'GL_RG8' : 'pass'} > Testing GL_RG16 > Segmentation fault (core dumped) > > Let's run piglit-run.py with "-t ARB_texture_rg/fbo-generatemipmap-formats$ > -v": > > running :: spec/ARB_texture_rg/fbo-generatemipmap-formats > crash :: spec/ARB_texture_rg/fbo-generatemipmap-formats > [1/1] crash: 1 > > Looks good, right? But there is no crash in the HTML summary: > > http://people.freedesktop.org/~mareko/summary/ > > Do you have any idea how to fix this? > > Marek > _______________________________________________ > Piglit mailing list > [email protected] > http://lists.freedesktop.org/mailman/listinfo/piglit _______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
