On Fri, Oct 09, 2015 at 10:33:35AM -0700, Dylan Baker wrote: > On Fri, Oct 09, 2015 at 09:44:01AM -0700, Mark Janes wrote: > > Jose Fonseca <[email protected]> writes: > > > > > On 09/10/15 01:21, [email protected] wrote: > > >> This series updates the junit backend to allow it to properly load junit > > >> and convert it back into piglit's internal representation, thus allowing > > >> it to be summarized using the piglit summary tools > > >> > > >> There is still some work that needs to be done beyond this, most of the > > >> platform metadata isn't stored yet and restored, but I have a plan for > > >> that. I have some other refactoring work that I think will make that > > >> easier, and I'd like to get there before landing that. > > >> > > >> This is enough to be able to compare junit and json results using the > > >> console and html summaries. > > >> > > >> There is a caveat here, and that's patch 3. To compare json and junit we > > >> need to be able to restore the names of the junit tests to *exactly* > > >> what they were before, and currently we don't have a way to reverse the > > >> '.' -> '_' conversion. My proposal is to change '.' into '___', which is > > >> very unlikely in a real test name (though we could change it to almost > > >> anything that would be unique). This may break some existing setup > > >> (Mark, I think this will probably break some of our expected fail/crash > > >> data). > > >> > > > > > > I don't object this, but instead of this brittle testname (de)munge, > > > have you considered/tried using an additional XML attribute with > > > unmodified piglit test name? I expect that Jenkins junit parser will > > > just outright ignore it. > > > > My recollection is that we found the junit parser to be strict: > > > > https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd > > > > > > > > Jose > > I don't know if we tried, It definately allows some not strictly valid > junit to be passed to it (IIRC we didn't originally pass the total > number of tests in, and that is required per the SPEC). > > If it will accept arbitrary tags that would be very nice. We might want > to guard them behind a --non-strict-junit switch (or inversely provide a > --strict-junit flag) in case that changes or someone whats to us a > consumer of junit that is strict. > > I think I'll give it a try and see what happens.
Mark is correct, I added an extra tag and jenkins ignored the result. So it is at least too strict for this use.
signature.asc
Description: PGP signature
_______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
