On 6/25/20 7:53 PM, Jacob Bachmeyer wrote:
> The original rationale for using UNRESOLVED here was that DejaGnu
> converts a test result to UNRESOLVED if too many errors or warnings were
> produced. The DejaGnu manual indicates (section "A POSIX Compliant Test
> Framework") that UNRESOLVED is correct for a test where execution was
> interrupted or was set up incorrectly. UNTESTED is specifically listed
> as a placeholder for an as-yet-unwritten testcase. A "typical" GDB
> testsuite run has almost a hundred UNTESTED results but (without this
> patch) zero UNRESOLVED results.
>
> To me, this seems that UNRESOLVED is correct here, or that the manual
> has an error.
So having written both, there is always the chance I've interpreted
things differently. If the test is interrupted, or has errors/warnings,
like timing problems for example, then it's UNRESOLVED. But a bug in Tcl
code enough to trigger unknown is UNTESTED, as the test never really
ran. I'd go with which definition the toolchain teams prefer, as this
effects validation testruns. I could go either way.
I don't know if POSIX 1003.3 is still considered a standard, that was
a long time ago. The testsuites were primarily focused on XOpen and
POSIX conformance tests, not toolchains. I was
on the standards committee, so used them as a model cause they seemed a
good standard. But the details of the interpretation can now be whatever
we want it to be. We're probably the only ones left running TET
compliant testsuites anyway. :-)
Anyway, whether a test run aborts on an error or not is not defined,
so it's up to us to decide.
- rob -
_______________________________________________
Bug-dejagnu mailing list
Bug-dejagnu@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-dejagnu
