Am Montag, 2. November 2015 um 08:36:05, schrieb Guenter Milde <mi...@users.sf.net> > Dear Scott and Kornel, > > On 2015-11-02, Scott Kostyshak wrote: > > On Sun, Nov 01, 2015 at 10:36:17PM +0100, Kornel Benko wrote: > >> Am Sonntag, 1. November 2015 um 21:27:11, schrieb Guenter Milde > >> <mi...@users.sf.net> > > >> > Could we introduce a new status "suspended" - meaning just skip the > >> > test until a known bug is solved as we know the result is > >> > insignificant until then? > > > As Kornel mentions, we have "ignoredTests" for this. > > Not exactly, see below. > > >> We already have such (specified in file "ignoredTests"). But as this > >> tests are never executed, nobody cares for them anymore. > >> The tests here are such, that we know, we never resolve them. > > > I agree. We should not use ignored tests for temporary issues. Once we > > put something in ignoredTests chances are strong that we will forget > > about them for a long time. > > This is why I said "suspended": > > Suggestion: > > Specified in file "suspendedTests") with the reason for suspending > (bug report, commit that turned hidden problems into export failure, ...) > > These tests are normally skipped, but they are not forgotten. > > The tests here are such, that we know, we can resolve them but their > failure is a minor issue that can be postponed (comparable to enhancement > requests in Trac). > > Suspending instead of reverting also frees us from the need to reassess > them if a change in the "real life documents" or a fix makes them pass > again. Instead, they could/should be revised at a time we have fixed > major known problems and no release pressure... > > > Candidates for "suspended" tests are > > * "fragile" documents (many packages, ERT, heavy preamble, ...), > * "fragile" export routes (XeTeX/LuaTeX with TeX fonts), > * non-default export routes > > and especially combinations of these. > > > Günter >
OK, here is my suggestion
1.) We add the appropriate tests into revertedTests
The content in suspendedTests may be (in our case) e.g.
pdf4_texF
1.a)
If a test is to be inverted, we check suspendedTest, and if
there, we ignore the testcase.
or
1.b)
Such a test may became a label "suspended" instead of "export",
so that
'ctest -L export' will be clean, but we also have the
possibility to
use 'ctest -L suspended'.
This does neither effect non-inverted nor ignored.
I prefer 1.b.
Kornel
signature.asc
Description: This is a digitally signed message part.
