* Stefano Lattarini wrote on Thu, Jun 23, 2011 at 09:52:55AM CEST:
> On Wednesday 22 June 2011, Ralf Wildenhues wrote:
> > Also, have you thought about a quoting mechanism so that test authors do
> > not have to think about breaking the test driver mechanism accidentally?
> > That would seem more robust to me.
> >
> I don't understand. A quoting mechanism for own our testsuite is already
> offered by the new `dump_log_files' function (and the new maintainer
> checks try to unsure that this function is used when required). OTOH,
> the authors of third-party test drivers *must* allow the direct writing
> of `:test-result:' directives in the driver output -- this is the only
> way to declare test results that are understood and accounted for by the
> new Automake test harness.
So, we are doing inband signaling.
But Automake's own testsuite is not a TAP or a subunit one. Thus it
does not need to do inband-signaling. Right?
The reason I'm harping on this is that, if all existing users of
Automake test drivers are updated to use the same kind of inband
signaling, then they all are liable to similar bad signals. That
is a potential source for regression.
What do TAP and subunit do to protect against this? Quote normal
log output? Like email, prepend '> ' to lines? Please explain.
> > Also, most of your uses of "useless" are actually superfluous: either
> > you go on to explain why something is not actually useless after all
> > (in which case there seems no point in calling it useless in the first
> > place), or it is useless, in which case why have it?
> >
> What about "extra quotes" instead? I'll use that if you don't object.
Well, just explain the reason /for/ something. For example:
+# FIXME: the useless quoting below is to please maintainer-check.
+env VERBOSE'='yes $MAKE -e check > stdout && { cat stdout; Exit 1; }
You can just write:
# Quote for maintainer-check (FIXME).
(I'd even leave out the FIXME, but I can see why you'd like that.)
It has all information your comment gave.
> > No time for more detailed review in the short span allowed for feedback.
> > Why BTW?
> >
> Because without this patch I'm experiencing spurious failures in the
> testsuite. OK, I can visually scan the output of "make check" and
> see that there is no unexpected "foo.test: FAIL" line in there, and
> thus conclude that the failure is really only spurious. But this is
> cumbersome to say the least, and surely error-prone.
> > You can always git rebase later, and it's not like this is
> > security relevant
> >
> This is true.
>
> > or needs to be rolled out this week.
> >
> I disagree: Ggven what I said above, I'd rather apply the patch today
> (I'll wait until this evening or tomorrow morning though, to give you
> enough time to answer).
None of your reasons make sense to me. Of course, you should fix this
in your tree in order to not see spurious errors any more. But there is
absolutely no reason to rush with a push, or not to go back and update
the patch after I review it in three days. Please give a reason against
*that*.
Thanks,
Ralf
