On Sun, Mar 29, 2020 at 11:45 PM Dan Eble <[email protected]> wrote:
>
> On Mar 29, 2020, at 17:39, Han-Wen Nienhuys <[email protected]> wrote:
> >> test-output-distance was removed on the grounds that the self-test
> >> serves the same purpose, but I don't see how it does.
> >
> > Could you elaborate? What failure scenario are you worried about?
>
> My question is, how does the self-test "verify that the regression tests
> have, in fact, run"? I don't see how it could do more than verify that
> itself has run.
How does test-output-distance verify that the regression tests have run?
The verification doesn't come from the test file. It comes from the
fact that someone is looking at the test output.
The self-test gives us assurance that the test result index.html file
is making sense. You can verify that the tests were run by checking
that the index.html file is there.
For further context, I am interested in automating our test setup
further: I want to have CI that can post feedback ("test failed!") on
code reviews automatically. test-output-distance is a test that always
fails, which makes that more complicated.
--
Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen
