On 05/08/13 14:16, Andrew Gregory wrote: > On 08/05/13 at 10:52am, Allan McRae wrote: >> On 02/08/13 22:34, Andrew Gregory wrote: >>> This patchset converts the output of all of our tests to tap [1] and fully >>> integrates them with automake so that tests can be run in parallel with >>> `make >>> check`. The test suite may also be run with other test harnesses such as >>> perl's prove which can do such interesting things as remember which tests >>> failed and run only those on subsequent invocations. The documentation for >>> integrating tests is here [2]. >>> >>> [1] http://podwiki.hexten.net/TAP/TAP.html?page=TAP >>> [2] >>> http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html >>> >> >> Have you any ideas on how to fix the "unexpected" pass on the time test >> for x86_64 to not have the test suite return non-zero? I believe this >> is essential. >> >> Allan > > I think that "unexpected" passes are rightly considered failures. The test > should reflect what we actually expect to happen. We should either update the > test so that it succeeds or fails uniformly on all systems or set > expectfailure > only on systems where we actually expect it to fail. Personally, I would > prefer that the test use the maximum values that the testing system could be > expected to support and unset expectfailure, but the easier solution is to > just > set expectfailure only on 32 bit systems.
Setting expected failure on 32bit systems would actually be my preferred solution in this case. Can our test suite handle that?
