On 08/12/13 at 09:10pm, Allan McRae wrote: > On 05/08/13 14:47, Andrew Gregory wrote: > > On 08/05/13 at 02:18pm, Allan McRae wrote: > >> On 05/08/13 14:16, Andrew Gregory wrote: > >>> On 08/05/13 at 10:52am, Allan McRae wrote: > >>>> On 02/08/13 22:34, Andrew Gregory wrote: > >>>>> This patchset converts the output of all of our tests to tap [1] and > >>>>> fully > >>>>> integrates them with automake so that tests can be run in parallel with > >>>>> `make > >>>>> check`. The test suite may also be run with other test harnesses such > >>>>> as > >>>>> perl's prove which can do such interesting things as remember which > >>>>> tests > >>>>> failed and run only those on subsequent invocations. The documentation > >>>>> for > >>>>> integrating tests is here [2]. > >>>>> > >>>>> [1] http://podwiki.hexten.net/TAP/TAP.html?page=TAP > >>>>> [2] > >>>>> http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html > >>>>> > >>>> > >>>> Have you any ideas on how to fix the "unexpected" pass on the time test > >>>> for x86_64 to not have the test suite return non-zero? I believe this > >>>> is essential. > >>>> > >>>> Allan > >>> > >>> I think that "unexpected" passes are rightly considered failures. The > >>> test > >>> should reflect what we actually expect to happen. We should either > >>> update the > >>> test so that it succeeds or fails uniformly on all systems or set > >>> expectfailure > >>> only on systems where we actually expect it to fail. Personally, I would > >>> prefer that the test use the maximum values that the testing system could > >>> be > >>> expected to support and unset expectfailure, but the easier solution is > >>> to just > >>> set expectfailure only on 32 bit systems. > >> > >> Setting expected failure on 32bit systems would actually be my preferred > >> solution in this case. Can our test suite handle that? > > > > I don't have any 32-bit systems readily available to test it at the moment, > > but > > checking either platform.architecture [1] or sys.maxsize [2] should be > > sufficient. > > > > [1] http://docs.python.org/2/library/platform.html#platform.architecture > > [2] http://docs.python.org/2/library/sys.html#sys.maxsize > > > > I guess I can test this in a chroot (or you could...). > > It also looks like .gitignore needs updated: > > # test-suite.log > # test/pacman/tests/clean001.log > # test/pacman/tests/clean001.trs > # test/pacman/tests/clean002.log > # test/pacman/tests/clean002.trs > # test/pacman/tests/clean003.log > # test/pacman/tests/clean003.trs > # test/pacman/tests/clean004.log > # test/pacman/tests/clean004.trs > # test/pacman/tests/clean005.log
Erm, I did update .gitignore... Did you by any chance run make check with these patches then switch to a different branch? Otherwise I have no idea why those would show up. apg
