Hi I already mentioned this over IRC, but I have written a proto-type test suite for britney. While this is a "good thing", it sounds a lot better than it actually is.
The primary issue with this suite is that you have to hand-craft *all* the data needed, which is a pain. I strongly suspect that the long term implications of this is that tests will end up being very hard to understand and that people will generally avoid writing tests. The "second" problem is that the test-suite only checks the heidi result, so it is cannot tell the difference between "pkg with rc bugs" and "migrating pkgX would break pkgY". To fix the primary issue, I am considering to create a small language to describe various parts of the data (packages, their relations, their RC bugs, etc) and then generate the "real" data from that. I am not quite ready to start on that yet, so I will follow up on this "soon". For now, feel free to give the test suite a spin. Either clone it from [1] or run it from franck [2]. If you clone it, have a look at the README (there are no "-h" or "--help"). In case you want to debug a test, you can read the results from the "test-out/$test" dir. This will be the state after the britney run. All of britney's stdout output will be available in "test-out/$test/log.txt" and the conf file will be "test-out/$test/britney.conf". The expected testing state is "test-out/$test/expected" (it is a "simplified" heidi file) and is compared to the HeidiResult file. The comparision is not affected by ordering of those files. ~Niels [1] ssh://release.debian.org/home/nthykier/britney-tests.git http://release.debian.org/~nthykier/britney-tests.git/ Have a look at the README. [2] $ ~nthykier/bin/run-britney-tests \ /srv/release.debian.org/britney/code/b2/britney.py \ ~nthykier/britney-tests/t \ test-out Running basic-with-rc-reduction...done Running basic-uninstallable...done Running basic-with-rc...done Running basic...done Summery: Ran 4 tests Failed tests: 0 Note: The last argument ("test-out") must not exist and must be "creatable". -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected] Archive: http://lists.debian.org/[email protected]
