On 2011-10-18 14:56, Mehdi Dogguy wrote: > On 15/10/2011 12:20, Niels Thykier wrote: >> >> The primary issue with this suite is that you have to hand-craft *all* >> the data needed, which is a pain. I strongly suspect that the long >> term implications of this is that tests will end up being very hard to >> understand and that people will generally avoid writing tests. >> > > I guess that's why you added "description" files for (almost) each test? > I think it is enough to not forget why we added $some test. >
Yes, at least for the more complicated ones. Still I do not like the amount of copy-waste(-errors) I have do(/fix) for each test, but it does sort of work... :) >> The "second" problem is that the test-suite only checks the heidi >> result, so it is cannot tell the difference between "pkg with rc bugs" >> and "migrating pkgX would break pkgY". >> > > But we do have (thanks to you) minimal tests for those two situations. So, > somehow, they are distinguishable. But, I agree it could be a problem with > more complicated tests. > >> >> For now, feel free to give the test suite a spin. Either clone it from >> [1] or run it from franck [2]. If you clone it, have a look at the >> README (there are no "-h" or "--help"). >> > > I didn't find time to play with it enough yet but I find this project very > nice. It helps (some of) us to understand better what britney does in some > specific situations, serves as a regression test base and tells new briney > implementations what they should implement (at least). > Indeed, :) > I have some minor patches: > > #1) fixes a typo. > #2) makes run-single-test exit with a non zero value when test fails > (the change is not pretty but kinda wfm™) Ideally, run-single-test should > also tell us why the test failed (maybe by `cat diff`?). Applied > #) a bug in britney fixed by Adam recently (that requires disabling > compatible mode). But the test passes using the buggy britney… so maybe I > did something wrong. > > Kind regards, > And I applied your corrected version. Thanks for the contributions :) ~Niels -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected] Archive: http://lists.debian.org/[email protected]
