--- Alexandre Julliard <[EMAIL PROTECTED]> wrote: > Andriy Palamarchuk <[EMAIL PROTECTED]> writes: > > > Always succeed *under Windows*. Do you really, > really, > > really think all the tests will succeed under Wine > > from day 1 and we will be able to maintain them > > failure-free? > > Absolutely. There's a very simple way of enforcing > that: I'm not > comitting anything that causes make test to fail. > > > The value of unit tests is exactly in failures! > The > > more failures of unit tests we have - the better > test > > developers do their work. > > Well, I guess it's a philosophical point, but for me > the value of the > tests is for regression testing. If you allow the > tests to fail you'll > pretty soon have 90% of the tests fail somewhere, > and this is > completely useless except maybe as a list of things > we still have to > do. While if the tests usually succeed, as soon as > something fails you > know there's a regression and you have to fix it.
As Francois mention, this is why TODO tests exists. Even without TODO tests it is not wise to reject a perfectly correct patch from a Windows developer who even does not have Wine. Without TODO construction compromise is still possible. E.g we can use conditional compilation or execution to select only tests which succeed for finding regressions. I believe there will be somebody, interested in mining "dirty" tests. Other option is to store failture lists and compare them from time to time in a search of regressions. > What you can do with my make test patch is run make > test -k first, let > it go through everything, and then run make test > again and it will > only run the tests that failed the first time, or > that have been > modified. This is a major gain. It could certainly > be done some other > way too without using make, but AFAICS your test > harness would force > to run all tests all the time (or to disable them > manually). This is > not acceptable once you have a large number of > tests. I don't see poing in selecting subsets of tests. From experience - even with pretty big number of tests it does not take long time to execute them. > OTOH a Wine test suite can happen (I hope) because > this is something > Wine developers need when they write code, so there > is at least some > motivation for them to write tests. Exactly! I do not want to spend resources of Wine project on some "nice documentation". On the contrary, goal of this change is to invite external resources. Andriy Palamarchuk __________________________________________________ Do You Yahoo!? Send your FREE holiday greetings online! http://greetings.yahoo.com
