Andreas Mohr wrote:
> I guess we really should change our development
model from trying tons of
> programs to *systematically* testing functions and
Windows mechanisms now.
> If we can show everyone where stuff is failing, it
might be a lot easier
> to attract new people.
I *completely* support this idea. Benefits of such
test suite are enormous. Existing developers can
contribute a lot by adding test snippets for the
functions they create. Now they create such snippets
anyway and throw them away.
> I attached a preview of the posting I intend to post
on *tons* of Windows
> devel newsgroups ("Call For Volunteers"). That way
we might actually get
> hold of hundreds of Windows developers helping us
implement a complete
> test suite (complete tests of up to 12000 Windows
functions).
> Not to mention the additional PR we might get out of
this...
Lets ask for help only after the suite structure is
more/less defined and we'll be able to give people
something to work on.
Comments:
- Don't want to reinvent the weel. Is there any
existing test suite framework we can use? Sorry, I
can't suggest any for C but I'm very impressed with
JUnit in Java. It is even Ok if the framework is GPLed
or LGPLed - I don't think any company will make
buziness based on the test suite.
- I /personally/ prefer CL interface only for such
suite
- it would be better if the suite print summary
information and information about failed tests only
- make the test suite more "visible" for existing
developers. Ask them to run the test suite before
submitting a patch?
- I think the suite test will consist from a few
separate applications because different tests may have
different requirements to GUI configuration,
processes, etc. We need a way to run all the
applications in one batch.
- define variable which indicates whether the suite
runs under Wine. Such indicator can be used for Wine
"white-box" testing.
- it would be greate to have functionality to support
output comparison? For some functionality it is easier
to write tests to compare output instead of doing
explicit checks (e.g. tests, involving a few
processes). The output can be redirected to file and
files compared. If we use files we need to store files
for Wine and a few versions of Windows :-(
- the suite applications size will be pretty big. Is
it better to move it to separate CVS tree?
- what about running the suite weekly (or daily)
automatically and publishing the results to
wine-devel?
- most developers on this list have access to one
version of Windows. Is it difficult to create "testing
farm" with remote access to a few versions of windows?
This would help developers to test their code on a few
platforms. Existing environments in the companies,
involved in the project can be used.
- I remember long time ago there was a post on
wine-devel about using Perl or Perl-like language for
unit testing.
What is current status of that project?
Thanks,
Andriy Palamarchuk
__________________________________________________
Do You Yahoo!?
Send your FREE holiday greetings online!
http://greetings.yahoo.com
