Ben Klein wrote:
2009/4/13 chris ahrendt <[email protected]>:
Is there a guide documenting what each test is supposed to do etc?
Source code.
Before you say that's an unacceptable answer, the sheer number of test
cases (especially considering those that keep getting added) would
make it impractical to the point of impossibility to maintain proper
documentation on each test. And when the tests are only intended for
developers anyway, the source code is perfectly suitable as
documentation. The tests are relatively simple, and it's clear which
case is being test by the use of ok().
I run the test set to see where the current version of wine is on any of my
particular hardware.
Then you should learn to interpret the ERRs correctly.
I am not allowed to develop wine due to NDA's , and so forth I am under. And
have discussed
with the Alexandre and Dan concerning these NDA's and what I can do to help.
That does not make any sense... There has to be a consistent way for
developers and users
to report or work on issues..
It's called bugzilla.
I believe you may be missing my point Ben. By consistent I mean an ERR means
this and only means this. Warning, info etc...
As far as I'm concerned, that already exists, even if it's not written
explicitly and finitely in the dev guide. ERRs caused by the test
suite can be ignored as long as the tests run correctly. By the same
logic, ERRs caused by real applications can be ignored as long as the
apps still run correctly. In the former case, there is no motivation
for the devs to fix them. In the latter, there is minimal motivation,
and the devs have bigger problems to worry about than things that
aren't actually *breaking* anything at the moment.
From what I have been following here and also seeing in the code an error
doesn't always follow the
coding standard set forth in the developers guide.
I don't think you understand the nature of the test suite. There are
some tests that are 100% valid logic and are expected to create no
ERRs or WARNs. These are called positive cases. There are some tests
that are not 100% valid logic, and do things wrong deliberately, and
are expected to create both ERRs and WARNs. That's an "and" there, not
an "or". Both ERRs and WARNs. This cannot change.
and like Vincent says the use has gotten a little skewed.
Or been misinterpreted ... though a review of the developers' guide
description of ERR wouldn't hurt.
I would agree it may be... and maybe a review is in order..
So who wants to volunteer? :)
Not I =)
2009/4/13 chris ahrendt <[email protected]>:
An Excellent point was mad in one of the bugs I reported :
Comment #25 from Austin English <[email protected]>
The problem is that this isn't a 'normal' application doing weird things.
It's our testsuite, which does some _really_ strange stuff, e.g., lots of
corner case testing.
Our implementation code, however, knows this is weird, and prints an error.
The problem is that there's no way to tell, e.g, dlls/mshtml/htmldoc.c that
what's
being tested is dlls/mshtml/tests/doc.c, rather than foobar.exe. It's a bit
misleading, which makes it a bit harder for users.
This is EXACTLY what I've been saying. You ignore me, and listen to
Austin. Absolutely fantastic.
not AT All ben =D I do and have understood that there are positive and
negative test cases.
The point I guess I am trying to make is in the test cases where there
is an expected failure just
print something out saying this is expected. Not Complicated at all.
So, there has to be a happy middle in this...
why not have in the test suite some printf or the like that says to the
effect that err output is to be expected or the like.. Most users if they
see ERROR on the screen
be it in a test case or an application will think its a bug and not just
ignore it.
Most users DON'T RUN THE TEST SUITE. It's not useful for users, it's
ONLY useful for developers, and developers know how to interpret these
ERRs correctly. You don't.
Thanks for the attack ben... again you miss the point.. alot more users
do run the tests than you may think.
People evaluating the code for use in their enterprise will run the test
sets.. and if they don't complete or run
clean than they don't implement.
C
