Re: [RESEND] Sync (47): shlwapi tests
On Sun, 24 Aug 2003, Alexandre Julliard wrote: [...] I disagree, as far as I'm concerned they are definitely regression tests. If we only wanted to check conformance then it would be OK for tests to fail everywhere we are not compatible; it's because the tests must be usable to find regressions that we cannot have failing ones in the tree. And that's why we have the todo_wine macro to mark tests that are expected to fail. Hmmm, when we use todo_wine the test itself, i.e. the ok() statement, is still failing. It's just that it's in a block where we expect failures and complain about successes. Bah, ok, my previous message was not clearg. But I still consider these tests to be primarily conformance tests because that's much more useful than tests that verify that we preserve the old bugs. Fortunately with the todo_wine mechanism they are also pretty useful for regression testing. -- Francois Gouget [EMAIL PROTECTED]http://fgouget.free.fr/ Linux: the choice of a GNU generation
Re: [RESEND] Sync (47): shlwapi tests
Francois Gouget [EMAIL PROTECTED] writes: On Sun, 24 Aug 2003, Jon Griffiths wrote: [...] Note that a regression is something that used to work, and now doesn't. So any regression testing should only report failures for tests that used to work and now don't. New tests that fail, or those that have never succeeded, _aren't_ regressions, and shouldn't be marked These are not regression tests but conformance tests. They should report anything that does not conform to the Windows behavior as an error. I disagree, as far as I'm concerned they are definitely regression tests. If we only wanted to check conformance then it would be OK for tests to fail everywhere we are not compatible; it's because the tests must be usable to find regressions that we cannot have failing ones in the tree. And that's why we have the todo_wine macro to mark tests that are expected to fail. -- Alexandre Julliard [EMAIL PROTECTED]
Re: [RESEND] Sync (47): shlwapi tests
Jon Griffiths [EMAIL PROTECTED] writes: Any problems with this one? You said yourself that the some of the tests are currently failing; I cannot commit tests that fail, that would make the regression test suite useless. -- Alexandre Julliard [EMAIL PROTECTED]
Re: [RESEND] Sync (47): shlwapi tests
Alexandre == Alexandre Julliard [EMAIL PROTECTED] writes: Alexandre Jon Griffiths [EMAIL PROTECTED] writes: Any problems with this one? Alexandre You said yourself that the some of the tests are currently Alexandre failing; I cannot commit tests that fail, that would make the Alexandre regression test suite useless. But we have a keyword for tests that we expect to fail. So changing the expected result to failing should make the patch appyable. Bye -- Uwe Bonnes[EMAIL PROTECTED] Institut fuer Kernphysik Schlossgartenstrasse 9 64289 Darmstadt - Tel. 06151 162516 Fax. 06151 164321 --
Re: [RESEND] Sync (47): shlwapi tests
On Sun, 24 Aug 2003, Jon Griffiths wrote: [...] Note that a regression is something that used to work, and now doesn't. So any regression testing should only report failures for tests that used to work and now don't. New tests that fail, or those that have never succeeded, _aren't_ regressions, and shouldn't be marked These are not regression tests but conformance tests. They should report anything that does not conform to the Windows behavior as an error. Which just hapens to be what they do. -- Francois Gouget [EMAIL PROTECTED]http://fgouget.free.fr/ Good judgment comes from experience, and experience comes from bad judgment -- Barry LePatner
