Re: C testing framewok. Updated. SystemParametersInfo unit test.
Alexandre Julliard wrote: It's not enough to simply pass converted ASCII strings to the W functions, we have to test with real Unicode to check for lossy W-A-W conversions, surrogate handling, non-spacing chars, etc. Aren't all these requirements just additional to testing with converted ASCII strings? At least we'll have minimal tests of W functions without spending much time. One more advantage in creation TCHAR-portable code - we can eventually test with it multibyte character set support. We still can have specific A and W tests sections, enclosed in #ifdef _UNICODE. Andriy Palamarchuk __ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/
Re: C testing framewok. Updated. SystemParametersInfo unit test.
On Thu, 17 Jan 2002, Alexandre Julliard wrote:
...and makes sure that the W functions are never actually tested with
Unicode input. It's not enough to simply pass converted ASCII strings
to the W functions, we have to test with real Unicode to check for
lossy W-A-W conversions, surrogate handling, non-spacing chars, etc.
The functions *are* really different and have different testing
requirements.
OK, this is good point. But the truth of the matter is that the semantics
of 99.99% of the functions should be encoding independent, because the
documentation from MS is, the way you are supposed to write your code is,
etc. On the other hand, as you pointed out, we would really want to test
all the special handling for the Unicode cases. I think these two
requirements will in fact push us to a better sollution.
As you well pointed out, when we do write tests, we want to test a lot of
things: W-A-W conversions, surrogate handling, non-spacing chars,
etc. With the current compiler support, and the current developer
knowladge, I am afraid we will only see tests like this:
WCHAR teststr[] = { 'U', 'n', 'i', 'c', 'o', 'd', 'e', '\0' };
which tests nothing of the interesting cases you mentioned, and we will
endup with twice as much code foolishly.
My proposal:
-- abstract away the generation of test strings (more of this later)
-- write mosts tests in an encoding independent manner
-- if explicit A or W handling is required, simply use the xxxA or xxxW
without any #ifdefs
-- always compile and run the tests two times for ANSI/UNICODE
The code will look as such:
TCSTR str = TSC_RAND;
while ( (str = teststr(str) ) {
your test which uses 'str' goes here
}
The TSC_XXX set of constants (TSC stands for Test String Class),
identifies a class of strings (such as, 3 charactes long, etc.). The
teststr() iterates though those strings. It has the following prototype:
TCSTR teststr(TCSTR);
And it should be fairly simple to implement (I can provide code, if people
are interested).
Advantages:
-- we can pass in one for the ANSI case, but a few for the Unicode case
where we want to test various things.
-- for regular unit testing, we may want to return only one
representative string for ANSI, and one complicated string for Unicode,
but from time to time we may want to run the tests over a large
number of test strings.
-- the tests so writen can be used to identify and test performance
problems, by simply returning a lot more strings, and by logging how
long it took to run the test.
-- we have a clean way to abstract the testing string, and so most of
the test can be written in an encoding independent way.
-- it makes it no harder, in _any_ way, to write encoding specific
tests. In fact, one can use the teststrA/teststrW versions for their
encoding specific tests to get the above mentioned benefits.
-- it should be trivial to instrument the Makefiles to compile the tests
twice, and thus we should test both the ANSII and the Unicode cases
all the time. (that is, this should not be an Alexandre only task :))
Comments?
--
Dimi.
Re: C testing framewok. Updated. SystemParametersInfo unit test.
--- Dimitrie O. Paun [EMAIL PROTECTED] wrote: On Thu, 17 Jan 2002, Alexandre Julliard wrote: -- if explicit A or W handling is required, simply use the xxxA or xxxW without any #ifdefs You definitely need #ifdefs or just unicode conditional block around explicit calls of xxxW functions. Otherwise these functions will be called on platforms which do not support Unicode, even if test application is compiled in ANSI mode. In general I like your idea. Andriy Palamarchuk __ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/
Re: C testing framewok. Updated. SystemParametersInfo unit test.
On Fri, 18 Jan 2002, Andriy Palamarchuk wrote:
You definitely need #ifdefs or just unicode
conditional block around explicit calls of xxxW
functions. Otherwise these functions will be called on
platforms which do not support Unicode, even if test
application is compiled in ANSI mode.
Fine, you have a point. Just do:
if (UNICODE) {
W explicit code
}
instead. I hate #ifdefs!
If we do that, we can use the tests to also determine the performance
difference between A W, which is great 'cause it will give us an idea of
the penalty for all those string translations and memory allocations.
(I think this can be added to the 'Advantages' section of my previous
mail, :) )
In general I like your idea.
Thanks.
--
Dimi.
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Alexandre Julliard wrote: You don't need to set the variable when running make test, but you have the possibility to set it if you want to override the default for whatever reason. I don't see why we should impose a less flexible solution that won't be more robust anyway. Alexandre, by your suggestion I implemented the switch, but left automated platform detection in place. Platform detection will work when the environment variable is not set or set to unknown value. Can you suggest more reliable way to detect if test runs on Wine? One more important change in C framework is support of Unicode in testing. Dimitry, can you look at it? See how it is used in sysparams.c, search for has_unicode function call. I'm interested in your comments. Andriy Palamarchuk __ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/ sysparams.tar.gz Description: sysparams.tar.gz
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Andriy Palamarchuk [EMAIL PROTECTED] wrote: One more important change in C framework is support of Unicode in testing. Dimitry, can you look at it? See how it is used in sysparams.c, search for has_unicode function call. I'm interested in your comments. I would suggest to explicitly use A and W suffixes to avoid confusion: not just SystemParametersInfo, but SystemParametersInfoA. -- Dmitry.
Re: C testing framewok. Updated. SystemParametersInfo unit test.
On Thu, 17 Jan 2002, Dmitry Timoshkov wrote: I would suggest to explicitly use A and W suffixes to avoid confusion: not just SystemParametersInfo, but SystemParametersInfoA. For tests, I think we should in fact use SystemParametersInfo, and then compile the test twice -- for A and for W. Both versions should behave the same, right? If we make people write the same code twice (for A and for W), we will either (1) get no tests, or (2) have tests that exercise only one of the two cases. -- Dimi.
Re: C testing framewok. Updated. SystemParametersInfo unit test.
--- Dimitrie O. Paun [EMAIL PROTECTED] wrote: On Thu, 17 Jan 2002, Dmitry Timoshkov wrote: For tests, I think we should in fact use SystemParametersInfo, and then compile the test twice -- for A and for W. Both versions should behave the same, right? IMHO (after consulting with MSDN) you can write A-W portable code only with MFC. Let me know if I'm wrong. Andriy Palamarchuk __ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Dimitrie O. Paun [EMAIL PROTECTED] writes: The Win32 API is meant to be used that way, and so we should test it. Besides, I don't see why they are so ugly. Writing xxx instead of xxxA or xxxW is not ugly in any stretch of the word. What's ugly is that you don't compile what you write. This is the most sure recipe to make sure the code doesn't compile. People will test code in ASCII mode, and when some poor soul (like me ;-) tries to compile in Unicode mode to run regression tests it won't work because nobody ever tried to compile it that way. Or worse it will compile but not do what was intended because someone has been mixing char and tchar or whatever. xxxA and xxxW are different functions (and in fact do not necessarily behave the same way for all inputs), and they need to be tested independently. -- Alexandre Julliard [EMAIL PROTECTED]
Re: C testing framewok. Updated. SystemParametersInfo unit test.
On 17 Jan 2002, Alexandre Julliard wrote: [...] What's ugly is that you don't compile what you write. This is the most sure recipe to make sure the code doesn't compile. People will test code in ASCII mode, and when some poor soul (like me ;-) tries to compile in Unicode mode to run regression tests it won't work because nobody ever tried to compile it that way. Or worse it will compile but not do what was intended because someone has been mixing char and tchar or whatever. I would call it part of normal test debugging. You could also reject tests that don't work in both modes. We could also have a Mr 'Unicode' whose work would be to make sure tests work in Unicode and fix them if necessary. xxxA and xxxW are different functions (and in fact do not necessarily behave the same way for all inputs), and they need to be tested independently. Most of the time they behave the same way, and in the few cases where they don't, one can still call xxxA and xxxW. And having a single test that can be used to test both A and W reduces code duplication, which helps maintainability and reduces the amount of work we have to do. The only issue I see is if the xxxA is implemented but xxxW is not or reciprocally. Then we have a test that fails but cannot really be put as a TODO since xxxA works. the right way to fix this is to implement the function that is missing. Another way to handle this issue is to also have a 'TODOA' and a 'TODOW'. -- Francois Gouget [EMAIL PROTECTED]http://fgouget.free.fr/ RFC 2549: ftp://ftp.isi.edu/in-notes/rfc2549.txt IP over Avian Carriers with Quality of Service
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Francois Gouget [EMAIL PROTECTED] writes: And having a single test that can be used to test both A and W reduces code duplication, which helps maintainability and reduces the amount of work we have to do. ...and makes sure that the W functions are never actually tested with Unicode input. It's not enough to simply pass converted ASCII strings to the W functions, we have to test with real Unicode to check for lossy W-A-W conversions, surrogate handling, non-spacing chars, etc. The functions *are* really different and have different testing requirements. -- Alexandre Julliard [EMAIL PROTECTED]
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Francois Gouget wrote: The only issue I see is if the xxxA is implemented but xxxW is not or reciprocally. Then we have a test that fails but cannot really be put as a TODO since xxxA works. the right way to fix this is to implement the function that is missing. Another way to handle this issue is to also have a 'TODOA' and a 'TODOW'. Oh, that's an easy one :-) : #ifdef _UNICODE TODO(xxxW) #else xxxA #endif Andriy Palamarchuk
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Andriy Palamarchuk [EMAIL PROTECTED] writes: As we discussed before we can define platform at runtime, so do we really need the WINETEST_PLATFORM switch? This is one more thing to break. Somebody will eventually mess with the switch when they run the test application manually. Well, that's the point, so that you can change the platform to see what happens. I needed that to test my todo implementation without having to reboot under Windows. Automatic platform definition is pretty reliable. I don't think checking for a Wine registry key is reliable at all. Nothing prevents someone from having this key under Windows. -- Alexandre Julliard [EMAIL PROTECTED]
Re: C testing framewok. Updated. SystemParametersInfo unit test.
Andriy Palamarchuk [EMAIL PROTECTED] writes: Can you suggest how to implement these switches - as command-line arguments or environment variables? Environment variables are better. I did that already in the perl framework, look at the runtest script for the variables I'm using. You probably want to do things the same way so that it's easier to mix Perl and C tests. I don't have a die on failure flag though, but I don't think this is really necessary. -- Alexandre Julliard [EMAIL PROTECTED]
