Hello Zoe, Friday, March 16, 2007, 3:26:20 PM, you wrote:
> Hi Marcus > Thanks - that answers the question ...and test cases are on their way:-) > I have just committed 3 tests in the PHP 6 stream (strcasecmp, strcmp, > strlen) and would very much appreciate any feedback on them. We are > aiming for both line coverage and functional coverage, it would be good > to know if we are missing obvious things. Well line coverage can be seen by gcov.php.net. Domain coverage can't hardly be detcted. From what I can see you did a good job of trying to capture all kinds of input to those functions. You also used echo lines to group the test blocks which makes it easier to find failing locations. Then I saw you changed a hell lot of expectations from "(-%d)" to "(%i)". While this pretty much works it drops an important detail of the expectation. The old version was aware of the sign of the result while the new one is not. That means that now you might fix a bug in those functions that lead to an erroneous change in the sign and not being able to detect that. Though a lot os code is relying on the sign (e.g. any sorting algorithm). > For the future I think we will probably split out the test cases into > good-path tests and error-path tests, putting both into one test case > has resulted in the test cases being quite big. That sounds like a good idea. Long expect sections make it hard to identify the spot that causes the test failure. > I have one specific question though, I notice that the string that > follows Warning: is changing in PHP6, for example in PHP5 one would expect: > Warning: Wrong parameter count for strcasecmp() in blah.php on line 3 > In PHP6: > Warning: strcasecmp() expects exactly 2 parameters, 0 given in blah.php > on line 3 > The PHP6 message is clearly an improvement and the test cases that I > have just put in PHP6 check the Warning: like this: > --EXPECTF-- > Warning: strcasecmp() expects exactly 2 parameters, 0 given in %s on line %d > But of course, this will only pass on PHP6. We can deal with this in > two ways: > (a) Use "Warning: %s" in the EXPECTF section. This will pass test on > PHP5 and PHP6 but it won't check that PHP is giving the right error > mesage - ie "exactly 2 parameters" as opposed to "exactly 42 parameters" > (b) We can have different test cases for PHP5 and PHP6. > I don't particularly like either of these, but have a slight preference > for (b) because it's a more satisfactory test of the implementation. > What's your view? Or anyone else's? Do b) and be as precise as possible. That means we usually even test for the name of the testfile in the message excluding the directory and any (back)slash. best regards marcus > Marcus Boerger wrote: >> Hello Zoe, >> >> we decided to go the --INI-- route to test as many configurations as >> possible. And only for specific tests change specific ini settings as >> necessary. If one usees phpt as a test framework for his on web stuff >> then it might be a good idea to do as you suggested. You would simply >> do "php run-tests.php -c <ini-file>". Hope this satisfies you :-) >> >> best regards >> marcus >> >> p.s.: Looking forward for new tests from your side >> >> Wednesday, March 14, 2007, 4:39:43 PM, you wrote: >> >> >>> Hi - I wonder if anyone could help me with a testing question? >>> >> >> >>> We have written a few additional test cases for string functions which I >>> was about to check-in, then I noticed that the test cases pass OK if I >>> use the php.ini-dist file and fail if I don't specify an file (ie use >>> all default settings). >>> >> >> >>> The difference turns out to the be the use of precision =12 in the >>> php.ini-dist, rather than the the default value of 14. >>> >> >> >>> It's easy enough to work around this and ensure that test cases pass in >>> both environments with an --INI-- section in the test case, however - I >>> wondered if there would be any value in having a php-ini file that was >>> explicitly used for tests? >>> >> >> >>> We had been using the php.ini-dist file because it's slightly easier to >>> be precise about the test environment if you can say what you are using >>> rather than saying "I'm using the default values". I also find it rather >>> too easy to get the wrong php.ini file accidentally :-( so specifying >>> one is easier, even if it only contains default values. Best regards, Marcus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
