spriebsch Sun Apr 26 14:41:02 2009 UTC Added files: /phpruntests/documentation introduction.txt Log: Started introductory documentation.
http://cvs.php.net/viewvc.cgi/phpruntests/documentation/introduction.txt?view=markup&rev=1.1 Index: phpruntests/documentation/introduction.txt +++ phpruntests/documentation/introduction.txt Introduction to run-tests ------------------------- runtests is a test environment written in PHP. It is used to tests PHP itself. It is not a replacement for unit test frameworks like PHPUnit. The basic principle of automated tests is very simple. Compare a computed result with some pre-calculated "known good", expected value. If both match, the test has passed. If they do not match, the tests failes, and there is probably a bug in PHP, or some precondition for the test was not met. Testing PHP is a far more complex issue than it may seem at first glance, because each test may require an individual PHP configuration, certain settings at operting system level (for example environment variables), or even external services (like databases, LDAP servers or IMAP servers). A test that requires external services can be complex to set up, but there is no way around that. How could you make sure that running a SQL statement against a database works without actually running it against a database? Since a failing test will probably leave its environment in an unpredictable state, a high level of test isolation is required. In other words, this means that we need to set up a new preconfigured PHP process for each test. This is the only way to ensure that every test runs in a clean environment. So, to execute each test, runtests must set up a PHP process, have it run the test, collect the output, and compare it to the pre-calculated "known good" expected output. That spawned PHP process (hopefully) terminates, so at the process level, we do not have to worry about the potential mess that a test has left behind. Still, if the test has created files, databases, or modified the global system environment, additional work may be required to clean up after the test. Since PHP runs on most of the available platforms, runtests must also run on all these platforms. From an implementation point of view, this means that runtests must work on a minimal, stock PHP installation, and should make as few as possible assumptions on the system environment. All the more or less known cross-platform issues like different line endings, directory separators, case handling, as well as limits of different operating system families (file name path length, include path length) have to be taken into account and being dealt with in runtests. Since runtests spawns off a separate PHP process to run a test, it is not necessary to run each test in the same PHP version that runtests runs on. In other words, that means that you can test a different PHP version than you are actually running - and in fact, it may be a good idea to use a "known good" PHP version to test a new, less tested PHP version. After all, bugs in the PHP version runtests itself is running on might affect runtests itself, and thus make the test results less reliable. We have already mentioned that tests need an individual environment. Not all tests can be run at the command line, for example. If a test need to make sure that PHP returns correct HTTP header, or processes GET or POST input, the test probably requires the CGI SAPI to run. -- PHP CVS Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php