On Aug 25, 2004, at 12:08 AM, Stas Bekman wrote:
Great work, David.
Thanks.
David, First of all, please read: http://perl.apache.org/contribute/docs/style.html
as most style things are all documented there:
Done.
I just quickly scrolled through, haven't had a chance to do a thorough reading, so just a few comments for things that stood out:
-=head1 Basics of Perl Modules Testing +=head1 Basics of Perl Module Testing
Make sure that you check whether there are some xrefs to any of the =head entries, before modifying them. I haven't checked if there are any, just a reminder :)
Right, good catch. Looks like there aren't any in this case:
% grep -ri basics_of_perl . %
mm, not sure where that "" came from but as you work on the doc, please drop the quotes, there is already a way too much alien non-text chars in xrefs to further pollute them with extra quotes. so it should be just
L<Writing Tests|/Writing_Tests>. (I know you didn't write that :)
Will do.
Please don't do that (here and below). There should be /^\s\s\n/ and not /^\n/ in the code. If you run bin/build and look at your patched version's html you will see why (it won't get a continuous bar on the left side).
It's in the docs style guide.
Bleh. That's nasty. I'll leave the spaces in from now on.
> unmodified version in C<blib/lib>. Of course, if you always run C<make
Why C<blib/lib>? It's a dir/file, so either I<> or F<>
I didn't write that; it was there. I'll change it.
How does this look? Both the diff and the revised version of the file attached.
Regards,
David
Index: testing.pod =================================================================== RCS file: /home/cvspublic/modperl-docs/src/docs/general/testing/testing.pod,v retrieving revision 1.34 diff -c -r1.34 testing.pod --- src/docs/general/testing/testing.pod 16 Jul 2004 21:46:11 -0000 1.34 +++ src/docs/general/testing/testing.pod 26 Aug 2004 01:20:36 -0000 @@ -6,12 +6,12 @@ The title is self-explanatory :) -The C<Apache::Test> framework was designed for creating test suits for -products running on Apache httpd webserver (not necessarily +The C<Apache::Test> framework was designed for creating test suites for +products running on the Apache httpd webserver (not necessarily mod_perl). Originally designed for the mod_perl Apache module, it was extended to be used for any Apache module. -This chapter is talking about the C<Apache::Test> framework, and in +This chapter discusses the C<Apache::Test> framework, and in particular explains how to: =over @@ -24,32 +24,33 @@ =back -For other C<Apache::Test> resources, see the L<References|/"References"> +For other C<Apache::Test> resources, see the L<References|/References> section at the end of this document. -=head1 Basics of Perl Modules Testing +=head1 Basics of Perl Module Testing The tests themselves are written in Perl. The framework provides an -extensive functionality which makes the tests writing a simple and +extensive functionality which makes writing tests a simple and therefore enjoyable process. -If you have ever written or looked at the tests most Perl modules come -with, C<Apache::Test> uses the same concept. The script I<t/TEST> is -running all the files ending with I<.t> it finds in the I<t/> -directory. When executed a typical test prints the following: +If you have ever written or looked at the tests that come with most Perl +modules, you'll recognize that C<Apache::Test> uses the same +concepts. The script F<t/TEST> executes runs all the files ending with +F<.t> that it finds in the F<t/> directory. When executed, a typical test +prints the following: 1..3 # going to run 3 tests ok 1 # the first test has passed ok 2 # the second test has passed not ok 3 # the third test has failed -Every C<ok> or C<not ok> is followed by the number which tells which -sub-test has succeeded or failed. +Every C<ok> or C<not ok> is followed by a number that identifies which +sub-test has passed or failed. -I<t/TEST> uses the C<Test::Harness> module which intercepts the -C<STDOUT> stream, parses it and at the end of the tests print the -results of the tests running: how many tests and sub-tests were run, -how many succeeded, skipped or failed. +F<t/TEST> uses the C<Test::Harness> module, which intercepts the +C<STDOUT> stream, parses it and at the end of the tests, prints the +results of the tests: how many tests and sub-tests were run and +how many passed, failed, or were skipped. Some tests may be skipped by printing: @@ -57,13 +58,13 @@ Usually a test may be skipped when some feature is optional and/or prerequisites are not installed on the system, but this is not -critical for the usefulness of the test. Once you test that you cannot -proceed with the tests and it's not a must pass test, you just skip -it. - -By default print() statements in the test script are filtered out by -C<Test::Harness>. if you want the test to print what it does (if you -decide to debug some test) use C<-verbose> option. So for example if +critical for the usefulness of the test. Once you determine that +you cannot proceed with the tests, and it is not a requirement that +the tests pass, you can just skip them. + +By default, C<print> statements in the test script are filtered out by +C<Test::Harness>. If you want the test to print what it does (for example, +to debug a test) use the C<-verbose> option. So for example if your test does this: print "# testing : feature foo\n"; @@ -72,25 +73,50 @@ ok $expected eq $received; in the normal mode, you won't see any of these prints. But if you run -the test with I<t/TEST -verbose>, you will see something like this: +the test with C<t/TEST -verbose>, you will see something like this: # testing : feature foo # expected: 2 # received: 2 ok 2 -When you develop the test you should always put the debug statements -there, and once the test works for you do not comment out or delete -these debug statements. This is because if some user reports a failure -in some test, you can ask him to run the failing test in the verbose -mode and send you back the report. It'll be much easier to understand -what the problem is if you get these debug printings from the user. +When you develop the test you should always insert the debug statements, +and once the test works for you, do not comment out or delete these +debug statements. It's a good idea to leave them in because if some user +reports a failure in some test, you can ask him to run the failing test +in the verbose mode and send you the report. It'll be much easier to +understand the problem if you get these debug printings from the user. + +A simpler approach is to use the C<Test::More> module in your test +scripts. This module offers many useful test functions, including +C<diag>, a function that automatically escapes and passes strings to +C<print> to bypass C<Test::Harnes>: + + use Test::More; + diag "testing : feature foo\n"; + diag "expected: $expected\n"; + diag "received: $received\n"; + ok $expected eq $received; + +In fact, for an example such as this, you can just use Test::More's +C<is> function, which will output the necessary diagnostics in the event +of a test failure: + + is $received, $expected; + +For which the output for a test failure would be something like: -In the section L<Writing Tests|/"Writing_Tests"> several helper functions which make -the tests writing easier are discussed. +not ok 1 +# Failed test (-e at line 1) +# got: '1' +# expected: '2' + +The L<Writing Tests|/Writing_Tests> section documents several helper +functions that make simplify the writing of tests. For more details about the C<Test::Harness> module please refer to its -manpage. Also see the C<Test> manpage about Perl's test suite. +manpage. Also see the C<Test> and C<Test::More> manpages for +documentation of Perl's test suite. =head1 Prerequisites @@ -102,19 +128,18 @@ % perl Makefile.PL % make && make test && make install -If you install mod_perl 2.0, you get C<Apache::Test> installed as -well. +If you install mod_perl 2.0, C<Apache::Test> will be installed with it. =head1 Running Tests -It's much easier to copy-cat things, than creating from scratch. It's -much easier to develop tests, when you have some existing system that -you can test, see how it works and build your own testing environment -in a similar fashion. Therefore let's first look at how the existing -test enviroments work. +It's much easier to copy existing examples than to create something from +scratch. It's also simpler to develop tests when you have some existing +system that to test, so that you can see how it works and build your own +testing environment in a similar fashion. So let's first look at how the +existing test enviroments work. You can look at the modperl-2.0's or httpd-test's (I<perl-framework>) -testing environments which both use C<Apache::Test> for their test +testing environments, both of which use C<Apache::Test> for their test suites. =head2 Testing Options @@ -123,73 +148,73 @@ % t/TEST -help -to get the list of options you can use during testing. Most options +to get a list of options you can use during testing. Most options are covered further in this document. =head2 Basic Testing Running tests is just like for any CPAN Perl module; first we generate -the I<Makefile> file and build everything with I<make>: +the F<Makefile> file and build everything with C<make>: % perl Makefile.PL [options] % make Now we can do the testing. You can run the tests in two ways. The -first one is usual: +first one is the usual: % make test -but it adds quite an overhead, since it has to check that everything -is up to date (the usual C<make> source change control). Therefore you -have to run it only once after C<make> and for re-running the tests -it's faster to run the tests directly via: +But this approach adds quite an overhead, since it has to check that +everything is up to date (the usual C<make> source change control). +Therefore, you have to run it only once after C<make>; for re-running +the tests, it's faster to run them directly via: % t/TEST -When C<make test> or C<t/TEST> are run, all tests found in the I<t> -directory (files ending with I<.t> are recognized as tests) will be +When C<make test> or C<t/TEST> is run, all tests found in the F<t> +directory (files ending with F<.t> are recognized as tests) will be run. =head2 Individual Testing -To run a single test, simple specify it at the command line. For -example to run the test file I<t/protocol/echo.t>, execute: +To run a single test, simply specify it at the command line. For +example, to run the test file F<t/protocol/echo.t>, execute: % t/TEST protocol/echo -Notice that you don't have to add the I<t/> prefix and I<.t> extension -for the test filenames if you specify them explicitly, but you can -have these as well. Therefore the following are all valid commands: +Notice that the F<t/> prefix and the F<.t> extension for the test +filenames are optional when you specify them explicitly. Therefore the +following are all valid commands: % t/TEST protocol/echo.t % t/TEST t/protocol/echo % t/TEST t/protocol/echo.t The server will be stopped if it was already running and a new one -will be started before running the I<t/protocol/echo.t> test. At the +will be started before running the F<t/protocol/echo.t> test. At the end of the test the server will be shut down. When you run specific tests you may want to run them in the verbose -mode, and depending on how the test was written, you may get more -debug information under this mode. This mode is turned on with +mode and, depending on how the tests were written, you may get more +debugging information under this mode. Verbose mode is turned on with I<-verbose> option: % t/TEST -verbose protocol/echo -You can run groups of tests at once. This command: +You can run groups of tests at once, too. This command: % ./t/TEST modules protocol/echo -will run all the tests in I<t/modules/> directory, followed by -I<t/protocol/echo.t> test. +will run all the tests in F<t/modules/> directory, followed by +F<t/protocol/echo.t> test. =head2 Repetitive Testing -By default when you run the test without I<-run-tests> option, the +By default, when you run tests without the I<-run-tests> option, the server will be started before the testing and stopped at the end. If -during a debugging process you need to re-run tests without a need to -restart the server, you can start the server once: +during a debugging process you need to re-run tests without the need to +restart the server, you can start it once: % t/TEST -start-httpd @@ -203,39 +228,39 @@ % t/TEST -stop-httpd -When the server is started you can modify I<.t> files and rerun the -tests without restarting the server. However if you modify response +When the server is running, you can modify F<.t> files and rerun the +tests without restarting it. But if you modify response handlers, you must restart the server for changes to take an -effect. However the changes are done in the perl code only, it's -possible to orrange for Apache::Test to L<handle the code reload +effect. However, if the changes are only to perl code, it's +possible to arrange for Apache::Test to L<handle the code reload without restarting the server|/Using_Apache__Test_to_Speed_up_Project_Development>. The I<-start-httpd> option always stops the server first if any is running. -Normally when I<t/TEST> is run without specifying the tests to run, +Normally, when F<t/TEST> is run without specifying the tests to run, the tests will be sorted alphabetically. If tests are explicitly -passed as arguments to I<t/TEST> they will be run in a specified +passed as arguments to F<t/TEST> they will be run in the specified order. =head2 Parallel Testing Sometimes you need to run more than one C<Apache::Test> framework instances at the same time. In this case you have to use different -ports for each instance. You can specify explicitly which port to use, -using the I<-port> configuration option. For example to run the server -on port 34343: +ports for each instance. You can specify explicitly which port to use +using the I<-port> configuration option. For example, to run the server +on port 34343, do this: % t/TEST -start-httpd -port=34343 -or by setting an evironment variable C<APACHE_TEST_PORT> to the -desired value before starting the server. +You can also affect the port by setting the C<APACHE_TEST_PORT> +evironment variable to the desired value before starting the server. Specifying the port explicitly may not be the most convenient option if you happen to run many instances of the C<Apache::Test> framework. -The I<-port=select> option comes to help. This option will -automatically pick for the next available port. For example if you +The I<-port=select> option helps such situations. This option will +automatically select the next available port. For example if you run: % t/TEST -start-httpd -port=select @@ -243,48 +268,46 @@ and there is already one server from a different test suite which uses the default port 8529, the new server will try to use a higher port. -There is one problem that remains to be resolved though. It's possible +There is one problem that remains to be resolved, though. It's possible that two or more servers running I<-port=select> will still decide to use the same port, because when the server is configured it only tests -whether the port is available but doesn't call bind() -immediately. Thefore there is a race condition here, which needs to be -resolved. Currently the workaround is to start the instances of the -C<Apache::Test> framework with a slight delay between each other. -Depending on the speed of you machine, 4-5 seconds can be a good -choice. that's approximately the time it takes to configure and start -the server on a quite slow machine. +whether the port is available but doesn't call bind() immediately. This +race condition needs to be resolved. Currently the workaround is to +start the instances of the C<Apache::Test> framework with a slight delay +between them. Depending on the speed of your machine, 4-5 seconds can be +a good choice, as this is the approximate the time it takes to configure +and start the server on a quite slow machine. =head2 Verbose Mode -In case something goes wrong you should run the tests in the verbose -mode: +In case something goes wrong you should run the tests in verbose mode: % t/TEST -verbose -In this case the test may print useful information, like what values +In verbose mode, the test may print useful information, like what values it expects and what values it receives, given that the test is written -to report these. In the silent mode (without C<-verbose>) these -printouts are filtered out by C<Test::Harness>. When running in the -I<verbose> mode usually it's a good idea to run only problematic tests -to minimize the size of the generated output. +to report these. In silent mode (without C<-verbose>), these printouts +are filtered out by C<Test::Harness>. When running in I<verbose>, mode +usually it's a good idea to run only problematic tests in order to +minimize the size of the generated output. -When debugging problems it helps to keep the I<error_log> file open in -another console, and see the debug output in the real time via +When debugging tests, it often helps to keep the F<error_log> file open +in another console, and see the debug output in the real time via tail(1): % tail -f t/logs/error_log Of course this file gets created only when the server starts, so you cannot run tail(1) on it before the server starts. Every time C<t/TEST --clean> is run, I<t/logs/error_log> gets deleted, therefore you have -to run the tail(1) command again, when the server is started. +-clean> is run, F<t/logs/error_log> gets deleted; therefore, you'll have +to run the tail(1) command again once the server starts. =head2 Colored Trace Mode If your terminal supports colored text you may want to set the -environment variable C<APACHE_TEST_COLOR> to 1 to enable the colored -tracing when running in the non-batch mode, which makes it easier to -tell the reported errors and warnings, from the rest of the +environment variable C<APACHE_TEST_COLOR> to 1 to enable any colored +tracing when running in the non-batch mode. Colored tracing mode can +make it easier to discriminate errors and warnings from other notifications. =head2 Controlling the Apache::Test's Signal to Noise Ratio @@ -296,14 +319,14 @@ emerg alert crit error warning notice info debug -where I<emerg> is the for the most important messages and I<debug> for -the least important ones. +where I<emerg> is the for the most important messages and I<debug> is +for the least important ones. -Currently the default level is I<info>, therefore any messages which -fall into the I<info> category and above (I<notice>, I<warning>, etc). -This tracing level is unrelated to the Apache's C<LogLevel> mechanism, -which Apache-Test sets to C<debug> in F<t/conf/httpd.conf> and you can -override it F<t/conf/extra.conf.in>. +Currently, the default level is I<info>; therefore, any messages which +fall into the I<info> category and above (I<notice>, I<warning>, etc) +will be output. This tracing level is unrelated to Apache's C<LogLevel> +mechanism, which Apache-Test sets to C<debug> in F<t/conf/httpd.conf> +and which you can override F<t/conf/extra.conf.in>. Let's assume you have the following code snippet: @@ -315,12 +338,12 @@ % t/TEST -trace=warning ... -now only the warning message +now only the warning message careful, perl on the premises -will be printed. If you want to see the I<debug> messages you can -change the default level using I<-trace> option: +will be printed. If you want to see I<debug> messages, you can +change the default level using C<-trace> option: % t/TEST -trace=debug ... @@ -330,29 +353,30 @@ to a file. Refer to the C<Apache::TestTrace> manpage for more information. -Finally you can use methods: C<emerg()>, C<alert()>, C<crit()>, -C<error()>, C<warning()>, C<notice()>, C<info()> and C<debug()> in -your client and server side code. This if useful for example if you -have some debug tracing that you don't want to be printed during the -normal C<make test>. However if some users have a problem you can ask -them to run the test suite with the trace level of 'debug' and voila -they can send you the extra debug output. Moreveor all these functions -use C<Data::Dumper> to dump arguments which are references to perl -structures. So for example your code may look like: +Finally, you can use the C<emerg()>, C<alert()>, C<crit()>, C<error()>, +C<warning()>, C<notice()>, C<info()> and C<debug()> methods in your +client and server side code. These methods are useful when, for example, +you have some debug tracing that you don't want to be printed during the +normal C<make test> or C<.Build test>. However, if some users have a +problem you can ask them to run the test suite with the trace level set +to 'debug' and, voila, they can send you the extra debug output. +Moreveor, all of these functions use C<Data::Dumper> to dump arguments +that are references to perl structures. So for example your code may +look like: use Apache::TestTrace; ... my $data = { foo => bar }; debug "my data", $data; -and only when run with C<-trace=debug> it'll output. +and only when run with C<-trace=debug> it'll output: my data $VAR1 = { 'foo' => 'bar' }; -Normally it will not print anything. +Normally it will print nothing. @@ -491,7 +515,7 @@ cases. You should create a small script to drive C<Apache::TestSmoke>, -usually I<t/SMOKE.PL>. If you don't have it already, create it: +usually F<t/SMOKE.PL>. If you don't have it already, create it: #file:t/SMOKE.PL #--------------- @@ -508,10 +532,10 @@ Apache::TestSmoke->new(@ARGV)->run; -Usually I<Makefile.PL> converts it into I<t/SMOKE> while adjusting the -perl path, but you can create I<t/SMOKE> in first place as well. +Usually F<Makefile.PL> converts it into F<t/SMOKE> while adjusting the +perl path, but you can create F<t/SMOKE> in first place as well. -I<t/SMOKE> performs the following operations: +F<t/SMOKE> performs the following operations: =over @@ -566,7 +590,7 @@ % t/SMOKE If you want to work on certain tests you can specify them in the same -way you do with I<t/TEST>: +way you do with F<t/TEST>: % t/SMOKE foo/bar foo/tar @@ -606,14 +630,14 @@ =item * -preamble -configuration directives to add at the beginning of I<httpd.conf>. +configuration directives to add at the beginning of F<httpd.conf>. For example to turn the tracing on: % t/TEST -preamble "PerlTrace all" =item * -postamble -configuration directives to add at the end of I<httpd.conf>. For +configuration directives to add at the end of F<httpd.conf>. For example to load a certain Perl module: % t/TEST -postamble "PerlModule MyDebugMode" @@ -680,7 +704,7 @@ =item * -head -Issue a C<HEAD> request. For example to request I</server-info>: +Issue a C<HEAD> request. For example to request F</server-info>: % t/TEST -head /server-info
testing.pod
Description: Binary data
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
