Nicolas Lehuen wrote:
Hi,
There is something I'd like to do for the 3.3 version : it is to
refactor the test suite. It's more a chore than real development, but
the current test suite is slowly becoming big and quite difficult to
maintain.
Also, I think the size and complexity may be intimidating to new
contributors. I don't mind so much now that I'm familiar with it but
when you first dive in it looks like a lot of code. We want the tests
themselves to be as robust as possible and if refactoring helps I'm all
for it. We've had a few problems where the a unit test was incorrect and
as a result was missing mod_python problems.
What I'd like to do is simply split the test runner and the published
tests in various parts, matching the different functionalities that
are tested.
What would be great, then, is to introduce platform-specific tests, of
even MPM-specific tests, so that we could exercise some parts of the
code that are only available with a threaded MPM (for example, the
MemorySession implementation, or the publisher reload test which I
erroneously implemented with a threaded MPM in mind).
+1
One thing that I'd like to settle with you is whether you are OK to
split the big PerRequestTestCase class into multiple classes, to ease
maintenance and configuration switches. This has the downside that
Apache server is likely to be stopped and restarted a few more times
during the tests, so the tests will take more time to run. On the
other hand, I'd like to introduce a way to select the tests to run
when launching the test suite, so that we won't have to wait for the
whole test suite to pass while debugging a given problem.
I think this is a good idea, and I've been thinking along the same lines
in light of the changes that are necessary to support apache 2.2 (ie,
different test configurations required for 2.0 and 2.2). I'm not too
concerned about taking more time for the tests to complete. Anything we
can do to make the tests more reliable and easier to write will be worth
the extra testing time and the initial refactoring effort.
Ah, and one thing I'd definitely get rid of - usage of backquotes like
in `rsp`. I really don't like this magic quoting thingy, it reminds me
all too much of PHP :).
Well, we can't have that now can we? :)
I don't know if this would be practical, but one idea I have is to put
each test in it's own file in a test_modules direcotry. This directory
would be scanned on test startup and the tests added dynamically. That
way if someone contributes some new code, they can also provide the
corresponding unit test without worrying about editing the unwieldy
test.py and htdocs/tests.py files. All they need to do is create
test_modules/mytest.py and htdocs/mytest.py files which get dropped into
the test framework with no additional configuration required.
Oh and if we are refactoring the tests, I want a "make tests" rule. I'm
tired of doing: ./configure; make; sudo make install; make tests; DOH!
cd test; python test.py. :)
Jim
