On 26 Dec, 2006, at 17:36, Mikeal Rogers wrote:
Personally, I'd vote against both asserts (which don't fire if
you're running optimized, as we'd want for the performance tests)
and the previous pass/fail approach (which is baroque and error-
prone). Instead, why not have tests inherit from
unittest.TestCase, since:
I'm not against integrating with unittest, but I'm hesitant to use
unittest _every_ time we run a script. One thing I forgot to
mention was that we plan on adding a submenu in test, that has a
list of all the standard tests we have written, that can be run at
any time. A big difference with the new test approach is that we're
using one feature, script recording/playback, for a few different
facilities. If someone records a script and plays it back, they
don't necessarily want to run a "test", they could just be
automating some piece of their workflow.
1) That has a pretty expressive API for validating state
(TestCase.failUnless, TestCase.failUnlessEqual, etc)
2) Tests that depend on setting up shared data can use
inheritance, usually in conjunction with setUp() methods.
With the way we have things now we're trying to keep people from
needing to write code to get a test going. In this case how do we
even handle tests that use shared data? This may be a feature we
want to build for continuous integration (tinderbox) that's outside
of the regular script recording/playback.
3) Less wheel reinvention, less having developers have to learn
new APIs.
So there shouldn't be any api learning for people writing tests,
because writing tests shouldn't require any code writing. The
script recorder will output a test a file with a "run" method that
contains everything needed to run the test.
How are you supposed to verify the data without writing code? And you
said '"run" method' -- what class is that method defined on? If
you're spitting out a script anyway, it's just as easy to stick in
the class definition and "def runTest(self)".
I think for the continuous integration case (tinderbox) we'll need
to run the tests a bit differently that we do when selecting them
from a file or menu as the case is now. A good idea would be to
write a TestLoader class that pulls all the test methods in from
all the files we've saved in script-recording and assigns them as
methods in TestCase clases so that we can run the full test suite
in unittest.
-Mikeal
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
