I like the idea of having one way to get a list of tests, that all the
different methods of running the tests can use.
I also like not having the tests not depend on each other.
What I don't understand is why you want to use a attribute in the test
to cause it to be excluded when a simple text file list of excluded
tests would seem to me to be:
easier to code
easier to discover and use
not have the problems with the test not importing outside of Chandler python
-Dan
Mikeal Rogers wrote:
We've been running recorded scripts as tests for some time now and
recently started solidifying the way the framework works and got them
running on tinderbox. What we have is now being documented, although
it's still a work in
progress, http://chandlerproject.org/Projects/ChandlerAutomatedTestSystemThree
.
There are two big differences between the old cats framework and the
new one. The first is that the tests don't, and in fact _must not_
depend on each other. The second is a much more contentious issue,
which is that the framework uses metadata in the test modules to tell
the framework how it should run.
When temporarily excluding tests from a single or all platforms you
add a special attribute to the module. Declaring common dependencies
(like account settings) is done in much the same way.
The first problems we ran in to were due to the fact that the test
scripts wouldn't import outside of chandler's python which meant that
the tests had to be collected differently in rt.py . I have a patch
in bug 9450 <https://bugzilla.osafoundation.org/show_bug.cgi?id=9450>
to help fix this, now the framework and the test modules can be
imported in rt.py's python and don't depend on any chandler code until
you execute the actual framework or the test methods.
We now have 3 ways that the tests are collected and executed; from a
menu inside the chandler Tools menu, automatically from the chandler
command line, or from rt.py. My patch should allow us to get all of
these paths using the same framework and clean up the problems we've
had with all the different execution methods in the past.
-Mikeal
------------------------------------------------------------------------
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
