When we were talking about this in the #QA channel on IRC I thought you were talking about a simple python file to just contain the tests to run or exclude and not about adding metadata to the tests themselves. Wouldn't this be simple to do as a tests.py file that is pure basic python that can be loaded by both chandler and rt and just contains an exclude list and then does a simple dir walk?
On Jun 7, 2007, at 5:52 PM, Mikeal Rogers wrote:
We've been running recorded scripts as tests for some time now and recently started solidifying the way the framework works and got them running on tinderbox. What we have is now being documented, although it's still a work in progress, http://chandlerproject.org/Projects/ ChandlerAutomatedTestSystemThree .There are two big differences between the old cats framework and the new one. The first is that the tests don't, and in fact _must not_ depend on each other. The second is a much more contentious issue, which is that the framework uses metadata in the test modules to tell the framework how it should run.When temporarily excluding tests from a single or all platforms you add a special attribute to the module. Declaring common dependencies (like account settings) is done in much the same way.The first problems we ran in to were due to the fact that the test scripts wouldn't import outside of chandler's python which meant that the tests had to be collected differently in rt.py . I have a patch in bug 9450 to help fix this, now the framework and the test modules can be imported in rt.py's python and don't depend on any chandler code until you execute the actual framework or the test methods.We now have 3 ways that the tests are collected and executed; from a menu inside the chandler Tools menu, automatically from the chandler command line, or from rt.py. My patch should allow us to get all of these paths using the same framework and clean up the problems we've had with all the different execution methods in the past.
--- Bear Build and Release Engineer Open Source Applications Foundation (OSAF) [EMAIL PROTECTED] http://www.osafoundation.org [EMAIL PROTECTED] http://code-bear.com PGP Fingerprint = 9996 719F 973D B11B E111 D770 9331 E822 40B3 CD29
PGP.sig
Description: This is a digitally signed message part
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev
