On Jun 8, 2007, at 4:15 AM, Mikeal Rogers wrote:
I thought the current test framework already included this information within FunctionalTestSuite. This single file method can easily be extended to include recorded tests and would also have the benefit of being a single place to run both Functional and recorded tests. After all, are not recorded tests just another functional test?It's always been my impression that we will be replacing the "old" functional tests with recorded tests. Running them from the same framework wasn't really considered and isn't how they run now. There were many bugs/feature requests for the old framework which we solved in the new one and as far as I know we've always intended on phasing out the old functional tests and replacing them with recorded scripts.
phasing out the older tests with new recorded tests makes perfect sense - I just wasn't aware at all that a complete replacement of the framework was in the plans. but if that's what the qa team recommend then ok.
Currently we are running tests by filename - is there a plan to have it run some other way?To clarify, the _framework_ only thinks about tests in terms of test modules. You can reference them by filename but it's all the same to the framework.If we are no longer running recorded tests by filename then how will they be run by tinderbox? How will developers use the existing --recordedTest command line parameter to try and reproduce a test failure?The --recordedTest command line runs execute_frame, which uses a prebuilt dictionary of tests that are collected as modules using get_test_modules. Sure, the command line references them by filename but it's really just treated as a dictionary key, the value of which is a test module.Thanks to Jacob, we have a new dynamic menu under the Tools -> scripting menu which lists all available tests ( and doesn't observer platform exclusions by design ) in which you can run each test inside of chandler and we think this will end being used by developers more often for debugging than the --recordedTest command line parameter.
As long as one thing remains constant - if I run "rt.py RecordedTestName" and then run "chandler --recordedTest=RecordedTestName" and the same *single* test runs we are ok. What would be bad is if the two runs cause different code paths to be run.
It sounds like the plan is to migrate to this new framework and that's fine as long as we get a chance to discuss the parameters of the framework and it's behaviour before it's phased in as a replacement.
--- Bear Build and Release Engineer Open Source Applications Foundation (OSAF) [EMAIL PROTECTED] http://www.osafoundation.org [EMAIL PROTECTED] http://code-bear.com PGP Fingerprint = 9996 719F 973D B11B E111 D770 9331 E822 40B3 CD29
PGP.sig
Description: This is a digitally signed message part
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev
