Sorry for chiming in late. I just caught up on this thread.
Mikeal,
It definitely seems like Bear and Dan, who are major stakeholders in
this, would like to have a single file that maintains the exclusion
list for various reasons already mentioned. I guess we should provide
for that in the new framework.
Also, as we begin to enhance the framework with additional features,
lets make sure we rope in the relevant people to be involved in the
design, so we don't spend precious time coding it up before it is
approved.
Aparna
On Jun 8, 2007, at 1:15 AM, Mikeal Rogers wrote:
I thought the current test framework already included this
information within FunctionalTestSuite. This single file method
can easily be extended to include recorded tests and would also
have the benefit of being a single place to run both Functional
and recorded tests. After all, are not recorded tests just
another functional test?
It's always been my impression that we will be replacing the "old"
functional tests with recorded tests. Running them from the same
framework wasn't really considered and isn't how they run now.
There were many bugs/feature requests for the old framework which
we solved in the new one and as far as I know we've always intended
on phasing out the old functional tests and replacing them with
recorded scripts.
Currently we are running tests by filename - is there a plan to
have it run some other way?
To clarify, the _framework_ only thinks about tests in terms of
test modules. You can reference them by filename but it's all the
same to the framework.
If we are no longer running recorded tests by filename then how
will they be run by tinderbox? How will developers use the
existing --recordedTest command line parameter to try and
reproduce a test failure?
The --recordedTest command line runs execute_frame, which uses a
prebuilt dictionary of tests that are collected as modules using
get_test_modules. Sure, the command line references them by
filename but it's really just treated as a dictionary key, the
value of which is a test module.
Thanks to Jacob, we have a new dynamic menu under the Tools ->
scripting menu which lists all available tests ( and doesn't
observer platform exclusions by design ) in which you can run each
test inside of chandler and we think this will end being used by
developers more often for debugging than the --recordedTest command
line parameter.
I'm guessing that I'm missing information as to why recorded tests
cannot use the same framework that the current functional tests
use (with appropriate changes for the setup/teardown needs).
The recorded tests have never used the old framework. At this point
it would be better to run the old tests in the new framework than
the other way around as we've taken some feature requests and
worked them in to the new framework. But like I said before, it's
always been my impression that we are phasing out the old tests as
the recorded scripts stabilize.
-Mikeal
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
