In Line.
-Mikeal
On Feb 13, 2006, at 6:06 PM, John Anderson wrote:
Aparna Kadakia wrote:
Finally, I noticed that TestNewCollection, when run alone fails by
hanging forever, but when run in the order of the functional tests,
fails with an attribute error. I was surprised to learn that each
functional test isn't run from the same known starting point,
e.g. they
are all run one after another in the same execution of Chandler.
For
someone like me, who just wants to add a new functional test, it
would
be convenient if my test results didn't depend on the random
state of
Chandler left over after the tests that ran before it --
especially when
some of the test access network resources.
We haven't seen too many problems with running all the tests
together in the test suite so far. More often than not it has
discovered real problems. We do have the flexibility of running
the tests individually on a new repo if we have to.
But I am open to the idea of having a functional test suite that
runs each test on a new repo copied over if that helps the
developers debugging their tests.
After looking into the TestNewCollection failure, it looks like it
is a real bug, so I now see the benefit of running them together.
How about if we run them both separately and together? That way
we'll know if it is an order related bug.
Grant had a suggestion today that it may be nice to have an option
that restarted chandler and cleared the repository in between tests,
so each test in a suite ran in a clean state.
This would be a slightly cleaner way of running them separately than
stringing together a bunch of do_test commands and we could even
capture the log output and get PASS or FAIL for the entire suite. But
this would require some work be done on do_tests.
This feature is simple in the new framework arch but there is no
clear delivery date for that, so if we want this feature in the short
term it will need to be done in do_test or as a short term fix we
would have to string together separate do_test runs.
-Mikeal
John
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
