I filed https://issues.apache.org/jira/browse/DERBY-6152 for the 10.10
upgrade and compatibility testing and started doing some work on it but
I think this is going to be more helpful for 10.10.2.1 by the time I
finish. The major problems are:
Just running the 10.8 tests against the 10.10 product jars fails
miserably ( Failures: 433, Errors: 74)
The failures fall into three categories.
1) Cascade failures and cleanup problems.
I hope to cleanup the tests and test infrastructure to cleanup
properly so that tests cannot interfere with the ones ahead. I think
fixtures and tests that cleanup after themselves should be a general
strategy when adding new tests.
2) Tests in the functional tests for internal or unstable interfaces
that have now changed behavior with 10.10.
It would be nice in the future to have a way to flag this testing
so it can be avoided on a compatibility test run. I am not sure the best
way to do that, but welcome ideas, maybe a system property and then a if
(testInternal()) block around the test code would allow us to run
compatibility tests.
3) Real behavior changes that might require release notes or be a problem.
These are the ones we want to see
Anyway, I will forge ahead with the cleanup issues which will be I hope
generally useful and welcome input on how to make this testing easier in
the future.
Best
Kathey
