I agree with Ulli here. Upgrade tests is something that would be great to have but turning ATH to what you describe does not make sense.
1) We _do_ want to know that version under test work on its own and not only if you upgrade your config from some earlier version. 2) If we start using config xml to configure/verify we are introducing another interface that can and will change. As we have to keep all our page objects there will be more breakage due to changes in SUT than before (changes in persistence does not affect ATH now). 3) What you suggest allows test creator to create test fixture using _some_ core/plugin version. Whole test suite will then inconsistently test if upgrade works from different version for every test. I expect more serious approach from upgrade tests.* Not to mention that such tests will be close to useless a year from now as it will be performing upgrades most users already performed. 4) Having one test failure for one change seem nice in theory but after a year and half of fixing selenium tests to reflect UI changes I am not really motivated to work against that goal. * What I would like to make sure is users can upgrade from LTS to LTS-rc and from master to master-rc preserving the same functionality. We should use existing POs to setup fixture, then upgrade Jenkins/plugins and execute WHEN and THEN using newer version. This comes for free as we are maintaining page objects already, no one need to update XML fixtures regularly to cover upgrade most people will undergo soon and adding new use case like upgrading to last LTS release line from the previous one is just a matter of test parameterization. -- oliver -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
