On Sun, Jun 8, 2014 at 1:35 PM, ogondza <[email protected]> wrote: > We _do_ want to know that version under test work on its own and not only > if you upgrade your config from some earlier version.
It is nice to verify this, but the plugin maintainer is often passively verifying this as they work on changes anyway, and it is much less important than verifying that everything still works after an upgrade. > If we start using config xml to configure/verify we are introducing > another interface that can and will change. As we have to keep all our page > objects there will be more breakage due to changes in SUT than before I think you only need to update the code generating the XML configs if and when a new test needs to work with new config options. > such tests will be close to useless a year from now as it will be performing > upgrades most users > already performed. Not necessarily true. Just because someone is running a newer version of core or a plugin already does not mean they have resaved a particular kind of configuration using a new format introduced by that version. And plenty of people are running rather old plugin releases, I am afraid. One way or another we will need to be testing a variety of configuration upgrades. (Some in-plugin integration tests already do so using @LocalData of course.) But a lot of tests are not “about” this topic; they are really testing some behavior which does not have much to do with a particular storage format version. So picking an overall harness infrastructure based on the ease of writing configuration upgrade tests is probably overkill. That said, I would much rather write tests which used a simple DSL for configuring Jenkins and some jobs and slaves etc. in a particular way as either XML or in-process Java/Groovy calls than have to set up everything from scratch using the current page objects, which are painfully slow to run and prone to irrelevant failures. Obviously the web driver is useful for the limited number of tests which are actually written to check the configuration UI. As a general comment, I would really like to see ATH live on a continuum with in-plugin unit and integration tests, rather than using a totally different mechanism with no overlap as today. Specifically I would like to be able to · Include acceptance tests in a plugin source repository, or otherwise rapidly rerun those tests after incremental changes to plugin source. · To be able to use Docker fixtures from JenkinsRule or other tests. (This system seems to have no important dependencies on the rest of ATH and ought to be a standalone project that could be used as a org.junit.Rule by any JUnit 4.x test.) · Use core/plugin Java API calls to set up a test but then use HtmlUnit and/or Selenium with page objects to run certain parts. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
