On Tue, May 9, 2017 at 4:11 AM, Oliver Gondža <[email protected]> wrote: > a PR that purports to fix > something you have no evidence of being broken referring to a TLA-named > testsuite you have never heared of and have no easy way to checking for > yourself
FWIW I have asked people filing such PRs to include some background information as well as instructions to reproduce the failure locally. In most (not all) cases this comes down to something like mvn clean test-compile && mvn -Djenkins.version=2.x.y surefire:test or even mvn -Djenkins.version=2.x.y clean test The last can be kept passing by writing your `Jenkinsfile` as buildPlugin(jenkinsVersions: [null, '2.x.y']) for whatever LTS line. There are also some cases where failures are due to plugin-to-plugin dependency bumps, which `buildPlugin` does not currently support as a testing option. As to whether there can be a public Jenkins job running precisely the same set of tests as CloudBees runs internally, I am doubtful, since the particular set of core & plugin versions being tested is defined by the versions promoted in CloudBees products, and having this as a scheduled job would make little sense since a test run is tied to a change to that revision matrix. Now what *would* be extremely useful, if someone cares to take the time to set it up and maintain it, is a job on ci.jenkins.io which periodically (daily?) runs the PCT against some defined but ever-increasing (transitively closed) set of plugins, using trunk snapshot versions of core and all included plugins. If failures are evaluated and assigned promptly, this would alert developers to compatibility regressions they need to either fix or acknowledge, before releases are even made (albeit after faulty PRs are merged—a harder problem to solve I think). -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3TK6GbtysPYVWyuyCdM-8PBzkaQZLsU22evZhvPg4U%2BQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
