Hi Oliver, As Jesse comments:
- We are trying to be as clear as possible on that kind of PRs, so that both the problem and the fix can be understood and acknowledged by the maintainer and whoever might evaluate such PR. - It's not easy to directly translate what we are doing internally as PCT is designed to use the plugins offered by UC, what would probably lead to ever lasting executions. Also we tight PCT runs to a certain set of core and plugin versions. Being said that, I agree that someone could miss some context or a more in-depth explanation both from a technical (how to execute it) and a conceptual perspective (what PCT intends to be useful for). So we will try to: - Put in place such information so that at least people can understand better the whole PCT itself. - Analyze what needs to be done in PCT to allow executing it with the proper conditions (for example a set of plugins) and put in place some work to make it feasible. Thanks! On Tuesday, May 9, 2017 at 3:46:10 PM UTC+2, Jesse Glick wrote: > > On Tue, May 9, 2017 at 4:11 AM, Oliver Gondža <[email protected] > <javascript:>> wrote: > > a PR that purports to fix > > something you have no evidence of being broken referring to a TLA-named > > testsuite you have never heared of and have no easy way to checking for > > yourself > > FWIW I have asked people filing such PRs to include some background > information as well as instructions to reproduce the failure locally. > In most (not all) cases this comes down to something like > > mvn clean test-compile && mvn -Djenkins.version=2.x.y surefire:test > > or even > > mvn -Djenkins.version=2.x.y clean test > > The last can be kept passing by writing your `Jenkinsfile` as > > buildPlugin(jenkinsVersions: [null, '2.x.y']) > > for whatever LTS line. There are also some cases where failures are > due to plugin-to-plugin dependency bumps, which `buildPlugin` does not > currently support as a testing option. > > As to whether there can be a public Jenkins job running precisely the > same set of tests as CloudBees runs internally, I am doubtful, since > the particular set of core & plugin versions being tested is defined > by the versions promoted in CloudBees products, and having this as a > scheduled job would make little sense since a test run is tied to a > change to that revision matrix. > > Now what *would* be extremely useful, if someone cares to take the > time to set it up and maintain it, is a job on ci.jenkins.io which > periodically (daily?) runs the PCT against some defined but > ever-increasing (transitively closed) set of plugins, using trunk > snapshot versions of core and all included plugins. If failures are > evaluated and assigned promptly, this would alert developers to > compatibility regressions they need to either fix or acknowledge, > before releases are even made (albeit after faulty PRs are merged—a > harder problem to solve I think). > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/d5f68e72-b0ae-4ed9-8fc7-713fb823f7a5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
