Hi, Just to bump the threads. As a part of JEP-200 I needed to run PCT against a number of plugins recommended in the installation wizard.
I spent some time studying PCT and finally created a pull-request with the documentation update and Docker image. It should simplify the custom PCT runs. If anybody is interested, the pull request is here: https://github.com/jenkinsci/plugin-compat-tester/pull/53 BR, Oleg среда, 10 мая 2017 г., 9:08:25 UTC+2 пользователь [email protected] написал: > > Hi Oliver, > > As Jesse comments: > > - We are trying to be as clear as possible on that kind of PRs, so > that both the problem and the fix can be understood and acknowledged by > the > maintainer and whoever might evaluate such PR. > - It's not easy to directly translate what we are doing internally as > PCT is designed to use the plugins offered by UC, what would probably lead > to ever lasting executions. Also we tight PCT runs to a certain set of > core > and plugin versions. > > > Being said that, I agree that someone could miss some context or a more > in-depth explanation both from a technical (how to execute it) and a > conceptual perspective (what PCT intends to be useful for). So we will try > to: > > - Put in place such information so that at least people can understand > better the whole PCT itself. > - Analyze what needs to be done in PCT to allow executing it with the > proper conditions (for example a set of plugins) and put in place some > work > to make it feasible. > > > Thanks! > > On Tuesday, May 9, 2017 at 3:46:10 PM UTC+2, Jesse Glick wrote: >> >> On Tue, May 9, 2017 at 4:11 AM, Oliver Gondža <[email protected]> wrote: >> > a PR that purports to fix >> > something you have no evidence of being broken referring to a TLA-named >> > testsuite you have never heared of and have no easy way to checking for >> > yourself >> >> FWIW I have asked people filing such PRs to include some background >> information as well as instructions to reproduce the failure locally. >> In most (not all) cases this comes down to something like >> >> mvn clean test-compile && mvn -Djenkins.version=2.x.y surefire:test >> >> or even >> >> mvn -Djenkins.version=2.x.y clean test >> >> The last can be kept passing by writing your `Jenkinsfile` as >> >> buildPlugin(jenkinsVersions: [null, '2.x.y']) >> >> for whatever LTS line. There are also some cases where failures are >> due to plugin-to-plugin dependency bumps, which `buildPlugin` does not >> currently support as a testing option. >> >> As to whether there can be a public Jenkins job running precisely the >> same set of tests as CloudBees runs internally, I am doubtful, since >> the particular set of core & plugin versions being tested is defined >> by the versions promoted in CloudBees products, and having this as a >> scheduled job would make little sense since a test run is tied to a >> change to that revision matrix. >> >> Now what *would* be extremely useful, if someone cares to take the >> time to set it up and maintain it, is a job on ci.jenkins.io which >> periodically (daily?) runs the PCT against some defined but >> ever-increasing (transitively closed) set of plugins, using trunk >> snapshot versions of core and all included plugins. If failures are >> evaluated and assigned promptly, this would alert developers to >> compatibility regressions they need to either fix or acknowledge, >> before releases are even made (albeit after faulty PRs are merged—a >> harder problem to solve I think). >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/6a4fdfd3-a8cb-4245-ab06-eab932abfa08%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
