Re: Speeding up JenkinsRule tests?

2016-09-29 Thread Stephen Connolly
On Thursday 29 September 2016, Jesse Glick wrote: > On Thu, Sep 29, 2016 at 1:30 AM, Stephen Connolly > > wrote: > > one has @ClassRule and has the > > more unity tests. […] > > For example, they could be actual unit tests (no

Re: Speeding up JenkinsRule tests?

2016-09-29 Thread Jesse Glick
On Thu, Sep 29, 2016 at 1:30 AM, Stephen Connolly wrote: > one has @ClassRule and has the > more unity tests. […] For example, they could be actual unit tests (no `JenkinsRule`) if it were not for the fact that something in the call chain is loading

Re: Speeding up JenkinsRule tests?

2016-09-28 Thread Stephen Connolly
You do need to know what you are doing to use @ClassRule Typically I end up with *two* test classes, one has @ClassRule and has the more unity tests. The other has @Rule with the more integrationy tests. If your tests don't clean up after themselves, @ClassRule is not for you. But very often I

Re: Speeding up JenkinsRule tests?

2016-09-28 Thread Gavin Mogan
Yea I just tried it, it made a bunch of my tests way faster but ran into a strange one on the pipeline tests java.lang.IllegalStateException: Jenkins has not been started, or was already shut down at com.saucelabs.jenkins.pipeline.SauceStepTest.sauceTest(SauceStepTest.java:53) So not a drop

Re: Speeding up JenkinsRule tests?

2016-09-27 Thread Jesse Glick
On Mon, Sep 26, 2016 at 4:37 PM, Andrew Bayer wrote: > turning the JenkinsRule into a @ClassRule rather than a @Rule Be very, very careful. You can easily get yourself into trouble this way. -- You received this message because you are subscribed to the Google Groups

Re: Speeding up JenkinsRule tests?

2016-09-26 Thread Stephen Connolly
Yes that is the exact use case that drove me to write JenkinsRule in the first place... the ability to use @ClassRule for the tests that do not need the instance wiped clean in-between On 26 September 2016 at 21:40, Andrew Bayer wrote: > Yipes! Even better for the full

Re: Speeding up JenkinsRule tests?

2016-09-26 Thread Andrew Bayer
Yipes! Even better for the full test run - from 51m47s end-to-end on ci.jenkins.io to 6m8s with @ClassRule. Looks like that actually is the magic bullet here. A. On Mon, Sep 26, 2016 at 1:37 PM, Andrew Bayer wrote: > huh, initial experiments with turning the JenkinsRule

Re: Speeding up JenkinsRule tests?

2016-09-26 Thread Andrew Bayer
huh, initial experiments with turning the JenkinsRule into a @ClassRule rather than a @Rule have been...surprisingly positive. 4+x decrease in one of the parameterized tests (from 95s to 21s). I'll experiment more in that direction. A. On Mon, Sep 26, 2016 at 1:24 PM, Andrew Bayer

Speeding up JenkinsRule tests?

2016-09-26 Thread Andrew Bayer
My tests in https://github.com/jenkinsci/pipeline-model-definition-plugin take...an ungodly amount of time. In part, that's 'cos I'm doing a *lot* of tests, and I'm using parameterized tests to make sure that failures are clear in some cases. But I gotta feel like things could be sped up. Does