Re: Speeding up JenkinsRule tests?

2016-09-29 Thread Stephen Connolly 
On Thursday 29 September 2016, Jesse Glick  wrote:

> On Thu, Sep 29, 2016 at 1:30 AM, Stephen Connolly
> > wrote:
> > one has @ClassRule and has the
> > more unity tests. […]
>
> For example, they could be actual unit tests (no `JenkinsRule`) if it
> were not for the fact that something in the call chain is loading
> `@Extension`s.
>
> > If your tests don't clean up after themselves, @ClassRule is not for you.
>
> I would go so far as to say that if you are tests *have* anything to
> clean up—created `Item`s, for example—using `@ClassRule JenkinsRule`
> is too risky.
>
>
As a principal of advice, yes. But for somebody that is looking to push the
edge *and* likes driving their car at speed without a seatbelt and with a
12" metal spike protruding from the steering wheel, the option is there and
it's not in and of itself bad...


> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com .
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/CANfRfr2A2FYdtx35p09VtCcXAH-CsDHj6iqk9m1noZigGjWKNQ%
> 40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Sent from my phone

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMynHyNrw8eNfA5Y0t-4tH4bWV_BRPx_9Qn_fJ%2BstQPJ2w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-29 Thread Jesse Glick 
On Thu, Sep 29, 2016 at 1:30 AM, Stephen Connolly
 wrote:
> one has @ClassRule and has the
> more unity tests. […]

For example, they could be actual unit tests (no `JenkinsRule`) if it
were not for the fact that something in the call chain is loading
`@Extension`s.

> If your tests don't clean up after themselves, @ClassRule is not for you.

I would go so far as to say that if you are tests *have* anything to
clean up—created `Item`s, for example—using `@ClassRule JenkinsRule`
is too risky.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2A2FYdtx35p09VtCcXAH-CsDHj6iqk9m1noZigGjWKNQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-28 Thread Stephen Connolly 
You do need to know what you are doing to use @ClassRule

Typically I end up with *two* test classes, one has @ClassRule and has the
more unity tests. The other has @Rule with the more integrationy tests.

If your tests don't clean up after themselves, @ClassRule is not for you.
But very often I have a large chunk of tests that would not cross interact
and @ClassRule is perfect for them

On Wednesday 28 September 2016, Gavin Mogan  wrote:

> Yea I just tried it, it made a bunch of my tests way faster but ran into a
> strange one on the pipeline tests
>
> java.lang.IllegalStateException: Jenkins has not been started, or was
> already shut down
> at com.saucelabs.jenkins.pipeline.SauceStepTest.
> sauceTest(SauceStepTest.java:53)
>
> So not a drop in replacement at all :(
>
> On Tuesday, September 27, 2016 at 7:00:49 AM UTC-7, Jesse Glick wrote:
>>
>> On Mon, Sep 26, 2016 at 4:37 PM, Andrew Bayer 
>> wrote:
>> > turning the JenkinsRule into a @ClassRule rather than a @Rule
>>
>> Be very, very careful. You can easily get yourself into trouble this way.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com
> 
> .
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/248236ce-0e2f-4bd1-8c74-a25a2b5da182%
> 40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Sent from my phone

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxiao6HTMZMCvLx9zP8_oJsSoehiZXWvZaFsG%2BM3eBwHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-28 Thread Gavin Mogan 
Yea I just tried it, it made a bunch of my tests way faster but ran into a 
strange one on the pipeline tests

java.lang.IllegalStateException: Jenkins has not been started, or was 
already shut down
at 
com.saucelabs.jenkins.pipeline.SauceStepTest.sauceTest(SauceStepTest.java:53)

So not a drop in replacement at all :(

On Tuesday, September 27, 2016 at 7:00:49 AM UTC-7, Jesse Glick wrote:
>
> On Mon, Sep 26, 2016 at 4:37 PM, Andrew Bayer  > wrote: 
> > turning the JenkinsRule into a @ClassRule rather than a @Rule 
>
> Be very, very careful. You can easily get yourself into trouble this way. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/248236ce-0e2f-4bd1-8c74-a25a2b5da182%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-27 Thread Jesse Glick 
On Mon, Sep 26, 2016 at 4:37 PM, Andrew Bayer  wrote:
> turning the JenkinsRule into a @ClassRule rather than a @Rule

Be very, very careful. You can easily get yourself into trouble this way.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1vfLJ%3D7f9EkRkv3ETfF_5v-EYpihR4Mz%2BRBbor2pyfjQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-26 Thread Stephen Connolly 
Yes that is the exact use case that drove me to write JenkinsRule in the
first place... the ability to use @ClassRule for the tests that do not need
the instance wiped clean in-between

On 26 September 2016 at 21:40, Andrew Bayer  wrote:

> Yipes! Even better for the full test run - from 51m47s end-to-end on
> ci.jenkins.io to 6m8s with @ClassRule. Looks like that actually is the
> magic bullet here.
>
> A.
>
> On Mon, Sep 26, 2016 at 1:37 PM, Andrew Bayer 
> wrote:
>
>> huh, initial experiments with turning the JenkinsRule into a @ClassRule
>> rather than a @Rule have been...surprisingly positive. 4+x decrease in one
>> of the parameterized tests (from 95s to 21s). I'll experiment more in that
>> direction.
>>
>> A.
>>
>> On Mon, Sep 26, 2016 at 1:24 PM, Andrew Bayer 
>> wrote:
>>
>>> My tests in https://github.com/jenkinsci/pipeline-model-definition-pl
>>> ugin take...an ungodly amount of time. In part, that's 'cos I'm doing a
>>> *lot* of tests, and I'm using parameterized tests to make sure that
>>> failures are clear in some cases. But I gotta feel like things could be
>>> sped up. Does anyone know of any obvious tricks with JenkinsRule that can
>>> speed things up for a bunch of tests in a single class?
>>>
>>> A.
>>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/CAPbPdOaWfXQ3vW1LQbSqFTj3NfL24
> MZScM6%2B%3DLwLPHuxy9A5_A%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMw8NhO69%3DMBT8FzfV3PkfktjqrCVaN7h5ahH9UPpYQ3zA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-26 Thread Andrew Bayer 
Yipes! Even better for the full test run - from 51m47s end-to-end on
ci.jenkins.io to 6m8s with @ClassRule. Looks like that actually is the
magic bullet here.

A.

On Mon, Sep 26, 2016 at 1:37 PM, Andrew Bayer 
wrote:

> huh, initial experiments with turning the JenkinsRule into a @ClassRule
> rather than a @Rule have been...surprisingly positive. 4+x decrease in one
> of the parameterized tests (from 95s to 21s). I'll experiment more in that
> direction.
>
> A.
>
> On Mon, Sep 26, 2016 at 1:24 PM, Andrew Bayer 
> wrote:
>
>> My tests in https://github.com/jenkinsci/pipeline-model-definition-plugin
>> take...an ungodly amount of time. In part, that's 'cos I'm doing a *lot* of
>> tests, and I'm using parameterized tests to make sure that failures are
>> clear in some cases. But I gotta feel like things could be sped up. Does
>> anyone know of any obvious tricks with JenkinsRule that can speed things up
>> for a bunch of tests in a single class?
>>
>> A.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdOaWfXQ3vW1LQbSqFTj3NfL24MZScM6%2B%3DLwLPHuxy9A5_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Speeding up JenkinsRule tests?

2016-09-26 Thread Andrew Bayer 
huh, initial experiments with turning the JenkinsRule into a @ClassRule
rather than a @Rule have been...surprisingly positive. 4+x decrease in one
of the parameterized tests (from 95s to 21s). I'll experiment more in that
direction.

A.

On Mon, Sep 26, 2016 at 1:24 PM, Andrew Bayer 
wrote:

> My tests in https://github.com/jenkinsci/pipeline-model-definition-plugin
> take...an ungodly amount of time. In part, that's 'cos I'm doing a *lot* of
> tests, and I'm using parameterized tests to make sure that failures are
> clear in some cases. But I gotta feel like things could be sped up. Does
> anyone know of any obvious tricks with JenkinsRule that can speed things up
> for a bunch of tests in a single class?
>
> A.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdOYKqVK9yL5NDcBUMOHAXBVza2X7yTk0QutSAYpwWqaYhQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Speeding up JenkinsRule tests?

2016-09-26 Thread Andrew Bayer 
My tests in https://github.com/jenkinsci/pipeline-model-definition-plugin
take...an ungodly amount of time. In part, that's 'cos I'm doing a *lot* of
tests, and I'm using parameterized tests to make sure that failures are
clear in some cases. But I gotta feel like things could be sped up. Does
anyone know of any obvious tricks with JenkinsRule that can speed things up
for a bunch of tests in a single class?

A.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdOY_uaJgZKpD_nZ93dcj2BpEMUWKmKmzTe2idGZF9DLpCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.