Re: Failsafe test reports for my Jenkins plugin are ignored on ci.jenkins.io

2019-05-15 Thread Chris Kilding 
One pull request coming up...

https://github.com/jenkins-infra/pipeline-library/pull/89

I have a couple of questions about it so I’ve marked it as a draft PR and 
posted them underneath.

Chris

On Wed, 15 May 2019, at 9:54 AM, Robert Sandell wrote:
> That repository is open to pull requests ;)
> 
> /B
> 
> Den ons 15 maj 2019 kl 10:39 skrev Ullrich Hafner :
>> I simply copied the buildPlugin step content into my JenkinsFile and 
>> modified it accordingly. This was also the only way to enable the warnings 
>> next generation plugin and some other tools (like coverage and pitest). 
>> 
>> Examples:
>> https://github.com/jenkinsci/analysis-model/blob/master/Jenkinsfile
>> https://github.com/jenkinsci/warnings-ng-plugin/blob/master/Jenkinsfile
>> 
>> 
>>> Am 14.05.2019 um 19:42 schrieb Chris Kilding 
>>> :
>>> 
>>> The standard ‘buildPlugin()’ Jenkinsfile directive appears to capture 
>>> Surefire test reports, but not Failsafe test reports. This means that 
>>> failing integration tests in my plugin are (a) unreported and (b) don’t 
>>> break the build. This might be happening to other plugins too.
>>> 
>>> The line of code in question is probably this: 
>>> https://github.com/jenkins-infra/pipeline-library/blob/66a14842e350bd5364a1a70ca4287535cacc0390/vars/buildPlugin.groovy#L118
>>> 
>>> I’ve worked around it by saving the Failsafe reports to the Surefire 
>>> directory but this is extremely fragile: if I have an integration test 
>>> class with the same name and package as a unit test class (which could 
>>> happen if integration tests are stored in a different test source 
>>> directory), then the report for one could overwrite the other.
>>> 
>>> Could someone confirm whether this is indeed a limitation in the current 
>>> buildPlugin.groovy script or whether there is a different (robust) way to 
>>> pick up Failsafe reports on ci.jenkins.io?
>>> 
>>> Regards
>>> 
>>> Chris
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/jenkinsci-dev/448e67fa-aef8-48bd-9ce7-cdbf7a7d5b7f%40www.fastmail.com
>>>  
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>> 
>> 

>> --
>>  You received this message because you are subscribed to the Google Groups 
>> "Jenkins Developers" group.
>>  To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>>  To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-dev/465C7830-6BA3-4EEA-B3BE-964F455E9F3D%40gmail.com
>>  
>> .
>>  For more options, visit https://groups.google.com/d/optout.
> 
> 
> -- 
> *Robert Sandell*
> Software Engineer
> CloudBees, Inc.
> CloudBees-Logo.png 
> E: rsand...@cloudbees.com
> Twitter: robert_sandell
> 

> --
>  You received this message because you are subscribed to the Google Groups 
> "Jenkins Developers" group.
>  To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>  To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/CALzHZS3Zzxdx0eT2FpCPAY6OyaWg24-T-98FyJaiFgBZrXsV%2BQ%40mail.gmail.com
>  
> .
>  For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/61137c39-9b54-41b7-a28d-ea18413bd751%40www.fastmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Failsafe test reports for my Jenkins plugin are ignored on ci.jenkins.io

2019-05-15 Thread Robert Sandell 
That repository is open to pull requests ;)

/B

Den ons 15 maj 2019 kl 10:39 skrev Ullrich Hafner :

> I simply copied the buildPlugin step content into my JenkinsFile and
> modified it accordingly. This was also the only way to enable the warnings
> next generation plugin and some other tools (like coverage and pitest).
>
> Examples:
> https://github.com/jenkinsci/analysis-model/blob/master/Jenkinsfile
> https://github.com/jenkinsci/warnings-ng-plugin/blob/master/Jenkinsfile
>
>
> Am 14.05.2019 um 19:42 schrieb Chris Kilding <
> chris+jenk...@chriskilding.com>:
>
> The standard ‘buildPlugin()’ Jenkinsfile directive appears to capture
> Surefire test reports, but not Failsafe test reports. This means that
> failing integration tests in my plugin are (a) unreported and (b) don’t
> break the build. This might be happening to other plugins too.
>
> The line of code in question is probably this:
> https://github.com/jenkins-infra/pipeline-library/blob/66a14842e350bd5364a1a70ca4287535cacc0390/vars/buildPlugin.groovy#L118
>
> I’ve worked around it by saving the Failsafe reports to the Surefire
> directory but this is extremely fragile: if I have an integration test
> class with the same name and package as a unit test class (which could
> happen if integration tests are stored in a different test source
> directory), then the report for one could overwrite the other.
>
> Could someone confirm whether this is indeed a limitation in the current
> buildPlugin.groovy script or whether there is a different (robust) way to
> pick up Failsafe reports on ci.jenkins.io?
>
> Regards
>
> Chris
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/448e67fa-aef8-48bd-9ce7-cdbf7a7d5b7f%40www.fastmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/465C7830-6BA3-4EEA-B3BE-964F455E9F3D%40gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
*Robert Sandell*
Software Engineer
CloudBees, Inc.
[image: CloudBees-Logo.png] 
E: rsand...@cloudbees.com
Twitter: robert_sandell

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CALzHZS3Zzxdx0eT2FpCPAY6OyaWg24-T-98FyJaiFgBZrXsV%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Failsafe test reports for my Jenkins plugin are ignored on ci.jenkins.io

2019-05-15 Thread Ullrich Hafner 
I simply copied the buildPlugin step content into my JenkinsFile and modified 
it accordingly. This was also the only way to enable the warnings next 
generation plugin and some other tools (like coverage and pitest). 

Examples:
https://github.com/jenkinsci/analysis-model/blob/master/Jenkinsfile 

https://github.com/jenkinsci/warnings-ng-plugin/blob/master/Jenkinsfile 



> Am 14.05.2019 um 19:42 schrieb Chris Kilding :
> 
> The standard ‘buildPlugin()’ Jenkinsfile directive appears to capture 
> Surefire test reports, but not Failsafe test reports. This means that failing 
> integration tests in my plugin are (a) unreported and (b) don’t break the 
> build. This might be happening to other plugins too.
> 
> The line of code in question is probably this: 
> https://github.com/jenkins-infra/pipeline-library/blob/66a14842e350bd5364a1a70ca4287535cacc0390/vars/buildPlugin.groovy#L118
>  
> 
> 
> I’ve worked around it by saving the Failsafe reports to the Surefire 
> directory but this is extremely fragile: if I have an integration test class 
> with the same name and package as a unit test class (which could happen if 
> integration tests are stored in a different test source directory), then the 
> report for one could overwrite the other.
> 
> Could someone confirm whether this is indeed a limitation in the current 
> buildPlugin.groovy script or whether there is a different (robust) way to 
> pick up Failsafe reports on ci.jenkins.io ?
> 
> Regards
> 
> Chris
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-dev+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/448e67fa-aef8-48bd-9ce7-cdbf7a7d5b7f%40www.fastmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/465C7830-6BA3-4EEA-B3BE-964F455E9F3D%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Failsafe test reports for my Jenkins plugin are ignored on ci.jenkins.io

2019-05-14 Thread Chris Kilding 
The standard ‘buildPlugin()’ Jenkinsfile directive appears to capture Surefire 
test reports, but not Failsafe test reports. This means that failing 
integration tests in my plugin are (a) unreported and (b) don’t break the 
build. This might be happening to other plugins too.

The line of code in question is probably this: 
https://github.com/jenkins-infra/pipeline-library/blob/66a14842e350bd5364a1a70ca4287535cacc0390/vars/buildPlugin.groovy#L118

I’ve worked around it by saving the Failsafe reports to the Surefire directory 
but this is extremely fragile: if I have an integration test class with the 
same name and package as a unit test class (which could happen if integration 
tests are stored in a different test source directory), then the report for one 
could overwrite the other.

Could someone confirm whether this is indeed a limitation in the current 
buildPlugin.groovy script or whether there is a different (robust) way to pick 
up Failsafe reports on ci.jenkins.io?

Regards

Chris

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/448e67fa-aef8-48bd-9ce7-cdbf7a7d5b7f%40www.fastmail.com.
For more options, visit https://groups.google.com/d/optout.