Jim Bim created JENKINS-13680:
---------------------------------
Summary: Aggregated Test Result drill down should compare the
results to builds linked to previous top-level build
Key: JENKINS-13680
URL: https://issues.jenkins-ci.org/browse/JENKINS-13680
Project: Jenkins
Issue Type: Bug
Components: core
Affects Versions: current
Environment: Jenkins ver. 1.457
Reporter: Jim Bim
Priority: Minor
(Not quite sure whether it's a bug or new feature request)
Assume I have three jobs, A, B, and C, with C being executed for both A and B
and C's test result is being aggregated to the corresponding top-level build (A
or B).
* build A#1, C#1 finishes with 10 failures
* build B#1, C#2 finishes with 0 failures
* build A#2, C#3 finishes with 10 failures
Now aggregated test result for A#2 will show 10 failures in total and drill
down will show C#3 with 10(+10) failures, because C#3 will be compared to C#2
(previous build) instead of C#1 (build corresponding to previous parent build).
This "different counting" makes the delta value misleading (one would conclude
that A#2 introduced 10 new failures in C, which is obviously not true). The
same applies for total test count, but it's not that big issue.
Real-life scenario: A and B are different branches (maintenance release and
development release) of the same software and C (and D, E, F...) are jobs that
execute tests on both these builds. Clearly, there will be a different number
of test failures in dev and maintenance branches.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
