In Jira as:
<http://hackydev.ics.hawaii.edu:8080/browse/HACK-772>
I am going to get to this REAL SOON NOW.
Cheers,
Philip
--On Wednesday, August 23, 2006 6:26 PM -1000 Aaron Kagawa
<[EMAIL PROTECTED]> wrote:
Hey Guys,
Austen and I found a CRITICAL bug in the Junit Sensor. Check the
following evidence.
#######################################
1) Here is a log of junit and the junit sensor execution from
StackyHack. Note, that i added a few fake temporary tests to demonstrate
the problem. Everything seems fine with the log.
C:\java\svn-csdl\StackyHack>ant -f junit.build.xml
Buildfile: junit.build.xml
compile:
[javac] Compiling 1 source file to
C:\java\svn-csdl\StackyHack\build\classes
junit.tool:
[junit] Running edu.hawaii.stack.TestClearStack
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.062 sec
[junit] Running edu.hawaii.stack.TestStack
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 0 sec
junit.report:
[junitreport] Transform time: 672ms
junit.sensor:
[hacky-junit] Sensor enabled?: true
[hacky-junit] Processing file:
C:\java\svn-csdl\StackyHack\build\junit\TEST-edu.hawaii.stack.TestClearSt
ack.xml [hacky-junit] Processing file:
C:\java\svn-csdl\StackyHack\build\junit\TEST-edu.hawaii.stack.TestStack.x
ml [hacky-junit] Hackystat data on 9 JUnit tests sent to
http://hackystat.ics.hawaii.edu/ (0 secs.)
junit:
BUILD SUCCESSFUL
Total time: 4 seconds
C:\java\svn-csdl\StackyHack>
#######################################
2) Here is the junit.0.log file from the .hackystat/logs directory. Again
everything looks okay here.
Hackystat Version: 7.5.818 (August 18 2006 13:30:59)
SensorShell started at: 08/23/2006 18:07:21
Type 'help' for a list of commands.
Host: http://hackystat.ics.hawaii.edu/ is available and key is valid.
Defined shell command: Dependency
Defined shell command: Issue
Defined shell command: EvolSdt
Defined shell command: CodeIssue
Defined shell command: Perf
Defined shell command: FileMetric
Defined shell command: ReviewIssue
Defined shell command: Activity
Defined shell command: Cli
Defined shell command: ReviewActivity
Defined shell command: Coverage
Defined shell command: DevEvent
Defined shell command: UnitTest
Defined shell command: BuffTrans
Defined shell command: Commit
Defined shell command: Build
#> AutoSend [10]
AutoSend OK (set to 10 minutes)
AutoSend enabled every 10 minutes.
Checking for offline data to recover.
No offline data found.
#> UnitTest [set, tool=JUnit]
set OK
#> UnitTest [add,
#> name=edu.hawaii.stack.TestClearStack.testNormalOperation,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestClearStack.ja
#> va, result=pass,
#> pMap=000600011elapsedTime00001000008testCase00019testNormalOperation000
#> 08testName00031edu.hawaii.stack.TestClearStack]
UnitTest add OK (1 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testNormalOperation,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00019testNormalOperation000
#> 08testName00026edu.hawaii.stack.TestStack]
UnitTest add OK (2 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testIllegalPop,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00014testIllegalPop00008tes
#> tName00026edu.hawaii.stack.TestStack]
UnitTest add OK (3 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testTemp1,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00009testTemp100008testName
#> 00026edu.hawaii.stack.TestStack]
UnitTest add OK (4 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testTemp2,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00009testTemp200008testName
#> 00026edu.hawaii.stack.TestStack]
UnitTest add OK (5 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testTemp3,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00009testTemp300008testName
#> 00026edu.hawaii.stack.TestStack]
UnitTest add OK (6 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testTemp4,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00009testTemp400008testName
#> 00026edu.hawaii.stack.TestStack]
UnitTest add OK (7 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testTemp5,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00009testTemp500008testName
#> 00026edu.hawaii.stack.TestStack]
UnitTest add OK (8 total)
#> UnitTest [add, name=edu.hawaii.stack.TestStack.testTemp6,
#> path=C:\java\svn-csdl\StackyHack\src\edu\hawaii\stack\TestStack.java,
#> result=pass,
#> pMap=000600011elapsedTime00001000008testCase00009testTemp600008testName
#> 00026edu.hawaii.stack.TestStack]
UnitTest add OK (9 total)
#> send
Sending sensor data (08/23 18:07:22)
Ping: Ping OK (contacted server http://hackystat.ics.hawaii.edu/ with
valid key.) Dependency: Send OK (No entries to send.)
AutoSend: AutoSend OK ('send' command ignored)
Issue: Send OK (No entries to send.)
EvolSdt: Send OK (No entries to send.)
CodeIssue: Send OK (No entries to send.)
Perf: Send OK (No entries to send.)
FileMetric: Send OK (No entries to send.)
ReviewIssue: Send OK (No entries to send.)
Activity: Send OK (No entries to send.)
Cli: Send OK (No entries to send.)
ReviewActivity: Send OK (No entries to send.)
DevEvent: Send OK (No entries to send.)
Coverage: Send OK (No entries to send.)
Commit: Send OK (No entries to send.)
BuffTrans: Send OK (No entries to send.)
UnitTest: Send OK (9 entries)
Build: Send OK (No entries to send.)
>>
#######################################
3) Now lets take a look at the UnitTest data in the List Sensor Data
analysis: http://csdl.ics.hawaii.edu/~kagawaa/UnitTestSensorData.JPG Note
that the last three entries are associated with the execution in the log
and sensor log. So, you can see that we have only three entries but we
should have 9 entries!
I should have data for the following methods.
TestStack.testNormalOperation()
TestStack.testIllegalPop()
TestStack.testTemp1()
TestStack.testTemp2()
TestStack.testTemp3()
TestStack.testTemp4()
TestStack.testTemp5()
TestStack.testTemp6()
But, I only have these:
TestStack.testTemp5()
TestStack.testTemp6()
TestStack.testNormalOperation()
So, I thought there might be a bug in the List Sensor Data analysis.
But, I checked the Daily Project Details
(http://csdl.ics.hawaii.edu/~kagawaa/DailyProjectDetails-StackyHack.JPG)
and it shows only 6 unit tests but it should show 12 (3 + 9 from the last
run).
It seems that the maximum amount of UnitTest entries is two per Java
file. Meaning two different name values for the same path value.
This isn't the only example that I've seen. Austen and I seen this
problem in two different cases in our Company's data. And it is
reproducible with StackHack. So, it seems that there is a CRITICAL bug in
the Junit sensor or something that is receiving the data.
thanks, Aaron
