Hi Aaron, You have identified a significant quality assurance problem. One thing that would help would be to test the StackyHack system automatically as part of the nightly build process.
The sensor documentation also varies a lot in quality, and you're right, sometimes updates get made without updating the documentation. I will look into this today and see if I can at least fix the Emma problem. The broader problem you identify requires a broader solution. Maybe we need to do documentation inspections. Cheers, Philip p.s. Incidentally, I have spent the past week doing a substantial re-write of the telemetry documentation, which was also very out of date. There is still more to add in some places, but at least the documentation that exists should now be in sync with the system. ----- Original Message ----- From: Aaron Kagawa <[EMAIL PROTECTED]> Date: Friday, May 12, 2006 9:09 am Subject: [HACKYSTAT-DEV-L] Emma Sensor Documentation Busted To: [email protected] > Hey Guys, > > The Emma sensor in StackyHack (and in my own use of it at my > company) > is busted. See the following error report at the end of the email. > > Furthermore, the DocBook is busted as well. I say that because the > DocBook for the Emma Sensor is old. What is worse is if someone > follows those directions they will get the same error that I got. > > Ok.. So, you all should know that the sensor is not actually > busted. It is working on HackyDev. The only reason I was able to > figure out the problem was by digging through the Emma Java and > emma.build.xml source code. Thus, it seems that the Emma Sensor > Documentation is busted, not the actual implementation. But, in my > opinion Implementation and Documentation are one and the same. > > Lets not forget that we have outside users of Hackystat. When I > started to debug this problem I immediately went to Jira and tried > to > search for hackySensor_Emma. I found only two issues that > contained > hackySensor_Emma in the workspace field. (Hm.. there must be much > more development on that sensor than that shows.). So, I dug into > the first issue, http://hackydev.ics.hawaii.edu:8080/browse/HACK- > 648 > (Evolve Coverage SDT). This issue contained did not mention a > change > to the sensor that would explain the error. So, as an "outside" > user > I continued to wonder what and why there was a change to the sensor. > > Moral of the Story: > 1) Someone must update the Emma Sensor DocBook chapter. Include > those cool little number dots to explain the Ant interface for the > Emma Sensor. I noticed that a couple of the DocBook chapters for > sensors were missing those. How would an outside user understand > how > to configure the sensor without that? > 2) We need to be more verbose in Jira comments. Explain what the > problem is, why it is a problem, and how it was fixed. Obviously > this verbosity is not for yourself; it allows the outside users to > understand what is changing. Bottom line: the worst thing you can > put > in the comment field is "fixed". > 3) Lastly, making a change to the system is not limited to just > changing the software code. A successful change is made when an > outside users can use the software without significant interaction > with a system developer (haha! the three prime directives strike > again). Therefore, a Jira issue that affects the "user interface" > of > Hackystat should not be closed until that implementation change is > reflected in Jira and in the DocBook. > > thanks, aaron > > [Error Report Below] > Tried these two versions of the Emma Sensor: > Emma Sensor 7.4.427 > Emma Sensor 7.4.511 > > emma.sensor: > [hacky-emma] Sensor enabled?: true > [hacky-emma] Processing emma report file: null > [hacky-emma] java.lang.NullPointerException > [hacky-emma] at java.io.File.<init>(File.java:194) > [hacky-emma] at > org.hackystat.sensor.emma.EmmaSensor.processCoverageXmlFile(EmmaSensor.java:203) > [hacky-emma] at > org.hackystat.sensor.emma.EmmaSensor.execute(EmmaSensor.java:178) > [hacky-emma] at > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275) > [hacky-emma] at org.apache.tools.ant.Task.perform(Task.java:364) > [hacky-emma] at > org.apache.tools.ant.Target.execute(Target.java:341)[hacky-emma] > at org.apache.tools.ant.Target.performTasks(Target.java:369) > [hacky-emma] at > org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216) > [hacky-emma] at > org.apache.tools.ant.Project.executeTarget(Project.java:1185) > [hacky-emma] at > org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40) > [hacky-emma] at > org.apache.tools.ant.Project.executeTargets(Project.java:1068) > [hacky-emma] at org.apache.tools.ant.Main.runBuild(Main.java:668) > [hacky-emma] at org.apache.tools.ant.Main.startAnt(Main.java:187) > [hacky-emma] at > org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)[hacky- > emma] at > org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)[hacky- > emma] Errors occurred while processing the coverage xml file: > org.hackyst > at.sensor.emma.EmmaSensorException: Failed to process null. > java.lang.NullPointerException > [hacky-emma] Hackystat data on 0 coverage entries sent to > http://hackystat.ics.hawaii.edu/ (0 secs.) > > emma: > > BUILD SUCCESSFUL > Total time: 7 seconds > C:\java\svn-csdl\StackyHack> >
