Greetings,
I am pleased to introduce you all to Christoph Lofi, a graduate student from the University of Kaiserslauten who has come to Hawaii to surf (oops, I mean work on his thesis).
Christoph has expertise in the Goal-Question-Metric (GQM) method, perhaps the most widely adopted and well respected method for developing measurement programs in software development organizations. (Google on "goal-question-metric" to find many pages of links if you're interested in details.)
While success stories using the GQM abound, application of the method tends toward "one off" measurement projects: there are one or more specific goals, the framework is used to figure out what questions to ask and what metrics to define in order to see if the goal(s) are achieved, the metrics are then gathered for a period of time, the goal is found to be achieved (or not), and things stop. A possible reason why GQM tends to be applied this way might be because the goals, questions, and metrics tend to be specified, collected, and analyzed manually.
For Christoph's thesis, he hopes to explore the idea of "Continuous GQM" (cGQM). The idea here is to create executable specifications for the Goals and Questions, plus use Hackystat to automatically derive the Metrics using Sensors (cGQM thus extends goal-question-metric into goal-question-metric-sensor).
This is pretty cool in a few ways:
(1) While cGQM imposes a new cost of defining the Goals and Questions in an executable way, there is a new benefit that the data can be collected and analyzed (and the determination of whether the goal(s) have been met or not) automatically. This may reduce the overall cost of cGQM implementation for an organization, particularly if we can provide a 'shrink-wrapped' library of executable "Frequently Asked cGQM Questions" that organizations can pick from and use in deriving their top-level Goals.
(2) cGQM changes the nature of the answer from "a goal has been achieved (or not)" to "a goal has been achieved during this time interval (or not)". A given cGQM network can be instantiated with data at the grain-size of days, weeks, or months. You can see if you are satisfying your goals or not over any given period of time, and if not, which questions gave the wrong answers.
(3) Telemetry streams provide an extremely natural way to represent the continuous, time-sensitive nature of the goals, questions, and metrics in cGQM, and to see trends as they appear over time.
(4) Requiring the goals and questions to be executable imposes constraints on them. The nature of these constraints requires further research.
(5) In cGQM, will some sensors need to be intrusive? In other words, is it feasible to create cGQM networks in which all Metrics can be generated from automatic, unobtrusive data collection, or will we need to build sensors that end up asking the user/developer/manager to answer a question? (My thought is that in many cases, we'll end up with some kind of mixture of unobtrusive and obtrusive metrics collection, but that hopefully the obtrusive measurement occurs quite intermittently.)
So, expect to see some build failures related to hackyGQM for a while as Christoph comes up to speed. :-)
Cheers, Philip
