I had a Big Idea last night. :-)
Software Project Telemetry is, of course, totally cool. It provides a unique and very powerful organizing principle for the abstraction and presentation of automatically collected software telemetry data streams from development, build, usage, run-time, performance evaluation, and other sources.
So, what's next?
Here's my thought. What if the results of telemetry analyses could be automatically fed back into the software development process? In other words, could we create feedback loops, whereby the sensor data actually makes changes to the way software development is done without it having to pass through human interpretation first?
Let's call this layer of coordination and control "iHackystat". It sits on top of Hackystat and Software Project Telemetry.
Here are some examples of what I mean:
* Assume that there are too many tests to run all of them all the time, such as is the case with MDS. iHackystat would monitor what files are being committed and the types of changes to automatically decide which subset of tests to run.
* Assume multiple developers are working in their own local copies of the system, such as is the case with CVS. iHackystat would be receiving information about what files are being worked on and the modifications being made. If iHackystat receives information that two developers are now working on files in such a way that a merge conflict could occur when they commit, it might 'pre-fetch' the changed files from the developer's local workspaces, to a 'pre-merge' and 'pre-build', and use that to tell the developers involved that there may be upcoming problems and the nature of the problem that may be upcoming.
* Based upon the nature of run-time errors occuring in the software system, iHackystat might automatically schedule a code review, or reprioritize tasks.
* Based upon the defect removal and insertion rates, iHackystat might adjust the release date for a stable release, or recommend scope adjustment.
The over-arching principle is one of "we want development to go as fast as possible, but no faster". Some of the automatic adjustments are intended to accelerate development (for example, figuring out what subset of tests to run, or the pre-commit stuff, helps developers learn about problems earlier so they can react to them faster and the ripple effects are smaller.) Some of the automatic adjustments are intended to slow down development that has gotten "too fast" (for example, automatically scheduling a code review because quality is dipping too low and too many defects are resulting).
If we could get this kind of thing to work, it might mean that:
* A small project, that has minimal management and process stuff in place, could grow bigger before additional management infrastructure would need to be put in place.
* Management could be decentralized; projects could be split into smaller autonomous units, with iHackystat helping to ensure coordination among the pieces without as much human oversight and supervision.
* These small autonomous projects would tend to be self-organizing, in the sense that iHackystat would provide constraints or process pressures that would result in reasonable levels of coordination as an implicit result of the mechanism. Projects could start out any old way they want, but over time, they will stabilize into an organize that plays well with others.
Comments? Anyone looking for a Ph.D. topic? :-)
Cheers, Philip
