Ryan,
Forgive the following dissertation on metrics. I hope that you find it to
be beneficial.
Useful metrics differ from one organization to the next. How does one
determine a set of metrics that are meaningful for his or her organization?
I'm a big advocate of Victor Basili's "GQM" metrics collection paradigm, in
which:
- The organization first sets Goals (the "G"). A goal might be something
like, "We will have no more than one released defect per thousand lines of
new source code or per thousand lines of existing source code that are
changed in the current release". BTW, 1 defect per 1000 LOC is
**difficult** and expensive to achieve. The norm in commercial software is
around 10 to 14 defects per 1000 LOC :-(
- Questions (the "Q") are defined that, when answered, will identify
whether the goals have been achieved
- Measurements (the "M") are defined that, when taken and analyzes, will
help provide the answers to the questions.
GQM is a highly pragmatic approach to metrics that helps ensure that the
measurement program which is executed actually provides value to the
organization. One thing to note here is that most organizations will have
different Goals ("G"), so they will need to answer different Questions
("Q"), which requires different Measures ("M").
BTW, Basili is a professor at the University of Maryland. His team's work
has helped Goddard Spaceflight Center become one of the most effective
software development labs in the world.
Metrics provide information that help managers and developers determine a
couple of things:
- How they are performing vs. historical norms. As an example:
+ An organization's successful projects in the past have been
characterized by a value for Metric X that is in the range of 1 to 3
+ Unsuccessful projects have been characterized by a value for Metrix X
that is above this range
+ The current project's value for X is 4.
+ Watch out!
- How tell-tale performance metrics are trending. If a project's value of
X has steadily trended over the past 4 weeks from 1.5 to the current value
of 2.8, again watch out! Something might be breaking.
Here is the kicker: in order for metrics to be useful to you, you must have
a baseline **for your local projects** against which the acceptability of
current performance can be assessed. Someone else's baseline generally is
of limited value. You can't go to some book or paper, see how someone else
has correlated Metric X to some project success factor such as cost per line
of code, and use that blindly. What you have to do is extract pertinent
metrics from your **own** past projects and develop correlations vs. your
**own** success factors. Once you have your own local correlations, you can
use these to guide your assessment of the health of the current project.
Ok, ok, I know that I haven't addressed your original question, which is
along the lines of "What metrics have been proven useful in assessing the
value of software modeling?" To the best of my knowledge, there is no
definitive answer to this question. However, some metrics have been
proposed that appear to relate to project characteristics that matter:
- Robert C. Martin's Relational Cohesion, Afferent Coupling, Efferent
Coupling, Abstractness, and Instability metrics appear to me to be quite
valuable. These can be used to assess the degree of coupling between
subsystems in a larger system. They also can be used to assess the likely
degree of impact on system design and maintenance if the classes that
compose a given subsytem are altered. BTW, these metrics were first
publically defined in Martin's book, "Designing Object-Oriented C++
Applications Using The Booch Method". They were revisited in several
articles that Martin published in "The C++ Report" a few years ago. Values
of these metrics can be correlated vs. project success metrics, such as cost
to implement a change in a given module. Once these correlations are
available, you can use them to help assess the health of your project, to
estimate the cost of planned project changes, etc.
- One of the more well-known papers on OO metrics is "A Metrics Suite for
Object-Oriented Design", by Chidamber and Kemerer of MIT. Their metrics
pertain to detailed aspects of the static structure of an object model, such
as depth of inheritance hierarchy, weighted methods per class, number of
children, etc. Once more, these metrics have value once you develop local
correlations between your values of these and your project success metrics.
As a final note, I suggest that you visit the Software Engineering
Institute's Web site at http://www.sei.cmu.edu/. The SEI has done a **lot**
of work on metrics. There are several reports, many in PDF format, that
contain information on OO systems and metrics.
Todd Dunnavant
Rational Software Corporation
Technical Lead, Texas/Oklahoma Geographic District
voice: 281-499-8789
email: [EMAIL PROTECTED]
-----Original Message-----
From: Parker, Ryan (GXS) [mailto:[EMAIL PROTECTED]]
Sent: Sunday, February 04, 2001 4:59 PM
To: '[EMAIL PROTECTED]'
Subject: (ROSE) Modeling Metrics
All,
Does anyone know of any metrics that can be collected to verify the
many theoretical benefits of software modeling?
Thanks,
Ryan Parker
************************************************************************
* Rose Forum is a public venue for ideas and discussions.
* For technical support, visit http://www.rational.com/support
*
* Admin.Subscription Requests: [EMAIL PROTECTED]
* Archive of messages:
http://www.rational.com/products/rose/usergroups/rose_forum.jtmpl
* Other Requests: [EMAIL PROTECTED]
*
* To unsubscribe from the list, please send email
*
* To: [EMAIL PROTECTED]
* Subject:<BLANK>
* Body: unsubscribe rose_forum
*
*************************************************************************
************************************************************************
* Rose Forum is a public venue for ideas and discussions.
* For technical support, visit http://www.rational.com/support
*
* Admin.Subscription Requests: [EMAIL PROTECTED]
* Archive of messages:
http://www.rational.com/products/rose/usergroups/rose_forum.jtmpl
* Other Requests: [EMAIL PROTECTED]
*
* To unsubscribe from the list, please send email
*
* To: [EMAIL PROTECTED]
* Subject:<BLANK>
* Body: unsubscribe rose_forum
*
*************************************************************************
