> Date: Thu, 3 Aug 2006 08:32:07 +0100
> From: "Peter Amey" <[EMAIL PROTECTED]>
> Subject: Re: [SC-L] Cost of provably-correct code
> To: <sc-l@securecoding.org>
> Message-ID:
> <[EMAIL PROTECTED]>
> Content-Type: text/plain; charset="US-ASCII"
>
> [Re-send, I am not sure the first copy made it to the list]
>
>
> I think we have to /aim/ for zero defects and choose technical
> approaches that make that aim credible. If we don't then what defect
> rate shall we aim for and how will we know if we have achieved it?
>
>
I agree - in the industrial world defects in workmanship and manufacture
were taken for granted until product liability laws and ever-more
intensive competition drove some desperate soles (the Japanese and
Demming come to mind) to seek to differentiate themselves on the basis
of low cost through the obsessive analysis and elimination of defects.
They went from having a reputation for producing "junk electronics" to
setting the standard for high value, low cost products.
The insight was that reproduce-able processes could be incrementally
improved.
It changed many industries.
Now we have a couple of decades of experience with efforts (Xerox'
Quality Improvement Process, Malcolm Baldridge Awards, Zero-Defect,
Six-Sigma programs, etc.) to apply similar monitoring and analysis
techniques to a whole range of manufacturing, service, administration
and other industries.
But not software. Oh, no - not software. Too hard, too complex, some say.
I like that the Software Engineering Institute folks at Carnegie-Mellon
are working to identify just WHICH process in software development is
reproduce-able, so they can develop the analytical tools and processes
to control them.
So far one they've come up with is the determinedly reproduce-ably way
software developers repeat the same mistakes they make, over and over
and over again.
We're not talking buffer over flows, here - we're talking careless use
of scanf() and strcpy(), and other similar "known bad practices".
I hope they're successful promoting their programs ("Personal Software
Process" and "Team Software Process"), because I believe they'll improve
the overall quality of software produced by development organizations
that use them.
But...
I don't think incremental improvement will get you to defect-free code -
because incremental improvement is likely to be something of a
random-walk sort of search algorithm for perfection.
The engineering approach the rest of the world uses, I hope, is a little
more structured -
Do you know what you want?
Do you know what you DON'T want?
Can you verify that what you design / built will
give you what you want
not give you what you don't want?
If you can verify, through formal analysis, logical reasoning, and
inspectible process discipline (to avoid corruption by subversion), then
you can talk about something being both correct (does what it should)
and secure (doesn't do what it shouldn't).
The effects of Six Sigma processes is supposed to control defects to a
rate of 3.4 per million. In conventional software terms, that's .0034
defects per thousand lines of code, if you treat the production of a
line of code as the reproduce-able process.
Compare that to conventional wisdom of software defect rates for
commercially developed code.
Indeed, even rates of .1 defect / KLOC (which is very good) are still
2-3 orders of magnitude greater than the goals set out by other
industries for their own quality measures
(http://www.softwaretechnews.com/stn8-2/praxis.html).
But that's what I think we, in the software industry, need to begin
aiming for in order to merit calling our discipline "engineering", in my
opinion. Other industries are doing this, and they're doing it in the
real-world context of customer service departments, documentation
writing, electronic and industrial manufacturing.
It can be done. It has been done. It is hard.
So was programming a GUI application when the Macintosh came out
(remember QuickDraw?). We learned to live with it.
Ed
> Of course, as good engineers, we should never allow ourselves the hubris
> of /believing/ we have achieved zero defects but that doesn't invalidate
> the aim. (Aircraft manufacturers do a great deal of mathematical
> analysis of stresses in wings but still proof load test each new design;
> they don't expect to find any problems because of the amount of analysis
> they have done but, very occasionally, they do).
>
> regards
>
>
>
> Peter
_______________________________________________
Secure Coding mailing list (SC-L)
SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
