All, The original blog entry stems from a CWE pie chart that won't die until we replace it with a more well-grounded pie chart.
We posted a followup here: http://www.matasano.com/log/912/finger-79tcp-christeymartin-evolution-of-the-cwe-pie-chart/ In short, CWE contains several types of nodes at multiple levels of abstraction, including general categories ("input validation problems") and arbitrary groupings ("problems related to memory management"). The original pie chart mixed these node types with 'real' weaknesses, and we included it in a CWE briefing as a demonstrative example of the utility of CWE in comparing code analysis tools. While that pie chart is still partially usable for showing a relative lack of overlap between tools (modulo the abstraction problem), the "only 45% of weakness types are found by tools" figure is probably low, since CWE currently has many nodes that are organizational in nature, so they would be excluded from any comparative analysis. (Although we're also probably relatively shallow with respect to design issues compared to implementation bugs, which might pull the numbers in another direction as CWE continues to fill in the gaps). As vaguely implied in the followup blog entry above, we will be working on a new pie chart with a better selection of CWE nodes, which should generate more credible numbers. We've been doing the ground work, e.g. explicitly identifying the types of nodes that could then be excluded from such analyses, but I can't be sure of when we'll have a new-and-improved pie chart. Rest assured that we are highly motivated to replace the existing chart, however, and I think we've learned our lesson about releasing "demonstrative statistics" in new technology areas that don't have any. - Steve _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
