On Tue, 13 Jan 2009, Gary McGraw wrote: > I thought you might get a kick out of it.
I did! :-) Always good to have debates. >Executives don't care about technical bugs No, but they do what PCI says they have to (i.e. listen to the OWASP Top Ten). They do care about the bottom line. They hate buying software and finding out how crappy it is afterward. The Siemens example comes to mind. It was mind-boggling to me to hear that they were forced to pay 150% of the original cost just to get the software they bought in a secure form. http://www.networkworld.com/news/2009/011209-software-security-effort.html?page=2 >Too much focus on bugs. The Top 25 has 4 or 5 items that are clearly design-related, maybe more if you think that improper input validation is related to design, and still more if you count all the "external control" items which may be implementation or design depending on who's talking and what the programmer's responsibilities are. The Top 25 even has a couple classic Saltzer-and-Schroeder examples in there (rephrased to avoid the incredible confusion and misinterpretation that has gone on with the original S&S). >Vulnerability lists help auditors more than developers. Agree - except the Top 25 has anywhere from 3 to 10 specific mitigations/preventions for each CWE. And whether we like it or not, auditors help to drive change. Maybe not the optimal change, but they drive change. Also - when the developers' software managers are told by their marketers that they could lose many, then the managers will figure out how to get the developers to improve. Long-term thinking here - I know, that's not allowed for this industry. >One person's top bug is another person's yawner. Absolutely, a point I brought up in the other post I made to SC-L on some interesting challenges. >Using bug parade lists for training leads to awareness but does not >educate. Yep - which is why we want universities to get cracking, and if the Top 25 helps to prod them on, then so be it. >Bug lists change with the prevailing technology winds. Yep - which is why the official name of the Top Ten begins with "2009". We'll do this again next year. >Top ten lists mix levels. Also brought up in my last email as a problem for the industry in general. Regarding Seven Pernicious Kingdoms - how does the Top 25 map to them? I could classify most of the Top 25 in multiple categories. Should "poor output encoding" be put under the Input validation kingdom? Sounds kind of like using the "Start" button in Windows to shut down ;-) Should "cleartext transmission" be put under Environment? or Security Features? Again, we *all* have this problem. >Automated tools can find bugs . let them. Yes, and a lesson of the Top 25 (that we all already know) is that when people start to apply it, they'll see how a tool won't be a silver bullet. Also covered in my last email... >When it comes to testing, security requirements are more important than >vulnerability lists. Which is cover a teeny bit in the other email I sent. Also, New York State has put up draft text that mentions the Top 25 as part of a condition for acquisition. Is that enough? Hardly. But things like the Black/Williams "software facts label" aren't mature either, and Dept. of Homeland Security probably has a couple years to go before their work on assurance cases begins to take shape. >Ten is not enough. Neither is 25. Number 26 (another design issue) is also mentioned in the other email I posted. I'm of the mindset that the Top 25 is, short-term, an awareness tool for developers and for customers. In the longer term, maybe it will be a little blip on the road to actionable software assurance. Given that approximately 1000 people have created delicious bookmarks for it, and I've alread seen comments from a couple developers "hey, we should go check this out" - then we are already seeing some success. Gary, it might seem ironic since I am leading the most comprehensive bug parade out there in CWE, but I agree with you that just following the bug parade is not enough. The Top 25 is a means to an end, and not the end itself. Only time will tell, though. - Steve _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
