Re: [SC-L] top 10 software security surprises
On Dec 16, 2008, at 1:25 PM, Gary McGraw wrote: Using the software security framework introduced in October (A Software Security Framework: Working Towards a Realistic Maturity Model http://www.informit.com/articles/article.aspx?p=1271382), we interviewed nine executives running top software security programs in order to gather real data from real programs. Wow, this is great stuff. Kudos to Gary, Sammy, and Brian. I have a couple comments/observations on some of your conclusions: - You obviously wrote the top-10 list in C, since it went from 9 to 0. :-) - Not only are there are no magic software security metrics, bad metrics actually hurt. This is an excellent point. I think it's also worth noting that it's important to carefully consider what metrics make sense for an organization _as early as possible_ in the life of their software security efforts. Trying to retro-engineer some metrics into a program after the fact is not a fun thing. - Secure-by-default frameworks can be very helpful, especially if they are presented as middleware classes (but watch out for an over focus on security stuff). Yes yes yes! I've found significantly more traction to prescriptive guidance vs. a don't do this list of bad practices. Plus, it inherently supports a mindset of positive validation instead of negative. It's important to look for common mistakes, but if you really want your devs to follow, give them clear coding guidelines with annotated descriptions of how to follow them. Efforts like OWASP's ESAPI are indeed a great starting point here for plugging in things like strong positive input validation and such. - Web application firewalls are not in wide use, especially not as Web application firewalls. I can't say I'm much surprised by this one. Even with PCI-DSS driving people to WAFs (or do external independent code reviews), I just don't often see them often. But you go on to say, But even these two didn't use them to block application attacks; they used them to monitor Web applications and gather data about attacks.--but you don't come back to this point. One serious benefit to WAFs can be enhancing the ability to do monitoring, especially of legacy apps. Adding one network choke point WAF can quickly add an app-level monitoring capability that few organizations considered when rolling the apps out in the first place. - Though software security often seems to fit an audit role rather naturally, many successful programs evangelize (and provide software security resources) rather than audit even in regulated industries This one too is very encouraging to see. - Architecture analysis is just as hard as we thought, and maybe harder. And this one is very discouraging. I've seen good results in doing architectural risk analyses, but the ones that produce useful results tend to be the more ad hoc ones -- and NOT the ones that follow rigorous processes. - All nine programs we talked to have in-house training curricula, and training is considered the most important software security practice in the two most mature software security initiatives we interviewed. That explains the quarter-million miles in my United account this year alone. :-) Ugh. - Though all of the organizations we talked to do some kind of penetration testing, the role of penetration testing in all nine practices is diminishing over time. Hallelujah! Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com smime.p7s Description: S/MIME cryptographic signature ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] top 10 software security surprises
Thanks Ken. For me this has been an incredibly eye-opening project. It can be hard for people to distinguish between ideas that merely look good on paper and ideas that are actually in widespread use. Once we’ve cleaned up the data and gotten approval from the organizations we canvassed, we think there’ll be plenty of ways to apply what we’ve learned. The project Pravir mentioned is one. Brian [Ed. This was from br...@fortify.com, but was dropped by the Mailman server since it's set to ignore emails from non-subscribed addresses (spam...). Issue was resolved re Brian's email address.] On 12/17/08 11:48 AM, Ken van Wyk k...@krvw.com wrote: On Dec 16, 2008, at 1:25 PM, Gary McGraw wrote: Using the software security framework introduced in October (A Software Security Framework: Working Towards a Realistic Maturity Model http://www.informit.com/articles/article.aspx?p=1271382), we interviewed nine executives running top software security programs in order to gather real data from real programs. Wow, this is great stuff. Kudos to Gary, Sammy, and Brian. I have a couple comments/observations on some of your conclusions: - You obviously wrote the top-10 list in C, since it went from 9 to 0. :-) - Not only are there are no magic software security metrics, bad metrics actually hurt. This is an excellent point. I think it's also worth noting that it's important to carefully consider what metrics make sense for an organization _as early as possible_ in the life of their software security efforts. Trying to retro-engineer some metrics into a program after the fact is not a fun thing. - Secure-by-default frameworks can be very helpful, especially if they are presented as middleware classes (but watch out for an over focus on security stuff). Yes yes yes! I've found significantly more traction to prescriptive guidance vs. a don't do this list of bad practices. Plus, it inherently supports a mindset of positive validation instead of negative. It's important to look for common mistakes, but if you really want your devs to follow, give them clear coding guidelines with annotated descriptions of how to follow them. Efforts like OWASP's ESAPI are indeed a great starting point here for plugging in things like strong positive input validation and such. - Web application firewalls are not in wide use, especially not as Web application firewalls. I can't say I'm much surprised by this one. Even with PCI-DSS driving people to WAFs (or do external independent code reviews), I just don't often see them often. But you go on to say, But even these two didn't use them to block application attacks; they used them to monitor Web applications and gather data about attacks.--but you don't come back to this point. One serious benefit to WAFs can be enhancing the ability to do monitoring, especially of legacy apps. Adding one network choke point WAF can quickly add an app-level monitoring capability that few organizations considered when rolling the apps out in the first place. - Though software security often seems to fit an audit role rather naturally, many successful programs evangelize (and provide software security resources) rather than audit even in regulated industries This one too is very encouraging to see. - Architecture analysis is just as hard as we thought, and maybe harder. And this one is very discouraging. I've seen good results in doing architectural risk analyses, but the ones that produce useful results tend to be the more ad hoc ones -- and NOT the ones that follow rigorous processes. - All nine programs we talked to have in-house training curricula, and training is considered the most important software security practice in the two most mature software security initiatives we interviewed. That explains the quarter-million miles in my United account this year alone. :-) Ugh. - Though all of the organizations we talked to do some kind of penetration testing, the role of penetration testing in all nine practices is diminishing over time. Hallelujah! Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com ) as a free, non-commercial service to the software security community. ___ smime.p7s Description: S/MIME cryptographic signature ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at -
[SC-L] top 10 software security surprises
hi sc-l, Using the software security framework introduced in October (A Software Security Framework: Working Towards a Realistic Maturity Model http://www.informit.com/articles/article.aspx?p=1271382), we interviewed nine executives running top software security programs in order to gather real data from real programs. Our goal is to create a maturity model based on these data, and we're busy working on that (stay tuned here for more). However, in the course of analyzing the data we gathered, we unearthed some surprises that we share in this month's informIT article: http://www.informit.com/articles/article.aspx?p=1315431 My bet is that some of the findings will come as a surprise to sc-l readers as well. Check the article out. Merry New Year to you all. gem company www.cigital.com podcast www.cigital.com/silverbullet blog www.cigital.com/justiceleague book www.swsec.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] top 10 software security surprises
Hey All. On the topic of maturity models, in Gary's first article he mentioned a draft model I created. Since I've mostly been discussing it in OWASP circles, I wanted to point out the Software Assurance Maturity Model (SAMM) project at http://www.opensamm.org I kicked off that work based on a few years experience running with CLASP and with help from the guys at Fortify. Currently, there's a BETA release ( http://www.opensamm.org/downloads/SAMM-BETA-0.8.1.pdf), but a new revision should be available by the end of year. That next revision will reflect feedback from individual reviewers, output from OWASP working sessions, and much of the real-world feedback that Gary talks about below. I'm always interested to hear comments/questions/flames, so please feel free to download it and send any feedback. Thanks! p. On Tue, Dec 16, 2008 at 10:25 AM, Gary McGraw g...@cigital.com wrote: hi sc-l, Using the software security framework introduced in October (A Software Security Framework: Working Towards a Realistic Maturity Model http://www.informit.com/articles/article.aspx?p=1271382), we interviewed nine executives running top software security programs in order to gather real data from real programs. Our goal is to create a maturity model based on these data, and we're busy working on that (stay tuned here for more). However, in the course of analyzing the data we gathered, we unearthed some surprises that we share in this month's informIT article: http://www.informit.com/articles/article.aspx?p=1315431 My bet is that some of the findings will come as a surprise to sc-l readers as well. Check the article out. Merry New Year to you all. gem company www.cigital.com podcast www.cigital.com/silverbullet blog www.cigital.com/justiceleague book www.swsec.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___ -- ~ ~ ~ ~~~ ~~ ~ Pravir Chandra chandraatlistdotorg PGP:CE60 0E10 9207 7290 06EB 5107 4032 63FC 338E 16E4 ~ ~~ ~~~ ~ ~ ~ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
