Re: [SC-L] InformIT: comparing static analysis tools
Hey Gary, Nice article. A brief note, Ounce is dead. The product was renamed IBM Rational AppScan Source Edition after IBM's acquisition of Ounce. Small matter but for what it's worth, Jim hi sc-l, John Steven and I recently collaborated on an article for informIT. The article is called Software [In]security: Comparing Apples, Oranges, and Aardvarks (or, All Static Analysis Tools Are Not Created Equal) and is available here: http://www.informit.com/articles/article.aspx?p=1680863 Now that static analysis tools like Fortify and Ounce are hitting the mainstream there are many potential customers who want to compare them and pick the best one. We explain why that's more difficult than it sounds at first and what to watch out for as you begin to compare tools. We did this in order to get out in front of test suites that purport to work for tool comparison. If you wonder why such suites may not work as advertised, read the article. Your feedback is welcome. gem company www.cigital.com podcast www.cigital.com/silverbullet blog www.cigital.com/justiceleague book www.swsec.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___
Re: [SC-L] InformIT: comparing static analysis tools
All, I followed this article up with a blog entry, more targeted at adopting organizations. I hope you find it useful: http://www.cigital.com/justiceleague/2011/02/02/if-its-so-hard-why-bother/ John Steven Senior Director; Advanced Technology Consulting Desk: 703.404.9293 x1204 Cell: 703.727.4034 Key fingerprint = 4772 F7F3 1019 4668 62AD 94B0 AE7F EEF4 62D5 F908 Blog: http://www.cigital.com/justiceleague Papers: http://www.cigital.com/papers/jsteven http://www.cigital.com Software Confidence. Achieved. hi sc-l, John Steven and I recently collaborated on an article for informIT. The article is called Software [In]security: Comparing Apples, Oranges, and Aardvarks (or, All Static Analysis Tools Are Not Created Equal) and is available here: http://www.informit.com/articles/article.aspx?p=1680863 Now that static analysis tools like Fortify and Ounce are hitting the mainstream there are many potential customers who want to compare them and pick the best one. We explain why that's more difficult than it sounds at first and what to watch out for as you begin to compare tools. We did this in order to get out in front of test suites that purport to work for tool comparison. If you wonder why such suites may not work as advertised, read the article. Your feedback is welcome. smime.p7s Description: S/MIME cryptographic signature ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___
Re: [SC-L] InformIT: comparing static analysis tools
Chris, I've tried to leverage Veracode in recent engagements. Here is how the conversation went: Jim: Boss, can I upload all of your code to this cool SaaS service for analysis? Client: Uh no, and next time you ask, I'm having you committed. I'm sure you have faced these objections before. How do you work around them? -Jim Manico http://manico.net On Feb 3, 2011, at 1:54 PM, Chris Wysopal cwyso...@veracode.com wrote: Nice article. In the 5 years Veracode has been selling static analysis services we have seen the market mature. In the beginning, organizations were down in the weeds. What false positive rate or false negative rate does the tool/service have over a test suite such as SAMATE. Then we saw a move up to looking at the trees. Did the tool/service support the Java frameworks I am using? Now we are seeing organizations look at the forest. Can I scale static analysis effectively over all my development sites, my outsourcers, and vendors? This is a good sign of a maturing market. It is my firm belief that software security has a consumption problem. We know what the defects are. We know how to fix them. We even have automation for detecting a lot of them. The problem is getting the information and technology to the right person at the right time effectively and managing an organization-wide program. This is the next challenge for static analysis. bias-alertI think SaaS based software is more easily consumed and this isn't any different for software security/bias-alert -Chris -Original Message- From: sc-l-boun...@securecoding.org [mailto:sc-l-boun...@securecoding.org] On Behalf Of Gary McGraw Sent: Wednesday, February 02, 2011 9:49 AM To: Secure Code Mailing List Subject: [SC-L] InformIT: comparing static analysis tools hi sc-l, John Steven and I recently collaborated on an article for informIT. The article is called Software [In]security: Comparing Apples, Oranges, and Aardvarks (or, All Static Analysis Tools Are Not Created Equal) and is available here: http://www.informit.com/articles/article.aspx?p=1680863 Now that static analysis tools like Fortify and Ounce are hitting the mainstream there are many potential customers who want to compare them and pick the best one. We explain why that's more difficult than it sounds at first and what to watch out for as you begin to compare tools. We did this in order to get out in front of test suites that purport to work for tool comparison. If you wonder why such suites may not work as advertised, read the article. Your feedback is welcome. gem company www.cigital.com podcast www.cigital.com/silverbullet blog www.cigital.com/justiceleague book www.swsec.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___
Re: [SC-L] InformIT: comparing static analysis tools
Great article, Gary. Many of your comments about static technology challenges I have seen and verified first-hand, including multi-million dollar cost overruns. After some great dialogue with John Stevens, I suspect we have had similar experiences. I was just about to write a similar article at a higher level - about how the vast majority of enterprise customers I work with are actively moving security into the SDLC. The time has come, the event has tipped, and SDLC security is indeed mainstream. This is an exciting time to be in the industry. However - I was curious about your comments about dynamic tools reaching their limit or something like that, as customers move security efforts deeper into the SDLC. What does that mean? I see customers making extensive use of dynamic testing, and leveraging it deeper and deeper into the SDLC. Enterprises are aggressively rolling out and expanding dynamic testing earlier in the SDLC. Newer dynamic testing technologies help solve/reduce some of the key pain points that static technologies alone are causing them, as you so well illustrated.. . I am very interested in why you sound dismissive of these successful technologies? Your article makes it sound like they are hitting some invisible limit, when in fact hundreds of enterprises are expanding dynamic testing in the SDLC. And these are serious projects that run into the 7-figures. Any insight you can share would be appreciated! Great work identifying the general shift SDLC security is moving mainstream, --- Arian Evans Software Security Referee On Wed, Feb 2, 2011 at 6:48 AM, Gary McGraw g...@cigital.com wrote: hi sc-l, John Steven and I recently collaborated on an article for informIT. The article is called Software [In]security: Comparing Apples, Oranges, and Aardvarks (or, All Static Analysis Tools Are Not Created Equal) and is available here: http://www.informit.com/articles/article.aspx?p=1680863 Now that static analysis tools like Fortify and Ounce are hitting the mainstream there are many potential customers who want to compare them and pick the best one. We explain why that's more difficult than it sounds at first and what to watch out for as you begin to compare tools. We did this in order to get out in front of test suites that purport to work for tool comparison. If you wonder why such suites may not work as advertised, read the article. Your feedback is welcome. gem company www.cigital.com podcast www.cigital.com/silverbullet blog www.cigital.com/justiceleague book www.swsec.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates ___
