That's a bit of dodging the question, I'd like to hear more. You comment below implied that it was your consistent use of vendor-based static analyis tool that allowed you to figure out top N list of bugs for a specific organization. "Leading with static analysis" as your primary analysis driver concearns me. Will you elaborate, please?
- Jim ----- Original Message ----- From: "Gary McGraw" <g...@cigital.com> To: "Jim Manico" <j...@manico.net>; "Steven M. Christey" <co...@linus.mitre.org> Cc: "Sammy Migues" <smig...@cigital.com>; "Dustin Sullivan" <dustin.sulli...@informit.com>; "Secure Code Mailing List" <SC-L@securecoding.org> Sent: Thursday, March 19, 2009 9:04 AM Subject: Re: [SC-L] BSIMM: Confessions of a Software Security Alchemist (informIT) Actually no. See: http://www.cigital.com/papers/download/j15bsi.pdf (John Steven, State of Application Assessment, IEEE S&P) I am not a tool guy, I am a software security guy. gem http://www.cigital.com/~gem On 3/19/09 2:58 PM, "Jim Manico" <j...@manico.net> wrote: > Many of the top N lists we encountered were developed through the > consistent use of static analysis tools. After looking at millions of > lines of code (sometimes constantly), a ***real*** top N list of bugs > emerges for an organization. You mean a "real list of what a certain vendors static analysis tools find". If you think that list really measures the risk of an organizations software security posture - that might ne considered to be insane! =) - Jim ----- Original Message ----- From: "Gary McGraw" <g...@cigital.com> To: "Steven M. Christey" <co...@linus.mitre.org> Cc: "Sammy Migues" <smig...@cigital.com>; "Dustin Sullivan" <dustin.sulli...@informit.com>; "Secure Code Mailing List" <SC-L@securecoding.org> Sent: Wednesday, March 18, 2009 11:54 AM Subject: Re: [SC-L] BSIMM: Confessions of a Software Security Alchemist (informIT) > Hi Steve, > > Many of the top N lists we encountered were developed through the > consistent use of static analysis tools. After looking at millions of > lines of code (sometimes constantly), a ***real*** top N list of bugs > emerges for an organization. Eradicating number one is an obvious > priority. Training can help. New number one...lather, rinse, repeat. > > Other times (like say in the one case where the study participant did not > believe in static analysis for religious reasons) things are a bit more > flip (and thus suffer from the "no data" problem I like to complain > about). I do not recall a case when the top N lists were driven by > customers. > > Sorry I missed your talk at the SWA forum. I'll chalk that one up to NoVa > traffic. > > gem > > http://www.cigital.com/~gem > > > On 3/18/09 5:47 PM, "Steven M. Christey" <co...@linus.mitre.org> wrote: > > > > On Wed, 18 Mar 2009, Gary McGraw wrote: > >> Because it is about building a top N list FOR A PARTICULAR ORGANIZATION. >> You and I have discussed this many times. The generic top 25 is >> unlikely to apply to any particular organization. The notion of using >> that as a driver for software purchasing is insane. On the other hand >> if organization X knows what THEIR top 10 bugs are, that has real value. > > Got it, thanks. I guessed as much. Did you investigate whether the > developers' personal top-N lists were consistent with what their customers > cared about? How did the developers go about selecting them? > > By the way, last week in my OWASP Software Assurance Day talk on the Top > 25, I had a slide on the role of top-N lists in BSIMM, where I attempted > to say basically the same thing. This was after various slides that tried > to emphasize how the current Top 25 is both incomplete and not necessarily > fully relevant to a particular organization's needs. So while the message > may have been diluted during initial publication, it's being refined > somewhat. > > - Steve > > > _______________________________________________ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - > http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > _______________________________________________ > _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
