The document properties suggests June 2009, and it's a shame that there isn't much details as we are looking to evaluate 3 of the code analysis tools for our development here.
CJC > -----Original Message----- > From: sc-l-boun...@securecoding.org > [mailto:sc-l-boun...@securecoding.org] On Behalf Of Jeremy Epstein > Sent: 29 September 2009 14:49 > To: sc-l > Subject: [SC-L] NSA comparison of source code analysis tools > > (Apologies if I already sent this to the group; I don't think I did.) > > There's an interesting presentation at > http://www.iarpa.gov/stonesoup_Merced_DHSAWGbrief.pdf about a study > done by the US NSA (National Security Agency) of C and Java source > code analysis tools. They developed a synthetic test suite, and then > ran six tools against the Java version and five tools against the C > version (the specific tools and versions used are identified in the > presentation). None of the tools found all of the problems, and 40% > of the problems weren't found by any of the tools. Even where the > problems were found, there was a surprising level of inconsistency > among the tools. > > Unfortunately, there's not much detail in the presentation. There's a > report that's been written, but so far not approved for release (or so > I'm told). I don't know whether the issue is classification (they > don't want the bad guys to know what sort of things can sneak past > their detectors), or proprietary information, or just bureaucracy. > > It would be interesting to hear comments from vendors on the list as > to the limitations on such a test (certainly using synthetic programs > isn't realistic), as well as whether they've adapted the tools to find > more of these types of problems. > > --Jeremy > > P.S. The report is undated, but I believe it's fairly recent. > _______________________________________________ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - > http://krvw.com/mailman/listinfo/sc-l > List charter available at - > http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC > (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > _______________________________________________ >
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
