RE: [SC-L] ZDNET: LAMP lights the way in open-source security
-Original Message- >From: Crispin Cowan [mailto:[EMAIL PROTECTED] > >Gavin, Michael wrote: >> Yeah, statistics can allow you to say and "prove" just about anything. >> >> OK, showing my ignorance here, since I haven't checked out any of the >> LAMP source trees and reviewed the code: how much of the code making up >> those modules is written in scripting languages vs. how much of it is >> written in C, C++ (and how much, if any, is written in any other >> compiled languages)? >> > That doesn't matter; what matters is what fraction of disclosed > vulnerabilities is in each segment of the code? If 90% of the > vulnerabilities come from the PHP part, then the fact that 90% of the > lines of code are in C doesn't help. [Gavin, Michael] Absolutely true! But from the perspective of improving static source code analysis tools, if 90% of the code is in C, which is one of the 2 languages supported by the Coverity product, then we now have one reasonable data point regarding how well that (moderate amount of) C code was written with respect to one vendor's notion/implementation of secure coding in C. Certainly not a huge win for anyone, but a potential starting point for comparing techniques and products. For example, I haven't been following the status of David Wheeler's flawfinder, but even if that hasn't been updated lately, it might be interesting to see which flaws it finds that Coverity found, which Coverity found that flawfinder doesn't, and which flawfinder finds that Coverity didn't. Unfortunately your comment below regarding the proprietary nature of Coverity makes such a comparison less useful for everyone but Coverity... >> If the LAMP source code itself is primarily C/C++, then arguably, the >> results are somewhat interesting, though I think they would be much more >> interesting if this DISA project was set up to test the open source code >> with a number of commercial scanners instead of just the Coverity >> scanner, then we could at least compare the merits of various scanning >> techniques and implementations. > The proprietary status of the Coverity scanner is a continuous pain. > That's why I tend to ignore it where possible :) > > Crispin > -- > Crispin Cowan, Ph.D. http://crispincowan.com/~crispin/ > Director of Software Engineering, Novell http://novell.com > Olympic Games: The Bi-Annual Festival of Corruption ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
Yeah, statistics can allow you to say and "prove" just about anything. OK, showing my ignorance here, since I haven't checked out any of the LAMP source trees and reviewed the code: how much of the code making up those modules is written in scripting languages vs. how much of it is written in C, C++ (and how much, if any, is written in any other compiled languages)? If the LAMP source code itself is primarily C/C++, then arguably, the results are somewhat interesting, though I think they would be much more interesting if this DISA project was set up to test the open source code with a number of commercial scanners instead of just the Coverity scanner, then we could at least compare the merits of various scanning techniques and implementations. In this case, the distinction to me is that they have tested the LAMP platform code, not the code that people write on top of it for their applications, and are making some statements about the software security of the LAMP platform compared to the rest of the open source code they scanned. If on the other hand, a significant portion of the LAMP code base itself is made up of scripting language code, then I agree with you, the results aren't terribly useful to anyone other than possibly Coverity and Stanford. Note: significant is open to interpretation, but doesn't have to be large; 10 or 15 per cent would seem significant enough to me. -Original Message- From: Jeremy Epstein [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 07, 2006 12:17 PM To: Gavin, Michael; Kenneth R. van Wyk; Secure Coding Mailing List Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source security All of which proves that there are lies, damn lies, and statistics (the statistic being the lower bug density, which ignores the most potentially vulnerable parts of the system). > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Gavin, Michael > Sent: Tuesday, March 07, 2006 11:49 AM > To: Kenneth R. van Wyk; Secure Coding Mailing List > Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source > security > > The Coverity product (Coverity Prevent) is a static source > code analysis tool for C and C++, see > http://www.coverity.com/library/pdf/coverity_prevent.pdf. > > It isn't actually scanning (or if it is, it isn't analyzing) > any of the scripting code, as far I as can tell. > > Michael > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth R. van Wyk > Sent: Tuesday, March 07, 2006 10:56 AM > To: Secure Coding Mailing List > Subject: [SC-L] ZDNET: LAMP lights the way in open-source security > > Interesting article out on ZDNet today: > > http://www.zdnetasia.com/news/security/0,39044215,39315781,00.htm > > The article refers to the US government sponsored study being > done by Stanford University, Symantec, and Coverity. It > says, "The so-called LAMP stack of open-source software has a > lower bug density--the number of bugs per thousand lines of > code--than a baseline of 32 open-source projects analyzed, > Coverity, a maker of code analysis tools, announced Monday." > > This surprised me quite a bit, especially given LAMP's > popular reliance on scripting languages PHP, Perl, and/or > Python. Still, the article doesn't discuss any of the root > causes of the claimed security strengths in LAMP-based code. > Perhaps it's because the scripting languages tend to make > things less complex for the coders (as opposed to more > complex higher level languages like Java and C#/.NET)? Opinions? > > Cheers, > > Ken > -- > Kenneth R. van Wyk > KRvW Associates, LLC > http://www.KRvW.com > > > ___ > Secure Coding mailing list (SC-L) > SC-L@securecoding.org > List information, subscriptions, etc - > http://krvw.com/mailman/listinfo/sc-l > List charter available at - > http://www.securecoding.org/list/charter.php > > ___ > Secure Coding mailing list (SC-L) > SC-L@securecoding.org > List information, subscriptions, etc - > http://krvw.com/mailman/listinfo/sc-l > List charter available at - > http://www.securecoding.org/list/charter.php > ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
The Coverity product (Coverity Prevent) is a static source code analysis tool for C and C++, see http://www.coverity.com/library/pdf/coverity_prevent.pdf. It isn't actually scanning (or if it is, it isn't analyzing) any of the scripting code, as far I as can tell. Michael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth R. van Wyk Sent: Tuesday, March 07, 2006 10:56 AM To: Secure Coding Mailing List Subject: [SC-L] ZDNET: LAMP lights the way in open-source security Interesting article out on ZDNet today: http://www.zdnetasia.com/news/security/0,39044215,39315781,00.htm The article refers to the US government sponsored study being done by Stanford University, Symantec, and Coverity. It says, "The so-called LAMP stack of open-source software has a lower bug density--the number of bugs per thousand lines of code--than a baseline of 32 open-source projects analyzed, Coverity, a maker of code analysis tools, announced Monday." This surprised me quite a bit, especially given LAMP's popular reliance on scripting languages PHP, Perl, and/or Python. Still, the article doesn't discuss any of the root causes of the claimed security strengths in LAMP-based code. Perhaps it's because the scripting languages tend to make things less complex for the coders (as opposed to more complex higher level languages like Java and C#/.NET)? Opinions? Cheers, Ken -- Kenneth R. van Wyk KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] Bugs and flaws
"Architecture" is also an overloaded term, often meaning either a design (the output of architects) or the implementation of certain types of design (the output of engineers). Hoping to clarify Chris's comment on architecture flaws: architecture defects as in the defects in the output produced by architects are "design flaws"; architecture defects as in the defects in the output of programmers/coders/engineers are "implementation flaws". FWIW, I agree with Chris, "design flaw" and "implementation flaw" seem better/more descriptive/less confusing than revised definitions for "flaw" and "bug". (Then again, I once worked at @stake..., on the other hand, IIRC this terminology is more consistent with what you find in Ross Anderson's classic "Security Engineering".) Michael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Wysopal Sent: Thursday, February 02, 2006 4:35 PM To: Gary McGraw Cc: William Kruse; Wall, Kevin; Secure Coding Mailing List Subject: RE: [SC-L] Bugs and flaws In the manufacturing world, which is far more mature than the software development world, they use the terminology of "design defect" and "manufacturing defect". So this distinction is useful and has stood the test of time. Flaw and defect are synonymous. We should just pick one. You could say that the term for manufacturing software is "implementation". So why do we need to change the terms for the software world? Wouldn't "design defect" and "implementation defect" be clearer and more in line with the manufacturing quality discipline, which the software quality discipline should be working towards emulating. (When do we get to Six Sigma?) I just don't see the usefulness of calling a "design defect" a "flaw". "Flaw" by itself is overloaded. And in the software world, "bug" can mean an implementation or design problem, so "bug" alone is overloaded for describing an implementation defect. At @stake the Application Center of Excellence used the terminology "design flaw" and "implementation flaw". It well understood by our customers. As Crispin said in an earlier post on the subject, the line is sometimes blurry. I am sure this is the case in manufacturing too. Architecture flaws can be folded into the design flaw category for simplicity. My vote is for a less overloaded and clearer terminology. -Chris P.S. My father managed a non-destructive test lab at a jet engine manufacturer. They had about the highest quality requirements in the world. So for many hours I was regaled with tales about the benefits of performing static analysis on individual components early in the manufacturing cycle. They would dip cast parts in a fluorescent liquid and look at them under ultraviolet light to illuminate cracks caused during casting process. For critical parts which would receive more stress, such as the fan blades, they would x-ray each part to inspect for internal cracks. A more expensive process but warranted due to the increased risk of total system failure for a defect in those parts. The static testing was obviously much cheaper and delivered better quality than just bolting the parts together and doing dynamic testing in a test cell. It's a wonder that it has taken the software security world so long to catch onto the benefits of static testing of implementation. I think we can learn a lot more from the manufacturing world. On Thu, 2 Feb 2006, Gary McGraw wrote: > Hi all, > > When I introduced the "bugs" and "flaws" nomenclature into the > literature, I did so in an article about the software security workshop > I chaired in 2003 (see http://www.cigital.com/ssw/). This was > ultimately written up in an "On the Horizon" paper published by IEEE > Security & Privacy. > > Nancy Mead and I queried the SWEBOK and looked around to see if the new > usage caused collision. It did not. The reason I think it is important > to distinguish the two ends of the rather slippery range (crispy is > right about that) is that software security as a field is not paying > enough attention to architecture. By identifying flaws as a subcategory > of defects (according the the SWEBOK), we can focus some attention on > the problem. > > >>From the small glossary in "Software Security" (my new book out > tomorrow): > > Bug-A bug is an implementation-level software problem. Bugs may exist in > code but never be executed. Though the term bug is applied quite > generally > by many software practitioners, I reserve use of the term to encompass > fairly > simple implementation errors. Bugs are implementation-level problems > that > can be easily discovered and remedied. See Chapter 1. > > Flaw-A design-level or architectural software defect. High-level defects > cause 50% of software security problems. See Chapter 1. > > In any case, I intend to still use these terms like this, and I would be > very pleased if you would all join me. > > gem > > > > -
