Re: [SC-L] Silver Bullet turns 2: Mary Ann Davidson
Thanks for the feedback Stephen. It's been a blast doing Silver Bullet for the last two years. For our next episode, I'm going to interview Jon Swartz who covers security for USA Today. That should be a twist! We're also planning to syndicate Silver Bullet through informIT soon. gem p.s. Can we have your permission to post this comment on the SB page? http://www.cigital.com/~gem On 4/4/08 1:19 AM, Stephen Craig Evans [EMAIL PROTECTED] wrote: Gary, Great interview. You've had some powerhouse interviews recently, for example with Chris Wysopal (my dream is that a static tool can fix business logic flaws) and Ed Amoroso (security researchers are the bomb defusers of the Internet). I laughed at your blunt comment: that would be great (everybody doing software assurance in 5 years) but also impossible. Andrew, in addition to your points: - I liked her self-deprecating humor when she talked about her coding skills - I think she made a justified, underhanded jab at the appsec community to make our stuff easier to use when she said: (At 4m 55sec) There are a lot of people who are very well-intended and very sharp who come up with laundry lists of 8000 good things that we should do in security and all these things we should be doing and all these metrics - and that's all great, but then ... what is the benefit for the cost of getting that information? and the do-gooders, and in this case I mean it in a good sense, need to do is to help people figure out what are the most important things to do first so that they'll get the biggest bang for the buck. - I liked her point, using a military analogy, is that products should be self-defended. Cheers, Stephen --- From: Andrew van der Stock [EMAIL PROTECTED] Date: Thu, Mar 27, 2008 at 7:32 AM Subject: Re: [SC-L] Silver Bullet turns 2: Mary Ann Davidson To: Gary McGraw [EMAIL PROTECTED] Cc: Kathy Clark-Fisher [EMAIL PROTECTED], Mary Ann Davidson [EMAIL PROTECTED], Secure Mailing List SC-L@securecoding.org Gary, Good interview. The discussion on being unable to develop trust relationships with contractors who release exploits was interesting, and I wished that there was more discussion on that point. I would have thought signing a contract made it easier to sue for breach of contract than untested laws (or bad laws like the UK's RIPA), so much so you'd really think twice as well as the negative downside of being considered untrustworthy with confidential data - which is like a plague to any consultancy business. I really wish Ms Davidson had gone into detail on their SDL, as to what is really in there, and where we could read it and review it. Oracle's is an interesting turn around considering back in 2005 / 2006, the research community and Oracle's relationship was at an all time low, essentially begging Oracle to put in an SDL and address the security defects properly without outside folks finding them first. I have since read that fences have been somewhat mended between researchers, such as David Litchfield, and Oracle. I still wince at that episode - it was entirely unprofessional of Oracle to attack Litchfield, who was practicing responsible disclosure for up to 600-800 days, when 30 is the norm. I personally was extremely unimpressed with Oracle's approach of shooting the messenger rather than fixing the product. I must admit that episode led me to dismiss Oracle as the walking dead as they obviously couldn't be trusted with data of value, and so didn't follow news about Oracle ... until this interview. I'm glad they're now using automated SCA tools and fuzzers, they're now finding most of the security issues themselves, have an internal review team, and my personal favorite - developer awareness / education. This is a 180 degree turnaround from the prior to 2005/2006 era. I particularly like that she's going to the universities and ask them to teach coding security. This is what they SHOULD have been doing rather than attacking the research community. I'm glad that Oracle is now drinking the kool aid and treating security as a fundamental software engineering requirement. It's about time. thanks, Andrew van der Stock Lead Author, OWASP Guide to Writing Secure Applications and OWASP Top 10 ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter
Re: [SC-L] Silver Bullet turns 2: Mary Ann Davidson
Mary -- Thank you for your reply and clarification. I am 100% on board with you about folks inventing taxonomies and then telling business owners and developers what artifacts they need to look for, measure, etc. without any real cost or business justification with regards to to your costs vs. return to track/measure in the first place, let alone *why* to fix them. I lumped a few things together which I shouldn't have... Defect decision making: Most organizations involved in the care and feeding of business software, whether or not they produced it, are still stuck at the instance level of do I fix this vulnerabilitly/asset now or later? You are at an entirely different level as a large producer. As one of the largest ISVs -- you likely have very different needs and perspectives. e.g.- you might be able to bear the cost of deeper software defect analysis than most software consumer organizations, due to the huge cost of regression testing and/or the support costs of post-hotfix deployments if they break things. The above is just a guess on my part, but we all know that hotfixing running production database servers is not the same game as implementing output encoding on a web application search field. /cost/effort/risk Anyway, long and short, like Andrew and others -- those of us in the pragmatic good enough daily increments security camps would love for you to share more of your experiences with this, Oracle's SDL journey, etc. and add that to our growing pool of what we know about this. A lot of what I mentioned we need to gather/answer is really more for the consumers of your software, writing their custom code on top of it, and then trying to operate in a reasonably safe manner while making money. As an ISV you have a slightly different operating model and bottom line, and you probably are afforded less ability to use temporary band-aids or mitigation steps versus flat-out fixing your software. Not everyone has to fix all their software, or make it self-defending. Which is ironic considering I used to present on the notion of self-defending web applications', and help people implement SDLs. But I've become less of a purist now that I'm trying to help people secure dozens to hundreds of applications at once, with limited time and budget. It's hard not to feel like a charlatan when you can't give the business folks hard metrics on defect implications, failure costs, let alone just the cost of measurement vs. potential return. Thank you for doing Gary's interview, and the stimulating thoughts. Have a good weekend all. Come see me at RSA if any of you SC-L'ers are around. I'll be putting people to sleep with a talk on encoding, transcoding, and cannonicalization issues in our software (and WAFs :). (At RSA. Lol.) Ciao -ae On Fri, Apr 4, 2008 at 2:50 PM, mary.ann davidson [EMAIL PROTECTED] wrote: Hi Arian Thanks for your comments. Just to clarify, I was not trying to look at this at the micro level of should we fix buffer overflows or fix SQL injections? We (collectively) now have pretty good tools that can find exploitable defects in software and the answer is, if it is exploitable you fix it, and if it is just poor coding practice (but not exploitable) you still probably should fix it, albeit maybe not with same urgency as exploitable defect and you may only fix it going forward. My issue is people who invent 50,000 idealogically pure development practices, or artifacts or metrics that someone might want you to produce (often, somoene who is an academic or a think tank person), and never look at Ok, what does it cost me to capture that information? Will being able to measure X or create a metric help me actually manage better? If I can't have a theologically perfect development process (and who does?), then what are the best things to do to actually improve? The perfect is really the enemy of the good or the enemy of the better. I like a lot of your suggestions after We really need to know I do realize that as you close one attack vector, persistent enemies will try something new. But one of the reasons I do feel strongly about get the dreck out of the code base is that, all things being equal, forcing attackers to work harder is a good thing. Also, reducing customers' cost of security operations (through higher security-worthy, more resilient code) is a good thing because resources are always constrained, and the resources people spend on patching, and/or random third party add security appliances and software takes scarce resources that might be put to better uses. If the Army has tank crews of 12, and 10 of them are busy fixing the tank treads because they keep slipping off, they aren't going to be too ready to fight an actual battle. Regards - Mary Ann Arian J. Evans wrote: I'll second this Gary. You've done nice work here. I think Mary Ann's comments are some of the most interesting concerning what our industry needs
Re: [SC-L] Silver Bullet turns 2: Mary Ann Davidson
and very sharp who come up with laundry lists of 8000 good things that we should do in security and all these things we should be doing and all these metrics – and that's all great, but then ... what is the benefit for the cost of getting that information? and the do-gooders, and in this case I mean it in a good sense, need to do is to help people figure out what are the most important things to do first so that they'll get the biggest bang for the buck. - I liked her point, using a military analogy, is that products should be self-defended. Cheers, Stephen --- From: Andrew van der Stock [EMAIL PROTECTED] Date: Thu, Mar 27, 2008 at 7:32 AM Subject: Re: [SC-L] Silver Bullet turns 2: Mary Ann Davidson To: Gary McGraw [EMAIL PROTECTED] Cc: Kathy Clark-Fisher [EMAIL PROTECTED], Mary Ann Davidson [EMAIL PROTECTED], Secure Mailing List SC-L@securecoding.org Gary, Good interview. The discussion on being unable to develop trust relationships with contractors who release exploits was interesting, and I wished that there was more discussion on that point. I would have thought signing a contract made it easier to sue for breach of contract than untested laws (or bad laws like the UK's RIPA), so much so you'd really think twice as well as the negative downside of being considered untrustworthy with confidential data - which is like a plague to any consultancy business. I really wish Ms Davidson had gone into detail on their SDL, as to what is really in there, and where we could read it and review it. Oracle's is an interesting turn around considering back in 2005 / 2006, the research community and Oracle's relationship was at an all time low, essentially begging Oracle to put in an SDL and address the security defects properly without outside folks finding them first. I have since read that fences have been somewhat mended between researchers, such as David Litchfield, and Oracle. I still wince at that episode - it was entirely unprofessional of Oracle to attack Litchfield, who was practicing responsible disclosure for up to 600-800 days, when 30 is the norm. I personally was extremely unimpressed with Oracle's approach of shooting the messenger rather than fixing the product. I must admit that episode led me to dismiss Oracle as the walking dead as they obviously couldn't be trusted with data of value, and so didn't follow news about Oracle ... until this interview. I'm glad they're now using automated SCA tools and fuzzers, they're now finding most of the security issues themselves, have an internal review team, and my personal favorite - developer awareness / education. This is a 180 degree turnaround from the prior to 2005/2006 era. I particularly like that she's going to the universities and ask them to teach coding security. This is what they SHOULD have been doing rather than attacking the research community. I'm glad that Oracle is now drinking the kool aid and treating security as a fundamental software engineering requirement. It's about time. thanks, Andrew van der Stock Lead Author, OWASP Guide to Writing Secure Applications and OWASP Top 10 ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Silver Bullet turns 2: Mary Ann Davidson
Gary, Good interview. The discussion on being unable to develop trust relationships with contractors who release exploits was interesting, and I wished that there was more discussion on that point. I would have thought signing a contract made it easier to sue for breach of contract than untested laws (or bad laws like the UK's RIPA), so much so you'd really think twice as well as the negative downside of being considered untrustworthy with confidential data - which is like a plague to any consultancy business. I really wish Ms Davidson had gone into detail on their SDL, as to what is really in there, and where we could read it and review it. Oracle's is an interesting turn around considering back in 2005 / 2006, the research community and Oracle's relationship was at an all time low, essentially begging Oracle to put in an SDL and address the security defects properly without outside folks finding them first. I have since read that fences have been somewhat mended between researchers, such as David Litchfield, and Oracle. I still wince at that episode - it was entirely unprofessional of Oracle to attack Litchfield, who was practicing responsible disclosure for up to 600-800 days, when 30 is the norm. I personally was extremely unimpressed with Oracle's approach of shooting the messenger rather than fixing the product. I must admit that episode led me to dismiss Oracle as the walking dead as they obviously couldn't be trusted with data of value, and so didn't follow news about Oracle ... until this interview. I'm glad they're now using automated SCA tools and fuzzers, they're now finding most of the security issues themselves, have an internal review team, and my personal favorite - developer awareness / education. This is a 180 degree turnaround from the prior to 2005/2006 era. I particularly like that she's going to the universities and ask them to teach coding security. This is what they SHOULD have been doing rather than attacking the research community. I'm glad that Oracle is now drinking the kool aid and treating security as a fundamental software engineering requirement. It's about time. thanks, Andrew van der Stock Lead Author, OWASP Guide to Writing Secure Applications and OWASP Top 10 ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
