Although pentesting isn't perfect, I think in the right scope it has the potential of acting in a vital role in the development lifecycle of a project.
Building known attack patterns into a library which can be run against a codebase has some merrit, as long as you understand what the resulting expectations will be. As an example, I would consider automated vulnerability assessment tools built to do input validation fuzzing to be part of pentesting. And most pentesters out there use said tools for that very reason. I agree that pentesting at the START of a project is futile. Evaluating the risks to the software by understanding its architecture, inputs and flows goes much deeper and allows you to better find design flaws instead of implementation bugs. But I am not so sure I would dismiss the act of pentesting because of the badness-ometer factor. If we did, we would also be dismissing things like static code analysis tools as they may show similar results to many out there. On a different tangent to this thread though, I don't think that the BEST use of pentesting is for determining how secure your code is in the first place. It is much more suited to allow you to stress test failure code paths for different implementation configurations. No matter how safe you write your code, pentesting can ferret out different scenarios that come from deployment configuration problems. That is, if the pentest tools and the user(s) of said tools know how to run through this properly. As you mentioned in your article, too many people pass themselves off as pentesting experts when they aren't. Just because they CAN run Nessus doesn't mean they are good pentesters. Its about using the right tool for the right job. As pentest tools mature I think we will be able to use the growing attack libraries to test against known patterns to eliminate the brain dead security bugs, while allowing the tools to go deeper and ferret out problems as it reaches more code coverage. The more we can automate that process and make it part of our daily tests... the quicker it can expose 'known problems' in a code base. Can anyone recommend a tool that CAN tell you how good your security is? I thought not. Here I go quoting Spaf again. "When the code is incorrect, you can't really talk about security. When the code is faulty, it cannot be safe." Problem is, no tool in the world is going to show green blinky lights to tell you the code is safe. Human heuristics come into play here, and we have to leverage what assets we have, both manual and automatic, to find the faulty code and eliminate it. And pentesting is just another one of those tools in the arsenal to help. Regards, Dana Epp [Microsoft Security MVP] http://silverstr.ufies.org/blog/ -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary McGraw Sent: Thursday, July 13, 2006 8:05 AM To: Nash Cc: Secure Coding Mailing List Subject: RE: [SC-L] ddj: beyond the badnessometer Excellent post nash. Thanks! I agree with you for the most part. You have a view of pen testing that is quite sophisticated (especially compared to the usual drivel). I agree with you so much that I included pen testing as the third most important touchpoint in my new book "Software Security" www.swsec.com. It is the subject of chapter 6. All the code review and architectural risk analysis in the world can still be completely sidestepped by poor decisions regarding the fielded software. Pen testing is ideal for looking into that. But there are two things I want to reiterate: 1) pen testing is a bad way to *start* working on software security...you'll get much better traction with code review and architectural risk assessment. {Of course, what nash says about the power of a live sploit is true, and that kind of momentum creation may be called for in a completely new situation where biz execs need basic clue.} 2) pen testing can't tell you anything about how good your security is, only how bad it is. 3) never use the results of a pen test as a "punch list" to attain security gem company www.cigital.com podcast www.cigital.com/silverbullet book www.swsec.com ------------------------------------------------------------------------ ---- This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ------------------------------------------------------------------------ ---- _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
