[SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
Here's an interesting article from Dark Reading about web fuzzers. Web fuzzing seems to be gaining some traction these days as a popular means of testing web apps and web services. http://www.darkreading.com/document.asp? doc_id=118162f_src=darkreading_section_296 Any good/bad experiences and opinions to be shared here on SC-L regarding fuzzing as a means of testing web apps/services? I have to say I'm unconvinced, but agree that they should be one part--and a small one at that--of a robust testing regimen. Cheers, Ken P.S. I'm over in Belgium right now for SecAppDev (http:// www.secappdev.org). HD Moore wowed the class here with a demo of Metasploit 3.0. For those of you that haven't looked at this (soon to be released, but available in beta now) tool, you really should check it out. Although it's geared at the IT Security pen testing audience, I do believe that it has broader applicability as a framework for constructing one-off exploits against applications. - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com PGP.sig Description: This is a digitally signed message part ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote: Given the complex manipulations that can work in XSS attacks (see RSnake's cheat sheet) as well as directory traversal, combined with the sheer number of potential inputs in web applications, multipied by all the variations in encodings, I wouldn't be surprised if they were effective in finding those kinds of implementation bugs, even in well-designed software. Although successfully diagnosing some XSS without live verification smells like a hard problem akin to the Ptacek/Newsham vantage point issues in IDS. With the track record of non-web fuzzers and PROTOS style test suites, why do you think web app fuzzing is less likely to succeed? It's not so much that I don't think fuzzing is useful, it's that I don't see one size fits all fuzzing _products_ being useful. To me, it gets to an issue of informed vs. uninformed (or white box vs. black box if you prefer) testing. While they're both useful and should both be exercised, I believe (though I have no hard statistics to validate) that issues of coverage/state are always going to doom uninformed testing to being less effective than informed testing. For a fuzzer to be really meaningful, I believe that a smart fuzzing approach is going to be the best bet, and that makes it hard for a one size fits all product solution to be feasible. To do smart fuzzing, a lot of setup time is necessary in establishing an appropriate test harness and cases that fully exercise the files, network interface data, user data, etc., that the software is expecting. Perhaps I'm totally off base, and I invite any product folks here to chime in and correct my misconceptions. Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com PGP.sig Description: This is a digitally signed message part ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web)Fuzz - Security News Analysis
Just for the record, the testing literature (non-security) supports ken's point of view. Possibly the most amusing thing about all of this discussion about black box versus white box is that this is only one of many many divisions in testing. Others include partition testing, fault injection, and mutation testing. We really have a long way to go with security testing to catch up. gem company www.cigital.com podcast www.cigital.com/silverbullet book www.swsec.com -Original Message- From: Kenneth Van Wyk [mailto:[EMAIL PROTECTED] Sent: Tue Feb 27 04:07:07 2007 To: Secure Coding Subject:Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web)Fuzz - Security News Analysis On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote: Given the complex manipulations that can work in XSS attacks (see RSnake's cheat sheet) as well as directory traversal, combined with the sheer number of potential inputs in web applications, multipied by all the variations in encodings, I wouldn't be surprised if they were effective in finding those kinds of implementation bugs, even in well-designed software. Although successfully diagnosing some XSS without live verification smells like a hard problem akin to the Ptacek/Newsham vantage point issues in IDS. With the track record of non-web fuzzers and PROTOS style test suites, why do you think web app fuzzing is less likely to succeed? It's not so much that I don't think fuzzing is useful, it's that I don't see one size fits all fuzzing _products_ being useful. To me, it gets to an issue of informed vs. uninformed (or white box vs. black box if you prefer) testing. While they're both useful and should both be exercised, I believe (though I have no hard statistics to validate) that issues of coverage/state are always going to doom uninformed testing to being less effective than informed testing. For a fuzzer to be really meaningful, I believe that a smart fuzzing approach is going to be the best bet, and that makes it hard for a one size fits all product solution to be feasible. To do smart fuzzing, a lot of setup time is necessary in establishing an appropriate test harness and cases that fully exercise the files, network interface data, user data, etc., that the software is expecting. Perhaps I'm totally off base, and I invite any product folks here to chime in and correct my misconceptions. Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
On 2/27/07, Kenneth Van Wyk [EMAIL PROTECTED] wrote: Here's an interesting article from Dark Reading about web fuzzers. Web fuzzing seems to be gaining some traction these days as a popular means of testing web apps and web services. http://www.darkreading.com/document.asp?doc_id=118162f_src=darkreading_section_296 Any good/bad experiences and opinions to be shared here on SC-L regarding fuzzing as a means of testing web apps/services? I have to say I'm unconvinced, but agree that they should be one part--and a small one at that--of a robust testing regimen. unconvinced of what? what fuzzing is useful? or that it's the best security testing method ever? or you remain unconvinced that fuzzing in web apps is fuzzing in os apps? fuzzing has obvious advantages. that's all anyone should care about. Cheers, Ken -- mike ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
On Feb 27, 2007, at 4:54 AM, Michael Silk wrote: unconvinced of what? what fuzzing is useful? or that it's the best security testing method ever? or you remain unconvinced that fuzzing in web apps is fuzzing in os apps? fuzzing has obvious advantages. that's all anyone should care about. No, not that it's useful or not. As I said in my other reply, my real wariness is of the one size fits all product solutions. It seems to me that the best fuzzing tools are in fact frameworks for building customized fuzzing tests. OWASP's jbrofuzz (in beta release currently) is an example of what I mean here. It gives the tester the means for identifying fields to fuzz and how to fuzz them (say, integer size testing), and then you press the fuzz button and it generates all the tests. That's useful, meaningful, and valuable, IMHO. But it's not a fire and forget general purpose tool that can test any web app. Beyond that, to me it's an issue of coverage. As was any uninformed testing, it's bound to miss things, which is to be expected. (E.g., a state tree that contains a format string vulnerability that doesn't execute because the testing never triggered that particular state -- hence my comments about test coverage/state earlier.) So, my impression is that fuzzing is useful (in Howard/Lipner's SDL book, they say that some 25% of the bugs they find during testing come out during fuzzing), but that it should only be a small, say 10-20%, part of a testing regimen. Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com PGP.sig Description: This is a digitally signed message part ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz- Security News Analysis
In my personal experience with web app testing, I have found that web fuzzers are not nearly as useful as fuzzers used for applications, and more specifically I have found numerous bugs doing direct API fuzzing. In the case of testing web applications I find that using something like SpiDynamics tool is great for a first pass as a black box test, but to really get an idea of how bad the vulnerability is, the extent, etc. manual testing is an absolute must. I know that most people on this list don't necessarily believe in fuzzing as a good security test, and I can hear Gary groaning already, but I think that fuzzing tools are becoming more and more intelligent, and you are soon going to see some extremely powerful tools in this arena. Check out the paper on genetic algorithms and fuzzing from BlackHat as well as the tool from Jared DeMott at Applied Security. As for Metasploit, its a very sweet tool, as well as a very useful framework for learning and developing exploits, particularly the tricky IE+ActiveX heap nastiness that requires a little kung fu and a lot of coffee. JS _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth Van Wyk Sent: Tuesday, February 27, 2007 12:06 AM To: Secure Coding Subject: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz- Security News Analysis Here's an interesting article from Dark Reading about web fuzzers. Web fuzzing seems to be gaining some traction these days as a popular means of testing web apps and web services. http://www.darkreading.com/document.asp?doc_id=118162 http://www.darkreading.com/document.asp?doc_id=118162f_src=darkreading_sec tion_296 f_src=darkreading_section_296 Any good/bad experiences and opinions to be shared here on SC-L regarding fuzzing as a means of testing web apps/services? I have to say I'm unconvinced, but agree that they should be one part--and a small one at that--of a robust testing regimen. Cheers, Ken P.S. I'm over in Belgium right now for SecAppDev (http://www.secappdev.org). HD Moore wowed the class here with a demo of Metasploit 3.0. For those of you that haven't looked at this (soon to be released, but available in beta now) tool, you really should check it out. Although it's geared at the IT Security pen testing audience, I do believe that it has broader applicability as a framework for constructing one-off exploits against applications. - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
[SC-L] Disclosure: vulnerability pimps? or super heroes?
Hi all, The neverending debate over disclosure continued at RSA this year with a panel featuring Chris Wysopl and others rehashing old ground. There are points on both sides, with radicals on one side (say marcus ranum) calling the disclosure people vulnerability pimps and radicals on the other saying that computer security would make no progress at all without disclosure. I've always sought some kind of middle ground when it comes to disclosure. The idea is to minimize risk to users of the broken system while at the samne time learning something about security and making sure the system gets fixed. Disclosure is the subject of my latest Darkreading column: http://www.darkreading.com/document.asp?doc_id=118174 What do you think? Should we talk about exploits? Should we out vendors? Is there a line to be drawn anywhere? gem company www.cigital.com podcast www.cigital.com/silverbullet book www.swsec.com This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Disclosure: vulnerability pimps? or super heroes?
J. M. Seitz wrote: On a related note, does anyone have an example where Company A was disclosing vulnerabilities about competing Company B's product and got into trouble over it? Is this something that could be litigated? In fact, Tom Ptacek found a hole in one of Marcus' products while working for a competitor. I suspect Tom reported it properly, though. This research pissed MJR off to no end, which he made clear at one Black Hat talk he gave, with Tom standing at the back of the room. I suspect this was a key point in MJR's life when his code got touched in an inappropriate way, and has led to his current level of curmudgeonry. Or, for a more contemporary example, witness Symantec researchers looking for holes in just about everything. I fail to see any merit for a legitimate lawsuit. Of course, in the US, you can sue whomever you please. BB ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Disclosure: vulnerability pimps? or super heroes?
On 2/28/07, Gary McGraw [EMAIL PROTECTED] wrote: Hi all, The neverending debate over disclosure continued at RSA this year with a panel featuring Chris Wysopl and others rehashing old ground. There are points on both sides, with radicals on one side (say marcus ranum) calling the disclosure people vulnerability pimps and radicals on the other saying that computer security would make no progress at all without disclosure. I've always sought some kind of middle ground when it comes to disclosure. The idea is to minimize risk to users of the broken system while at the samne time learning something about security and making sure the system gets fixed. I think havning extremists is a good thing. Forces people to re-evaluate their position and actually do things, instead of having a disucssion about it. Without that there would be middle grounders sitting around wondering about the best approach. With the extremists these middlegrounders have to react, or at least companies do. Which is only good. Disclosure is the subject of my latest Darkreading column: http://www.darkreading.com/document.asp?doc_id=118174 What do you think? Should we talk about exploits? Should we out vendors? Is there a line to be drawn anywhere? I think if you find an exploit do what you personally want. If I had time to research them, I'd probably be pimping them out for as much as I could; why not? I can decide. I found it. Same to you, with what you found. The only line will come if some authority in some country makes it illegal to sell them. And obviously there would be incredible difficulties in isolating that, IMHO. gem company www.cigital.com podcast www.cigital.com/silverbullet book www.swsec.com -- mike (s1, s2, s3) ; ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___