Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz- Security News Analysis
In my personal experience with web app testing, I have found that web fuzzers are not nearly as useful as fuzzers used for applications, and more specifically I have found numerous bugs doing direct API fuzzing. In the case of testing web applications I find that using something like SpiDynamics tool is great for a first pass as a black box test, but to really get an idea of how bad the vulnerability is, the extent, etc. manual testing is an absolute must. I know that most people on this list don't necessarily believe in fuzzing as a good security test, and I can hear Gary groaning already, but I think that fuzzing tools are becoming more and more intelligent, and you are soon going to see some extremely powerful tools in this arena. Check out the paper on genetic algorithms and fuzzing from BlackHat as well as the tool from Jared DeMott at Applied Security. As for Metasploit, its a very sweet tool, as well as a very useful framework for learning and developing exploits, particularly the tricky IE+ActiveX heap nastiness that requires a little kung fu and a lot of coffee. JS _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth Van Wyk Sent: Tuesday, February 27, 2007 12:06 AM To: Secure Coding Subject: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz- Security News Analysis Here's an interesting article from Dark Reading about web fuzzers. Web fuzzing seems to be gaining some traction these days as a popular means of testing web apps and web services. http://www.darkreading.com/document.asp?doc_id=118162 <http://www.darkreading.com/document.asp?doc_id=118162&f_src=darkreading_sec tion_296> &f_src=darkreading_section_296 Any good/bad experiences and opinions to be shared here on SC-L regarding fuzzing as a means of testing web apps/services? I have to say I'm unconvinced, but agree that they should be one part--and a small one at that--of a robust testing regimen. Cheers, Ken P.S. I'm over in Belgium right now for SecAppDev (http://www.secappdev.org). HD Moore wowed the class here with a demo of Metasploit 3.0. For those of you that haven't looked at this (soon to be released, but available in beta now) tool, you really should check it out. Although it's geared at the IT Security pen testing audience, I do believe that it has broader applicability as a framework for constructing one-off exploits against applications. - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
On Feb 27, 2007, at 4:54 AM, Michael Silk wrote: unconvinced of what? what fuzzing is useful? or that it's the best security testing method ever? or you remain unconvinced that fuzzing in web apps is > fuzzing in os apps? fuzzing has obvious advantages. that's all anyone should care about. No, not that it's useful or not. As I said in my other reply, my real wariness is of the "one size fits all" product solutions. It seems to me that the best fuzzing tools are in fact frameworks for building customized fuzzing tests. OWASP's jbrofuzz (in beta release currently) is an example of what I mean here. It gives the tester the means for identifying fields to fuzz and how to fuzz them (say, integer size testing), and then you press the fuzz button and it generates all the tests. That's useful, meaningful, and valuable, IMHO. But it's not a "fire and forget" general purpose tool that can test any web app. Beyond that, to me it's an issue of coverage. As was any uninformed testing, it's bound to miss things, which is to be expected. (E.g., a state tree that contains a format string vulnerability that doesn't execute because the testing never triggered that particular state -- hence my comments about test coverage/state earlier.) So, my impression is that fuzzing is useful (in Howard/Lipner's SDL book, they say that some 25% of the bugs they find during testing come out during fuzzing), but that it should only be a small, say 10-20%, part of a testing regimen. Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com PGP.sig Description: This is a digitally signed message part ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
On 2/27/07, Kenneth Van Wyk <[EMAIL PROTECTED]> wrote: > > Here's an interesting article from Dark Reading about web fuzzers. Web > fuzzing seems to be gaining some traction these days as a popular means of > testing web apps and web services. > > http://www.darkreading.com/document.asp?doc_id=118162&f_src=darkreading_section_296 > > Any good/bad experiences and opinions to be shared here on SC-L regarding > fuzzing as a means of testing web apps/services? I have to say I'm > unconvinced, but agree that they should be one part--and a small one at > that--of a robust testing regimen. unconvinced of what? what fuzzing is useful? or that it's the best security testing method ever? or you remain unconvinced that fuzzing in web apps is > fuzzing in os apps? fuzzing has obvious advantages. that's all anyone should care about. > Cheers, > > Ken -- mike ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web)Fuzz - Security News Analysis
Just for the record, the testing literature (non-security) supports ken's point of view. Possibly the most amusing thing about all of this discussion about black box versus white box is that this is only one of many many divisions in testing. Others include partition testing, fault injection, and mutation testing. We really have a long way to go with security testing to catch up. gem company www.cigital.com podcast www.cigital.com/silverbullet book www.swsec.com -Original Message- From: Kenneth Van Wyk [mailto:[EMAIL PROTECTED] Sent: Tue Feb 27 04:07:07 2007 To: Secure Coding Subject:Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web)Fuzz - Security News Analysis On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote: > Given the complex manipulations that can work in XSS attacks (see > RSnake's > cheat sheet) as well as directory traversal, combined with the sheer > number of potential inputs in web applications, multipied by all the > variations in encodings, I wouldn't be surprised if they were > effective in > finding those kinds of implementation bugs, even in well-designed > software. Although successfully diagnosing some XSS without live > verification smells like a hard problem akin to the Ptacek/Newsham > "vantage point" issues in IDS. > > With the track record of non-web fuzzers and PROTOS style test > suites, why > do you think web app fuzzing is less likely to succeed? It's not so much that I don't think fuzzing is useful, it's that I don't see "one size fits all" fuzzing _products_ being useful. To me, it gets to an issue of informed vs. uninformed (or "white box" vs. "black box" if you prefer) testing. While they're both useful and should both be exercised, I believe (though I have no hard statistics to validate) that issues of coverage/state are always going to doom uninformed testing to being less effective than informed testing. For a fuzzer to be really meaningful, I believe that a "smart fuzzing" approach is going to be the best bet, and that makes it hard for a "one size fits all" product solution to be feasible. To do smart fuzzing, a lot of setup time is necessary in establishing an appropriate test harness and cases that fully exercise the files, network interface data, user data, etc., that the software is expecting. Perhaps I'm totally off base, and I invite any product folks here to chime in and correct my misconceptions. Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote: Given the complex manipulations that can work in XSS attacks (see RSnake's cheat sheet) as well as directory traversal, combined with the sheer number of potential inputs in web applications, multipied by all the variations in encodings, I wouldn't be surprised if they were effective in finding those kinds of implementation bugs, even in well-designed software. Although successfully diagnosing some XSS without live verification smells like a hard problem akin to the Ptacek/Newsham "vantage point" issues in IDS. With the track record of non-web fuzzers and PROTOS style test suites, why do you think web app fuzzing is less likely to succeed? It's not so much that I don't think fuzzing is useful, it's that I don't see "one size fits all" fuzzing _products_ being useful. To me, it gets to an issue of informed vs. uninformed (or "white box" vs. "black box" if you prefer) testing. While they're both useful and should both be exercised, I believe (though I have no hard statistics to validate) that issues of coverage/state are always going to doom uninformed testing to being less effective than informed testing. For a fuzzer to be really meaningful, I believe that a "smart fuzzing" approach is going to be the best bet, and that makes it hard for a "one size fits all" product solution to be feasible. To do smart fuzzing, a lot of setup time is necessary in establishing an appropriate test harness and cases that fully exercise the files, network interface data, user data, etc., that the software is expecting. Perhaps I'm totally off base, and I invite any product folks here to chime in and correct my misconceptions. Cheers, Ken - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com PGP.sig Description: This is a digitally signed message part ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
[SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis
Here's an interesting article from Dark Reading about web fuzzers. Web fuzzing seems to be gaining some traction these days as a popular means of testing web apps and web services. http://www.darkreading.com/document.asp? doc_id=118162&f_src=darkreading_section_296 Any good/bad experiences and opinions to be shared here on SC-L regarding fuzzing as a means of testing web apps/services? I have to say I'm unconvinced, but agree that they should be one part--and a small one at that--of a robust testing regimen. Cheers, Ken P.S. I'm over in Belgium right now for SecAppDev (http:// www.secappdev.org). HD Moore wowed the class here with a demo of Metasploit 3.0. For those of you that haven't looked at this (soon to be released, but available in beta now) tool, you really should check it out. Although it's geared at the IT Security pen testing audience, I do believe that it has broader applicability as a framework for constructing one-off exploits against applications. - Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com PGP.sig Description: This is a digitally signed message part ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___