[SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis

2007-02-27 Thread Kenneth Van Wyk
Here's an interesting article from Dark Reading about web fuzzers.   
Web fuzzing seems to be gaining some traction these days as a popular  
means of testing web apps and web services.


http://www.darkreading.com/document.asp? 
doc_id=118162f_src=darkreading_section_296


Any good/bad experiences and opinions to be shared here on SC-L  
regarding fuzzing as a means of testing web apps/services?  I have to  
say I'm unconvinced, but agree that they should be one part--and a  
small one at that--of a robust testing regimen.


Cheers,

Ken

P.S. I'm over in Belgium right now for SecAppDev (http:// 
www.secappdev.org).  HD Moore wowed the class here with a demo of  
Metasploit 3.0.  For those of you that haven't looked at this (soon  
to be released, but available in beta now) tool, you really should  
check it out.  Although it's geared at the IT Security pen testing  
audience, I do believe that it has broader applicability as a  
framework for constructing one-off exploits against applications.

-
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com






PGP.sig
Description: This is a digitally signed message part
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis

2007-02-27 Thread Kenneth Van Wyk

On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote:
Given the complex manipulations that can work in XSS attacks (see  
RSnake's

cheat sheet) as well as directory traversal, combined with the sheer
number of potential inputs in web applications, multipied by all the
variations in encodings, I wouldn't be surprised if they were  
effective in

finding those kinds of implementation bugs, even in well-designed
software.  Although successfully diagnosing some XSS without live
verification smells like a hard problem akin to the Ptacek/Newsham
vantage point issues in IDS.

With the track record of non-web fuzzers and PROTOS style test  
suites, why

do you think web app fuzzing is less likely to succeed?


It's not so much that I don't think fuzzing is useful, it's that I  
don't see one size fits all fuzzing _products_ being useful.


To me, it gets to an issue of informed vs. uninformed (or white box  
vs. black box if you prefer) testing.  While they're both useful  
and should both be exercised, I believe (though I have no hard  
statistics to validate) that issues of coverage/state are always  
going to doom uninformed testing to being less effective than  
informed testing.  For a fuzzer to be really meaningful, I believe  
that a smart fuzzing approach is going to be the best bet, and that  
makes it hard for a one size fits all product solution to be feasible.


To do smart fuzzing, a lot of setup time is necessary in establishing  
an appropriate test harness and cases that fully exercise the files,  
network interface data, user data, etc., that the software is expecting.


Perhaps I'm totally off base, and I invite any product folks here to  
chime in and correct my misconceptions.


Cheers,

Ken
-
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com






PGP.sig
Description: This is a digitally signed message part
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web)Fuzz - Security News Analysis

2007-02-27 Thread Gary McGraw
Just for the record, the testing literature (non-security) supports ken's point 
of view.  Possibly the most amusing thing about all of this discussion about 
black box versus white box is that this is only one of many many divisions in 
testing.  Others include partition testing, fault injection, and mutation 
testing.  We really have a long way to go with security testing to catch up.

gem

company www.cigital.com
podcast www.cigital.com/silverbullet
book www.swsec.com


 -Original Message-
From:   Kenneth Van Wyk [mailto:[EMAIL PROTECTED]
Sent:   Tue Feb 27 04:07:07 2007
To: Secure Coding
Subject:Re: [SC-L] Dark Reading - Desktop Security - Here Comes the 
(Web)Fuzz - Security News Analysis

On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote:
 Given the complex manipulations that can work in XSS attacks (see  
 RSnake's
 cheat sheet) as well as directory traversal, combined with the sheer
 number of potential inputs in web applications, multipied by all the
 variations in encodings, I wouldn't be surprised if they were  
 effective in
 finding those kinds of implementation bugs, even in well-designed
 software.  Although successfully diagnosing some XSS without live
 verification smells like a hard problem akin to the Ptacek/Newsham
 vantage point issues in IDS.

 With the track record of non-web fuzzers and PROTOS style test  
 suites, why
 do you think web app fuzzing is less likely to succeed?

It's not so much that I don't think fuzzing is useful, it's that I  
don't see one size fits all fuzzing _products_ being useful.

To me, it gets to an issue of informed vs. uninformed (or white box  
vs. black box if you prefer) testing.  While they're both useful  
and should both be exercised, I believe (though I have no hard  
statistics to validate) that issues of coverage/state are always  
going to doom uninformed testing to being less effective than  
informed testing.  For a fuzzer to be really meaningful, I believe  
that a smart fuzzing approach is going to be the best bet, and that  
makes it hard for a one size fits all product solution to be feasible.

To do smart fuzzing, a lot of setup time is necessary in establishing  
an appropriate test harness and cases that fully exercise the files,  
network interface data, user data, etc., that the software is expecting.

Perhaps I'm totally off base, and I invite any product folks here to  
chime in and correct my misconceptions.

Cheers,

Ken
-
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com









This electronic message transmission contains information that may be
confidential or privileged.  The information contained herein is intended
solely for the recipient and use by any other party is not authorized.  If
you are not the intended recipient (or otherwise authorized to receive this
message by the intended recipient), any disclosure, copying, distribution or
use of the contents of the information is prohibited.  If you have received
this electronic message transmission in error, please contact the sender by
reply email and delete all copies of this message.  Cigital, Inc. accepts no
responsibility for any loss or damage resulting directly or indirectly from
the use of this email or its contents.
Thank You.


___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis

2007-02-27 Thread Michael Silk
On 2/27/07, Kenneth Van Wyk [EMAIL PROTECTED] wrote:

 Here's an interesting article from Dark Reading about web fuzzers.  Web
 fuzzing seems to be gaining some traction these days as a popular means of
 testing web apps and web services.

 http://www.darkreading.com/document.asp?doc_id=118162f_src=darkreading_section_296

 Any good/bad experiences and opinions to be shared here on SC-L regarding
 fuzzing as a means of testing web apps/services?  I have to say I'm
 unconvinced, but agree that they should be one part--and a small one at
 that--of a robust testing regimen.

unconvinced of what? what fuzzing is useful? or that it's the best
security testing method ever? or you remain unconvinced that fuzzing
in web apps is  fuzzing in os apps?

fuzzing has obvious advantages. that's all anyone should care about.


 Cheers,

 Ken

-- mike
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz - Security News Analysis

2007-02-27 Thread Kenneth Van Wyk

On Feb 27, 2007, at 4:54 AM, Michael Silk wrote:

unconvinced of what? what fuzzing is useful? or that it's the best
security testing method ever? or you remain unconvinced that fuzzing
in web apps is  fuzzing in os apps?

fuzzing has obvious advantages. that's all anyone should care about.


No, not that it's useful or not.  As I said in my other reply, my  
real wariness is of the one size fits all product solutions.  It  
seems to me that the best fuzzing tools are in fact frameworks for  
building customized fuzzing tests.  OWASP's jbrofuzz (in beta release  
currently) is an example of what I mean here.  It gives the tester  
the means for identifying fields to fuzz and how to fuzz them (say,  
integer size testing), and then you press the fuzz button and it  
generates all the tests.  That's useful, meaningful, and valuable,  
IMHO.  But it's not a fire and forget general purpose tool that can  
test any web app.


Beyond that, to me it's an issue of coverage.  As was any uninformed  
testing, it's bound to miss things, which is to be expected.  (E.g.,  
a state tree that contains a format string vulnerability that doesn't  
execute because the testing never triggered that particular state --  
hence my comments about test coverage/state earlier.)


So, my impression is that fuzzing is useful (in Howard/Lipner's SDL  
book, they say that some 25% of the bugs they find during testing  
come out during fuzzing), but that it should only be a small, say  
10-20%, part of a testing regimen.


Cheers,

Ken
-
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com






PGP.sig
Description: This is a digitally signed message part
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz- Security News Analysis

2007-02-27 Thread J. M. Seitz
In my personal experience with web app testing, I have found that web
fuzzers are not nearly as useful as fuzzers used for applications, and more
specifically I have found numerous bugs doing direct API fuzzing. In the
case of testing web applications I find that using something like
SpiDynamics tool is great for a first pass as a black box test, but to
really get an idea of how bad the vulnerability is, the extent, etc. manual
testing is an absolute must. I know that most people on this list don't
necessarily believe in fuzzing as a good security test, and I can hear Gary
groaning already, but I think that fuzzing tools are becoming more and more
intelligent, and you are soon going to see some extremely powerful tools in
this arena. Check out the paper on genetic algorithms and fuzzing from
BlackHat as well as the tool from Jared DeMott at Applied Security.
 
As for Metasploit, its a very sweet tool, as well as a very useful framework
for learning and developing exploits, particularly the tricky IE+ActiveX
heap nastiness that requires a little kung fu and a lot of coffee. 
 
JS

  _  

From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Kenneth Van Wyk
Sent: Tuesday, February 27, 2007 12:06 AM
To: Secure Coding
Subject: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web) Fuzz-
Security News Analysis


Here's an interesting article from Dark Reading about web fuzzers. Web
fuzzing seems to be gaining some traction these days as a popular means of
testing web apps and web services.

http://www.darkreading.com/document.asp?doc_id=118162
http://www.darkreading.com/document.asp?doc_id=118162f_src=darkreading_sec
tion_296 f_src=darkreading_section_296 

Any good/bad experiences and opinions to be shared here on SC-L regarding
fuzzing as a means of testing web apps/services? I have to say I'm
unconvinced, but agree that they should be one part--and a small one at
that--of a robust testing regimen. 

Cheers,

Ken

P.S. I'm over in Belgium right now for SecAppDev (http://www.secappdev.org).
HD Moore wowed the class here with a demo of Metasploit 3.0. For those of
you that haven't looked at this (soon to be released, but available in beta
now) tool, you really should check it out. Although it's geared at the IT
Security pen testing audience, I do believe that it has broader
applicability as a framework for constructing one-off exploits against
applications.

-
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com




___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___