Just for the record, the testing literature (non-security) supports ken's point of view. Possibly the most amusing thing about all of this discussion about black box versus white box is that this is only one of many many divisions in testing. Others include partition testing, fault injection, and mutation testing. We really have a long way to go with security testing to catch up.
gem company www.cigital.com podcast www.cigital.com/silverbullet book www.swsec.com -----Original Message----- From: Kenneth Van Wyk [mailto:[EMAIL PROTECTED] Sent: Tue Feb 27 04:07:07 2007 To: Secure Coding Subject: Re: [SC-L] Dark Reading - Desktop Security - Here Comes the (Web)Fuzz - Security News Analysis On Feb 27, 2007, at 3:33 AM, Steven M. Christey wrote: > Given the complex manipulations that can work in XSS attacks (see > RSnake's > cheat sheet) as well as directory traversal, combined with the sheer > number of potential inputs in web applications, multipied by all the > variations in encodings, I wouldn't be surprised if they were > effective in > finding those kinds of implementation bugs, even in well-designed > software. Although successfully diagnosing some XSS without live > verification smells like a hard problem akin to the Ptacek/Newsham > "vantage point" issues in IDS. > > With the track record of non-web fuzzers and PROTOS style test > suites, why > do you think web app fuzzing is less likely to succeed? It's not so much that I don't think fuzzing is useful, it's that I don't see "one size fits all" fuzzing _products_ being useful. To me, it gets to an issue of informed vs. uninformed (or "white box" vs. "black box" if you prefer) testing. While they're both useful and should both be exercised, I believe (though I have no hard statistics to validate) that issues of coverage/state are always going to doom uninformed testing to being less effective than informed testing. For a fuzzer to be really meaningful, I believe that a "smart fuzzing" approach is going to be the best bet, and that makes it hard for a "one size fits all" product solution to be feasible. To do smart fuzzing, a lot of setup time is necessary in establishing an appropriate test harness and cases that fully exercise the files, network interface data, user data, etc., that the software is expecting. Perhaps I'm totally off base, and I invite any product folks here to chime in and correct my misconceptions. Cheers, Ken ----- Kenneth R. van Wyk SC-L Moderator KRvW Associates, LLC http://www.KRvW.com ---------------------------------------------------------------------------- This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ---------------------------------------------------------------------------- _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
