On Wed, Feb 4, 2009 at 7:26 PM, Paco Hope <p...@cigital.com> wrote: > > Andy also said "I think we lose something when we start saying 'everything > is > relative.'" I think we lose something more important if we try to impose > abolutes: we lose the connection to the business. No business operates on > absolutes and blind imperatives. Few, if any, profit-focused businesses > dogmatically fix all remotely exploitable SQL injections. Every business > looks > pragmatically at these things. Fixing the bug might cause the release of > the > product to slip by 6 weeks or a major customer to buy a competitor's > product > this quarter instead of waiting for the release. It's always a judgment > call > by the business. Even if their goal and their track record is fixing 100% > of > sev 1 issues before release, you know that each sev 1 issue was considered > in > terms of its cost, impact, schedule delay and so on.
The ppint here though is that repeatable processes do matter. Having a standard of what constitutes a given severity of bug standardized in a policy statement is a good thing. Sure that is hard as every application is different, but you need a starting place. And so while my standards don't say "XSS always equals P1" they do say "XSS that can be discovered in an external facing application" or even slightly more generically than that. So my bug priority matrix does talk about business impact because that is what matters, but I still have to give real world examples to folks who aren't expert security testers of how to handle a bug when they come across it. And we need to provide clear guidance in standards because every single bug shouldn't require an ad-hoc trage process. > > It is an outstanding idea for infosec guys to provide security test cases, > or > the framework for them, to QA. That beats the heck out of what they usually > do. However, a bunch of test cases for XSS, CSRF, SQL injection and so on > will > not map easily to requirements or to the QA workflow. At what priority do > they > execute? When the business (inevitably) squeezes testing to try to claw > back a > day or two on the slipped schedule, can any of these security tests be left > out? Why or why not? Without hanging them into the QA workflow with clear > traceability, QA will struggle to prioritize them correctly and maintain > them. > Security requirements would make that priority and maintenance > straightforward. At this point I'm not disagreeing with you, but taking > your good approach and extending it a step farther. I undertsand this, but handing security requirements to QA folks without a set of reeatable test cases for doing them isn't going to help much, in mos organizations. James Whittaker doesn't work for me :) . And if you're developing web applications you're probably going to have some set of standardized testing you do. You need to have a repository of test cases for certain things, and I think testing for certain type of attacks is probably a decent starting point. Sure you want QA to own those, but if you're worried about buffer overflows you've going to have a bunch of standard test cases, test scenarios, test data (long input strings, inputs with null bytes in them, etc) that you're going to reuse a bunch of times so that each tester isn't starting from scratch when they see the security requirments - "Application must handle input properly and not crash." I don't think we're far off here in what we're saying, but repeatability is key. Leaving the interface with QA at the level of security requriements in a functional spec isn't going to cut it. And, you're probably going to have some standardized set of security requirements for a whole swath of your applications that you might not want to repeat ad-naseum in every single product/feature spec. This is the place for standards, policies, and testing guidelines so that this becomes just part of the regular QA cycle. -- Andy Steingruebl stein...@gmail.com
_______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
