[SC-L] FW: How Can You Tell It Is Written Securely?
Hello Jim, I tend to disagree with your statement that security requirements should be part of contractual agreements or added to a purchase order. In the Real World (™ ☺) this does not work. Once signed, contracts are never looked at again, unless the shit hits the fan and someone must be blamed for something that went wrong. Development teams (which is a lot broader than the term developers) _never_ read contracts or look at purchase orders. In itself, security statements in a paper nobody reads but the corporate lawyers will not make any application more secure. Lawyers will go over contracts and purchase orders and I shudder to what will happen if they disagree on a technical term in your ‘security requirements’. Please define (in legal terms) “configuration file”, “user data”, “regular expression”, “form nonce” …. Note that none of the business people responsible for the ‘security’ of the application/business process will understand anything of this, although they are ultimately responsible … IMHO all this (detailed technical requirements in contracts) is part of the security theater and does not aid in making applications more secure (whatever that term – ‘secure’ - means), it just aids in creating auditor checklists that can be completed by junior auditors that don’t know anything about application development. Or let the process control people create a CMM dashboard with a green light. The most you can expect from a contract is very high level statements, and once they are high-level, will they aid in the objective o have secure software? In my experience, once a contract is signed, the development team will create the functional requirements (or whatever this document is called in their methodology), and this document will result in a functional specification document, that requires sign-off by the customer before one line of code is written (don’t get me started on ‘agile’ development). If it is not in this specification, it will not be part of the code nor the tests afterwards nor the acceptance tests of the customers. The most important thing to notice here is that the trick to make the application more ‘secure’ (or at least abide to what people call here ‘secure development best practices’ – who decides what is ‘best practice’ anyway...) is to have the security requirements part of the standard documentation used in their methodology. Train the person who creates the requirements document to rewrite statements as ‘click –a- to do something’ in ‘click –a- to do something and ignore all other inputs”. There is of course a lot more to this, but this email is already getting too long. ☺ Recommendation: Do not create a separate ‘security requirements’ document because it will not be used (real life example: “yes there is always a statement in our functional requirements that we must include the requirements of the security policy/requirements document but we never look at those documents, because we are already developing for many years so we should be OK …”. Important remark: most security people will tell that you must do ‘security testing’ but completely fail to understand how bigger development shops work. All tests are created from the functional specification document! Need better input validation? Rewrite the functional specs! Note that there are a lot of standard methodologies to create tests from a functional spec document – all unknown to most security people - and that many big development shops completely outsource this part of the development process. Note that the above in itself is not enough, some other design/architecture/implementation/test ‘controls’ (I hate this word) are necessary. But start with the basics: secure development starts with having a good writer – not a coder, tester, architect, designer - with a common knowledge of security for your functional requirements/specification document. Do not 'add' a security layer on top of something, but 'include' security. Please note that all opinions are my own and that I’m not aware of any independent – and unbiased - studies that proof or disproof anything in mine or your email. ☺ Unfortunately the Information Security Profession has not reached the maturity to look and study what really matters in an objective way. So you are stuck with my biased opinion for now. But this opinion is based on a career as developer, security consultant, auditor and security manager so I hope it gives valuable insights or start a discussion so I can learn, maybe change my insights and better serve my customers. Sincerely, Herman Stevens [EMAIL PROTECTED] http://www.astyran.be From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jim Manico Sent: donderdag 27 november 2008 22:38 To: Mark Rockman Cc: Secure Mailing List Subject: Re: [SC-L] How Can You Tell It Is Written Securely? OK. So you decide to outsource your programming assignment to Asia and demand
Re: [SC-L] FW: How Can You Tell It Is Written Securely?
Steven, There are more than several managers of application security programs for F-100 companies that have written security requirements into their SLA's with outsourced development firms. One example uses application penetration testing and vulnerability assessment findings to enforce SLA requirements. Some companies employ an entire team of people to perform both whitebox and blackbox testing in addition to external/3rd-party assessments. And as you later state, security requirements should be written into the functional requirements, and not handed off in its own category or as some appendix document. -Marcin tssci-security.com On Mon, Dec 1, 2008 at 9:59 AM, Herman Stevens [EMAIL PROTECTED] wrote: I tend to disagree with your statement that security requirements should be part of contractual agreements or added to a purchase order. In the Real World (™ ☺) this does not work. Once signed, contracts are never looked at again, unless the shit hits the fan and someone must be blamed for something that went wrong. Development teams (which is a lot broader than the term developers) _never_ read contracts or look at purchase orders. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] FW: How Can You Tell It Is Written Securely?
Hello Marcin, I agree with your statement that many companies have some requirements in their SLA's with outsourced development firms. However, if these are really F-100 businesses they usually have all non-core processes out-sourced (because a Big4 company told them that would reduce costs), the relationship management with the outsourced companies is also out-sourced (probably to the same Big4). This means after a few years all knowledge has left the company and if a Request For Proposal needs to be written (e.g. for a new application supporting their core business functions) this is outsourced again to the same Big4 since the company itself does not even have the required knowledge to write its own RFPs ... I really doubt that anything that goes in that RFP (and ultimately in the contracts) will have any _real_ security value. Using penetration tests and vulnerability requirements might be part of the acceptance process, but I do not call these tests _security_ requirements. They are acceptance requirements ... The original request asked for how can someone determine if an application is written in a secure manner. My reasoning is that this is the wrong question (the application must _be_ secure and for this there is no direct link with coding practices). And even if one can proof the application is written in a secure manner, this will not be enough to be secure (e.g. about 99% of all security relevant features are nowadays in the configuration, the customer will never issue a change request for a new java library of javascript library yet in many of my penetration tests I 'break' the application because of old libs, ...). I do not think that penetration tests and vulnerability assessments are a 'proof' that an application is written securely. I've seen many applications that were written horrendously but were very secure (in the sense that they abided to all security-relevant business requirements) and I have seen many applications written using the 'best practices' in coding and developed with very mature processes that could be hacked in minutes. So, are there any studies that proof that a company that performs some tests (e.g. pen-tests) or include security requirements in the contracts ultimately is better off than a company that does not do what we consider 'best practices'? And if we don't have that proof, shouldn't we be very prudent in what we advise to our customers? Please note that my company sells security related software and performs vulnerability assessments, so I'm not saying that these are useless (:)), but maybe there are better methods than penetrate patch or enforcing very heavy processes on innocent development teams... So, this is question to this list: Are we on the right track? Is application security really improving? Do we measure the correct things and in the correct way? My point of view is that only certain vulnerabilities are less common than in the early days just because of more mature frameworks, but not due to better processes or after the fact testing. Does this mean all efforts were vain? Or did the threat landscape change? And yes, there are many vendor driven statistics floating around but they really cannot be considered unbiased ... Lots of questions, maybe not all relevant for the Secure Coding list, but Secure Coding should have an final objective. Or not? Herman [EMAIL PROTECTED] -Original Message- From: Marcin Wielgoszewski [mailto:[EMAIL PROTECTED] Sent: maandag 1 december 2008 17:06 To: Herman Stevens Cc: SC-L@securecoding.org Subject: Re: [SC-L] FW: How Can You Tell It Is Written Securely? Steven, There are more than several managers of application security programs for F-100 companies that have written security requirements into their SLA's with outsourced development firms. One example uses application penetration testing and vulnerability assessment findings to enforce SLA requirements. Some companies employ an entire team of people to perform both whitebox and blackbox testing in addition to external/3rd-party assessments. And as you later state, security requirements should be written into the functional requirements, and not handed off in its own category or as some appendix document. -Marcin tssci-security.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] FW: How Can You Tell It Is Written Securely?
Asking about security in terms of an RFP is a big joke and reminds me of tactics I used in sixth grade when I used to figure out creative ways of answering a question by turning the question into an answer. One has to acknowledge that RFPs are not authoratative and are usually completed by sales teams who don't have adequate detail. So, instead of focusing on comprehensive documentation, you need to be agile and think more about working software. I believe that penetration tests are sadly too late in the process in order to be meaningful and only have value in scenarios where you can mandate that the presses be stopped and that they fix immediately before you give them a single penny. Avoid the contract stuff as well as you can't really put meaningful things into agreements. You have to acknowledge that if I were successful in putting say a clause into our next EMC agreement requiring all document management software to support XACML and be OWASP Top Ten free, do you think that a developer on the other end would even have a chance to read it and address going forward? Way too many degrees of separation between those who write contracts and those who implement software. Again, I think you need to measure developers and their participation in groups such as OWASP since it is observable and measurable without requiring a lot of effort. It is not a guarantee but can be a predictor... -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Herman Stevens Sent: Monday, December 01, 2008 1:55 PM To: Marcin Wielgoszewski Cc: SC-L@securecoding.org Subject: Re: [SC-L] FW: How Can You Tell It Is Written Securely? Hello Marcin, I agree with your statement that many companies have some requirements in their SLA's with outsourced development firms. However, if these are really F-100 businesses they usually have all non-core processes out-sourced (because a Big4 company told them that would reduce costs), the relationship management with the outsourced companies is also out-sourced (probably to the same Big4). This means after a few years all knowledge has left the company and if a Request For Proposal needs to be written (e.g. for a new application supporting their core business functions) this is outsourced again to the same Big4 since the company itself does not even have the required knowledge to write its own RFPs .. I really doubt that anything that goes in that RFP (and ultimately in the contracts) will have any _real_ security value. Using penetration tests and vulnerability requirements might be part of the acceptance process, but I do not call these tests _security_ requirements. They are acceptance requirements ... The original request asked for how can someone determine if an application is written in a secure manner. My reasoning is that this is the wrong question (the application must _be_ secure and for this there is no direct link with coding practices). And even if one can proof the application is written in a secure manner, this will not be enough to be secure (e.g. about 99% of all security relevant features are nowadays in the configuration, the customer will never issue a change request for a new java library of javascript library yet in many of my penetration tests I 'break' the application because of old libs, ...). I do not think that penetration tests and vulnerability assessments are a 'proof' that an application is written securely. I've seen many applications that were written horrendously but were very secure (in the sense that they abided to all security-relevant business requirements) and I have seen many applications written using the 'best practices' in coding and developed with very mature processes that could be hacked in minutes. So, are there any studies that proof that a company that performs some tests (e.g. pen-tests) or include security requirements in the contracts ultimately is better off than a company that does not do what we consider 'best practices'? And if we don't have that proof, shouldn't we be very prudent in what we advise to our customers? Please note that my company sells security related software and performs vulnerability assessments, so I'm not saying that these are useless (:)), but maybe there are better methods than penetrate patch or enforcing very heavy processes on innocent development teams... So, this is question to this list: Are we on the right track? Is application security really improving? Do we measure the correct things and in the correct way? My point of view is that only certain vulnerabilities are less common than in the early days just because of more mature frameworks, but not due to better processes or after the fact testing. Does this mean all efforts were vain? Or did the threat landscape change? And yes, there are many vendor driven statistics floating around but they really cannot be considered unbiased ... Lots of questions, maybe not all relevant for the Secure Coding list, but Secure
Re: [SC-L] FW: How Can You Tell It Is Written Securely?
I think adding clear security requirements at the contractual level is *fundamental and necessary* when yes? yesworking with vendors who are writing software for you. Don't we talk about software security as being just another integrated part of writing software, not as an external activity? I'm not talking about foolish general requirements like Be OWASP top 10 free that does not help anyone. I'm talking about clear, somewhat undebatable requirements like: 1) Add in CSRF protection by using a form nonce 2) Make every field maps to a unique regular expression. Ensure that this regular expression exists in a properties file so it can be configured without needing to recompile code. 3) Ensure that every piece of user-driven data is encoded via HTML Entity Encoding before it is displayed to any user. 4) Use only prepared statements and binding of variables when selecting from a database Etc. Sure, we want our vendors to be security educated (good luck finding them these days). But really, we need to get the job done. We need to hire vendors. It's likely that software development vendors are not super security educated since software security is so new. We need to communicate contractual requirements anyways and those agreements do indeed matter. Our best bet is to add in clear functional security requirements to our contractual agreements if we have any hope of them being built in in a cost effective manner. - Jim Asking about security in terms of an RFP is a big joke and reminds me of tactics I used in sixth grade when I used to figure out creative ways of answering a question by turning the question into an answer. One has to acknowledge that RFPs are not authoratative and are usually completed by sales teams who don't have adequate detail. So, instead of focusing on comprehensive documentation, you need to be agile and think more about working software. I believe that penetration tests are sadly too late in the process in order to be meaningful and only have value in scenarios where you can mandate that the presses be stopped and that they fix immediately before you give them a single penny. Avoid the contract stuff as well as you can't really put meaningful things into agreements. You have to acknowledge that if I were successful in putting say a clause into our next EMC agreement requiring all document management software to support XACML and be OWASP Top Ten free, do you think that a developer on the other end would even have a chance to read it and address going forward? Way too many degrees of separation between those who write contracts and those who implement software. Again, I think you need to measure developers and their participation in groups such as OWASP since it is observable and measurable without requiring a lot of effort. It is not a guarantee but can be a predictor... -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Herman Stevens Sent: Monday, December 01, 2008 1:55 PM To: Marcin Wielgoszewski Cc: SC-L@securecoding.org Subject: Re: [SC-L] FW: How Can You Tell It Is Written Securely? Hello Marcin, I agree with your statement that many companies have some requirements in their SLA's with outsourced development firms. However, if these are really F-100 businesses they usually have all non-core processes out-sourced (because a Big4 company told them that would reduce costs), the relationship management with the outsourced companies is also out-sourced (probably to the same Big4). This means after a few years all knowledge has left the company and if a Request For Proposal needs to be written (e.g. for a new application supporting their core business functions) this is outsourced again to the same Big4 since the company itself does not even have the required knowledge to write its own RFPs .. I really doubt that anything that goes in that RFP (and ultimately in the contracts) will have any _real_ security value. Using penetration tests and vulnerability requirements might be part of the acceptance process, but I do not call these tests _security_ requirements. They are acceptance requirements ... The original request asked for how can someone determine if an application is written in a secure manner. My reasoning is that this is the wrong question (the application must _be_ secure and for this there is no direct link with coding practices). And even if one can proof the application is written in a secure manner, this will not be enough to be secure (e.g. about 99% of all security relevant features are nowadays in the configuration, the customer will never issue a change request for a new java library of javascript library yet in many of my penetration tests I 'break' the application because of old libs, ...). I do not think that penetration tests and vulnerability assessments are a 'proof' that an application is written securely. I've seen many applications that were
Re: [SC-L] FW: How Can You Tell It Is Written Securely?
I am of the belief that the examples you provided are more requirements for your own staff. For example, shouldn't your business analysts define regular expressions and include them as functional requirements where the firm simply calls them? If you want regex's externalized into properties files, shouldn't this be more about specifying acceptance criteria for the overall design vs waiting to measure it against code. For number three, you would have to think about such a clause as it is an out if performance isn't met. If you work for a large enterprise, I would think that number 4 would be encompassed into asking them to align with consistent DB access practices where you hand them your coding standards. It is different to have coding standards and having clauses that ask others to adhere to them vs embedding coding type requirements into the contracts themselves. -Original Message- From: Jim Manico [mailto:[EMAIL PROTECTED] Sent: Monday, December 01, 2008 4:44 PM To: McGovern, James F (HTSC, IT) Cc: SC-L@securecoding.org Subject: Re: [SC-L] FW: How Can You Tell It Is Written Securely? I think adding clear security requirements at the contractual level is *fundamental and necessary* when yes? yesworking with vendors who are writing software for you. Don't we talk about software security as being just another integrated part of writing software, not as an external activity? I'm not talking about foolish general requirements like Be OWASP top 10 free that does not help anyone. I'm talking about clear, somewhat undebatable requirements like: 1) Add in CSRF protection by using a form nonce 2) Make every field maps to a unique regular expression. Ensure that this regular expression exists in a properties file so it can be configured without needing to recompile code. 3) Ensure that every piece of user-driven data is encoded via HTML Entity Encoding before it is displayed to any user. 4) Use only prepared statements and binding of variables when selecting from a database Etc. Sure, we want our vendors to be security educated (good luck finding them these days). But really, we need to get the job done. We need to hire vendors. It's likely that software development vendors are not super security educated since software security is so new. We need to communicate contractual requirements anyways and those agreements do indeed matter. Our best bet is to add in clear functional security requirements to our contractual agreements if we have any hope of them being built in in a cost effective manner. - Jim Asking about security in terms of an RFP is a big joke and reminds me of tactics I used in sixth grade when I used to figure out creative ways of answering a question by turning the question into an answer. One has to acknowledge that RFPs are not authoratative and are usually completed by sales teams who don't have adequate detail. So, instead of focusing on comprehensive documentation, you need to be agile and think more about working software. I believe that penetration tests are sadly too late in the process in order to be meaningful and only have value in scenarios where you can mandate that the presses be stopped and that they fix immediately before you give them a single penny. Avoid the contract stuff as well as you can't really put meaningful things into agreements. You have to acknowledge that if I were successful in putting say a clause into our next EMC agreement requiring all document management software to support XACML and be OWASP Top Ten free, do you think that a developer on the other end would even have a chance to read it and address going forward? Way too many degrees of separation between those who write contracts and those who implement software. Again, I think you need to measure developers and their participation in groups such as OWASP since it is observable and measurable without requiring a lot of effort. It is not a guarantee but can be a predictor... -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Herman Stevens Sent: Monday, December 01, 2008 1:55 PM To: Marcin Wielgoszewski Cc: SC-L@securecoding.org Subject: Re: [SC-L] FW: How Can You Tell It Is Written Securely? Hello Marcin, I agree with your statement that many companies have some requirements in their SLA's with outsourced development firms. However, if these are really F-100 businesses they usually have all non-core processes out-sourced (because a Big4 company told them that would reduce costs), the relationship management with the outsourced companies is also out-sourced (probably to the same Big4). This means after a few years all knowledge has left the company and if a Request For Proposal needs to be written (e.g. for a new application supporting their core business functions) this is outsourced again to the same Big4 since the company itself does not even have the required
Re: [SC-L] FW: How Can You Tell It Is Written Securely?
Some other thoughts that I haven't heard others mention? 1. OK, if you find that they didn't meet all the security requirements, will your business customers still want you to put it into production anyway? If the answer is yes, do you still want them to support it? How do we quantify who is responsible if a breach happens and you gave them a waiver. 2. security clauses have a side effect in contracts that others need to noodle. If you have a clause that can only be measured over a longer timespan, it tickers with revenue recognition. So, how long do you want folks to certify that things are secure. 3. I like secure coding as much as the next guy and checking for CSRF is a good thing. How about noodling requirements around logging such that if they didn't get it right upfront that you at least have something forensically useful for after the fact? 4. While we are all developers, do you think there is merit in addressing roles of vendors especially non-development? For example, is it valuable to have them have on staff a security architect with lots of credentials that is separate from the development lifecycle (distinct from being totally ivory tower or hands-off)? 5. How much more are folks willing to pay to build security in? This kinda doesn't align with going offshore to get cheapest resource. It is in their best interest to be an impediment to this goal and you need to define things in a more friendly manner. Coming out of the gate by throwing others under the bus probably will not get you what you desire (of course, it is a tactic I use way too much) This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
