Re: [SC-L] Building Security In vs Auditing

2007-01-04 Thread Paco Hope
 Gary, I would love a little refinement of the benefits to badnessometers.
 Let's say I get a tool to tell me something I already suspect is wrong,
 what percentage of the population are better than they expected?

I won't speak for Gary, but working a few doors down I have seen a few of the 
same things he has.

Occasionally developers internally run free tools and surrepetitiously fix 
problems that the tools find (this happens in some cultures where management is 
particularly antagonistic towards security as a first class concern). In those 
unusual instances, I could see the results of a badnessometer coming out better 
than expected. Management would perceive that such things had never been run, 
and would be pleasantly surprised to learn that the sky might not be falling. 
Other than that, few people run a tool for the first time and see results 
better than they expected. Tools codify all manner of stuff that your 
developers almost certainly do not know how to check for (and if they do, they 
probably don't have time).

 Is it better to do such a badness test by doing a POC with one of the
 tool vendors in this space or do I get additional lift by going with
 a consulting firm in this regard?

I'm a consultant, take that as implied bias. But, I think you do get lift, and 
here's my analogy. Consider yourself a handy guy around the house who is going 
to do something moderately complicated, like redo a whole bathroom. You can buy 
all the tools and read all the books on how to do it for a lot less money than 
hiring a contractor to do the whole thing.  There's some pretty specialized 
tools in plumbing, though, and they're tools you probably haven't used more 
than once or twice. Do you gain some extra insight into the use of those tools 
by hiring someone experienced to assist on the complicated parts? I think so. 
That someone experienced will come in, help you wield the unfamiliar tool, show 
you some things that he has experienced, and get you through the difficult 
parts. Then you, being the handy guy you are, are left to finish the bathroom, 
doing things you know how to do well.

I think this analogy holds with a lot of the tools in security. You learn a lot 
by getting the experience someone brings, assuming you get a good someone. We, 
for example, have run a bunch of tools on a lot of different code bases. We 
know which rules tend to be alarmist and which ones are really important if 
they fire. Tool vendors won't give you that objectivity on their own tool, and 
some of the sales engineers don't have the insight into their own tool to know 
which warnings are just noise and which warnings are a big deal. A consultant 
can help you have a bake-off between tools, whereas a tool vendor typically 
lacks that objectivity.

Paco




This electronic message transmission contains information that may be
confidential or privileged.  The information contained herein is intended
solely for the recipient and use by any other party is not authorized.  If
you are not the intended recipient (or otherwise authorized to receive this
message by the intended recipient), any disclosure, copying, distribution or
use of the contents of the information is prohibited.  If you have received
this electronic message transmission in error, please contact the sender by
reply email and delete all copies of this message.  Cigital, Inc. accepts no
responsibility for any loss or damage resulting directly or indirectly from
the use of this email or its contents.
Thank You.


___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Building Security In vs Auditing

2007-01-03 Thread McGovern, James F (HTSC, IT)
Gary, I would love a little refinement of the benefits to badnessometers. Let's 
say I get a tool to tell me something I already suspect is wrong, what 
percentage of the population are better than they expected? The reason why I 
ask this question is that in our culture if I have a sense something is wrong, 
it usually isn't that difficult to find metrics as to why it is bad and 
therefore may have the perception of crying wolf as there are lots of bad 
things in all IT systems. Sometimes, going from good to great is a better 
approach than fixing bad and going to good.

Is it better to do such a badness test by doing a POC with one of the tool 
vendors in this space or do I get additional lift by going with a consulting 
firm in this regard (other than an opportunity to be smoozed regarding 
subsequent engagements and reused powerpoints and collateral from other gigs)?

What would it take to get some industry analyst coverage in this space? Lots of 
folks may be of the belief that it is a waste of time bothering but I would 
love to at least know if any of the firms here have at least made the effort.

-Original Message-
From: Gary McGraw [mailto:[EMAIL PROTECTED]
Sent: Tuesday, January 02, 2007 1:35 PM
To: McGovern, James F (HTSC, IT); sc-l@securecoding.org
Subject: RE: [SC-L] Building Security In vs Auditing


Hi all,

Very good questions.  

I think a service like the one you describe would be useful mostly as a way of 
identifying the depth of the problem.  Simply wielding a tool as a consultant 
does nothing to train the guys creating bugs not to do so in the future...and 
so the market will correct that over time in an efficient way.  But the fact 
remains that many potential customers and users of static analysis tools have 
no idea how much of a mess they have.  An outsourcing approach could help with 
that.  They'll find out they need em.

I believe so strongly in the do anything to get started thing that I also 
endorse the use of (really amazingly silly) application security testing tools. 
 I call these badnessometers (see chapter 1 of software security...and ken's 
slides for that matter).  But knowing that your web code sucks is better than 
remaining completely clueless.

In the end, tool integration *into dev* is the key to success with static 
analysis.  Many of our customers are having huge enterprise-wide success 
because they are learning to use, feed, tune, and train dev about these tools.  
The best are recycling the things they learn about their code back into 
training (and into better rules to enforce).

gem

company www.cigital.com
podcast www.cigital.com/silverbullet
book www.swsec.com.



*
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information.  If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited.  If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*


___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Building Security In vs Auditing

2007-01-02 Thread ljknews
At 9:46 AM -0500 1/2/07, McGovern, James F (HTSC, IT) wrote:

 I read a recent press release in which a security vendor (names removed
 to both protect the innocent along with the fact that it doesn't matter
 for this discussion ) partnered with a prominent outsourcing firm. The
 press release was carefully worded but if you read into what wasn't said,
 it was in my opinion encouraging something that folks here tend to fight
 against. The outsourcing firm would use this tool in an auditing capacity
 for whatever client asked for another service but it would not become
 part of the general software development lifecycle for all projects. 
 
 - It didn't mention any notion of all developers within the outsourcing
 firm having tools on their desktop to audit as they develop

From the information supplied, it is not clear that the tool is something
appropriate for the development environment.  I develop a tool that could
be used in a (certain) development environment, but that would only tell
how the development environment was secured, having no effect on the degree
to which the outsourced code was secure.

 - It didn't mention any notion of training all developers within the
 outsourcing firm on secure coding practices

From the information supplied, it is not clear that the security vendor
is one that would be involved in training anyone.  Limitations on a
joint press release (one that names another company) are subject to
severe negotiations.  Even if the security firm _was_ going to do what
you suggest, I can see a PR flack at the outsourcing firm resisting any
public suggestion that any of their staff needed further training on any
aspect of data processing.
-- 
Larry Kilgallen
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___