[SC-L] Fwd: SCARE metrics and tool release

2007-11-30 Thread Kenneth Van Wyk

Reposted with permission, FYI...

Cheers,

Ken
SC-L Moderator

Begin forwarded message:


From: Pete Herzog [EMAIL PROTECTED]
Date: November 30, 2007 10:30:18 AM EST
To: [EMAIL PROTECTED]
Subject: SCARE metrics and tool release

Hi,

Scare, the Source Code Analysis Risk Evaluation tool for measuring  
security complexity in C source code is now available.  The tool is  
written to support the OpenTC project (opentc.net) as the SCARE  
methodology project available at:


http://www.isecom.org/scare

We have done some test cases with the tool already do track trends  
in Xen and are now working on measuring trends in the Linux Kernel.


USE
The SCARE analysis tool is run against source code.  Currently only  
C code is supported.  The ouput file will contain all operational  
interactions possible which need controls (the current version does  
not yet say if and what controls are already there).  At the bottom  
of the list are three numbers: Visibilities, Access, and Trusts.   
These 3 numbers can be plugged into the RAV Calculation spreadsheet  
available at isecom.org/ravs.  The Delta value is then subtracted  
from 100 to give the SCARE percentage which indicates the complexity  
for securing this particular application.  The lower the value, the  
worse the SCARE.


Trends in Xen:

XEN ver. VisAccessesTrustsSCAREDelta

3.0.3_0   1   3142857758.26-41.74
3.0.4_1   1   3113106057.79-42.21
3.1.0 1   3163313957.43-42.57

As you can see, the security complexity of Xen is getting worse due  
to the increased numbers of Trusts (reliance on external variables  
which a user can manipulate as an input). Trust attacks can be  
tested according to the 4th point of the 4 Point test process in the  
OSSTMM 3: Intervention - changing resource interactions with the  
target or between targets.


At this stage, the tool cannot yet tell which interactions have  
controls already or if those controls are applicable however once  
that is available it will change the RAV but not the SCARE.  The  
SCARE will also not yet tell you where the bugs are in the code  
however if you are bug hunting, it will extract all the places where  
user inputs and trusts with user-accessible resources can be found  
in the code.



We need help!  We are looking for people to help us complete the  
SCARE methodology, add new programming languages to the tool, as  
well as even making a windows binary version for those who do not  
code in Linux. Contact me if you can do this.


Sincerely,
-pete.

--
Pete Herzog - Managing Director - [EMAIL PROTECTED]
ISECOM - Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.isestorm.org
---
ISECOM is the OSSTMM Professional Security Tester (OPST),
OSSTMM Professional Security Analyst (OPSA), and Hacker Highschool
Teacher certification authority.






smime.p7s
Description: S/MIME cryptographic signature
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Insecure Software Costs US $180B per Year - Application and Perimeter Security News Analysis - Dark Reading

2007-11-30 Thread Shea, Brian A
IMO the path to changing the dynamics for secure coding will reside in
the market, the courts, and the capacity of the software industry to
measure and test itself and to demonstrate the desired properties of
security, quality, and suitability for purpose.  In today's market we do
well in suitability for purpose (aka marketing then testing, pilot, and
purchase) but I believe we do poorly at security and quality.

Rather than try to tax software for being bad, it will likely end up
being more successful to reward them for being good in the form of
market support and sales.  That dynamic will probably work better, and
will empower the companies and individuals (who choose to get this
involved) to make the choices of security and quality against cost or
convenience.  The punishment for bad software is lost sales and eventual
loss of the use of that product within the market.

How?
Software vendors will need a 3 tier approach to software security:  Dev
training and certification, internal source testing, external
independent audit and rating.  The open source version of this can be
the same, but applied more individually or at the derivative product
level (ie if I make a Linux based appliance from open source, I become
the owner of the issues in that Linux derivative)

The legal side will need to alter the EULA away from a hold harmless
model to one where vendors and software buyers can assert that the
software is expected to perform at certain security or quality levels,
and have a known backing from a legal recourse side.  Companies that
make/sell software and can be sued offer more recourse than open source,
but that's not better or worse just different.  The buyer can choose the
degree of security and quality rating (based on the audit etc) and the
legal recourse they want via the SLA or choice to use open source.  For
a high assurance system one might choose a software vendor with a
contract and SLA, while also choosing open source for lower assurance
efforts that come with less recourse but less cost too.  This opens a
market for companies who choose to resell open source, but provide the
support and assurances.  It also allows anyone to choose which factors
are important, and buy / use accordingly.  

If choice results in downstream impacts, then the deployer of the
software is initially accountable, and they must determine if they have
recourse via SLA or contract to their provider.  If they accepted risk
of a non-supported software package, then their deployment and the
ensuing harm is their responsibility.  If they have recourse, as the
saying goes they pass the savings on.

Home users are also empowered if they choose to be, but overall they
gain in two major ways.  The market will be driven to more secure
software over time without their direct knowledge (as companies and
governments choose to require software to be more secure) and they can
benefit from any legal recourses that are available for notable security
failures or quality gaps. 

Using these factors anyone could make decisions based on the need for
recourse (courts), assurance (market), and quality (industry rating and
standards for security and quality) and come away with software that
meets their needs in each area, without excluding open or closed source
or leaving the corporate / consumer customers unprotected.

DISCLAIMER: Views are my own, and not those of my employer, and were
generated over a cup of coffee in a fairly stream of consciousness kind
of way.  Grains of salt not supplied, but are recommended when consuming
the contents. :)

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Leichter, Jerry
Sent: Friday, November 30, 2007 6:28 AM
To: der Mouse
Cc: SC-L@securecoding.org
Subject: Re: [SC-L] Insecure Software Costs US $180B per Year -
Application and Perimeter Security News Analysis - Dark Reading

|  Just as a traditional manufacturer would pay less tax by
|  becoming greener, the software manufacturer would pay less tax
|  for producing cleaner code, [...]
| 
|  One could, I suppose, give rebates based on actual field experience:
|  Look at the number of security problems reported per year over a
|  two-year period and give rebates to sellers who have low rates.
| 
| And all of this completely ignores the $0 software market.  (I'm
| carefully not saying free, since that has too many other meanings,
| some of which have been perverted in recent years to mean just about
| the opposite of what they should.)  Who gets hit with tax when a bug
| is found in, say, the Linux kernel?  Why?
I'll answer this along my understanding of the lines of the proposal at
hand.  I have my doubts about the whole idea, for a number of reasons,
but if we grant that it's appropriate for for-fee software, it's easy
decide what happens with free software - though you won't like the
answer:  The user of the software pays anyway.  The cost is computed in
some other way than as a percentage of the 

Re: [SC-L] Insecure Software Costs US $180B per Year - Application and Perimeter Security News Analysis - Dark Reading

2007-11-30 Thread der Mouse
 Just as a traditional manufacturer would pay less tax by
 becoming greener, the software manufacturer would pay less
 tax for producing cleaner code, [...]
 And all of this completely ignores the $0 software market.  Who
 gets hit with tax when a bug is found in, say, the Linux kernel?
 [T]f we grant that [the idea is] appropriate for for-fee software,
 it's easy decide what happens with free software - though you won't
 like the answer:  The user of the software pays anyway.

Well, it's full of enforcement issues; with for-pay, the point of sale
is a comparatively well-regulated point at which taxes can be applied,
but there is no such convenient choke-point for gratuit software.  How
would you even *find* all the relevant users?

Also, in the open-source world, people mix-and-match software to a
degree not seen in the closed-source world.  Does everyone who ever
downloaded a copy pay?  Everyone who's still running it?  Everyone who
ever ran it?  What about people who fixed the relevant bug themselves?

The only answers I can see are (1) to completely forbid software
sharing between end users, even when it's not against copyright law, or
(2) a massive DRM-style invasion of everyone's machines, so as to
report exactly what software they're running to some enforcement
authority.  I can't see either one flying.

And, incidentally, why would you think I wouldn't like that answer?  As
far as I know I'm not under any jurisdiction considering such a stupid
idea (yes, I consider it stupid), and if some other jurisdiction wants
to break their software industry that badly, it's their lookout.

 The argument the author is making is that security problems impose
 costs on *everyone*, not just on the party running the software.
 [...externalities...]
 Imposing a tax is the classic economic answer to such a market
 failure.  The tax's purpose is (theoretically) to transfer the
 externalized costs back to those who are in a position to respond.
 In theory, the cost for security problems - real or simply possible;
 we have to go with the latter because by the time we know about the
 former it's very late in the game -

So?  Why is that a problem?  It seems to me that someone who runs, say,
Windows, with all its horrible security record, in such a way as to not
cause a problem (this is not a hypothetical case), should not be taxed,
because that user is not imposing any externalized costs on the world
at large.

There's a problem finding everyone who's offended, but it's no worse
than the problems of finding all users of a piece of gratuit software.

 should be born by those who develop the buggy code, and by those who
 choose to use it.

I can argue both ways wrt imposing it on the developers.  Often enough,
the bugs are not bugs, but rather an end user misapplying software.
I've often enough written software that was perfectly fine in its
intended application but, if misapplied, could be a risk.

/~\ The ASCII   der Mouse
\ / Ribbon Campaign
 X  Against HTML   [EMAIL PROTECTED]
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Insecure Software Costs US $180B per Year - Application and Perimeter Security News Analysis - Dark Reading

2007-11-30 Thread Kenneth Van Wyk

On Nov 29, 2007, at 6:35 PM, Leichter, Jerry wrote:
So he's not completely naive, though the history of security metrics  
and

standards - which tend to produce code that satisfies the standards
without being any more secure - should certainly give on pause.

One could, I suppose, give rebates based on actual field experience:
Look at the number of security problems reported per year over a two-
year period and give rebates to sellers who have low rates.



Right, so this is where I believe the entire idea would fall apart.  I  
don't think we have adequate metrics today to measure products  
fairly.  Basing the tax on field experience would also be problematic  
to measure well, although I could see this leading to development  
organizations getting some sort of actuarial score.


But the real problem with it, as I said, is metrics.  Should it be  
based on (say) defect density per thousand lines of code as reported  
by (say) 3 independent static code analyzers?  What about design  
weaknesses that go blissfully unnoticed by code scanners?  (At least  
the field experience concept could begin to address these over time,  
perhaps.)


I do think that software developers who produce bad (security) code  
should be penalized, but at least for now, I still think the best way  
of doing this is market pressure.  I don't think we're ready for more,  
on the whole, FWIW.  But _consumers_ wield more power than they  
probably realize in most cases.


Cheers,

Ken

-
Kenneth R. van Wyk
SC-L Moderator
KRvW Associates, LLC
http://www.KRvW.com






smime.p7s
Description: S/MIME cryptographic signature
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Insecure Software Costs US $180B per Year - Application and Perimeter Security News Analysis - Dark Reading

2007-11-30 Thread Steven M. Christey

On Fri, 30 Nov 2007, Shea, Brian A wrote:

 Software vendors will need a 3 tier approach to software security:  Dev
 training and certification, internal source testing, external
 independent audit and rating.

I don't think I've seen enough emphasis on this latter item.  A
sufficiently vibrant set of independent testing organizations that follows
some established procedures would be one way for customers to get an
independent guarantee of software's (relative) security.  This in turn
could put pressure on other vendors to follow suit.

The challenges would be defining what those procedures should be,
maintaining them in a way so that they remain relevant, convincing
existing research organizations to participate, and handling the problem
of free (as in beer) software.

A gazillion years ago, John Tan of the L0pht proposed an Underwriters
Laboratories for software, and maybe its time is almost upon us.

- Steve
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___


Re: [SC-L] Insecure Software Costs US $180B per Year - Application and Perimeter Security News Analysis - Dark Reading

2007-11-30 Thread Leichter, Jerry
|  Just as a traditional manufacturer would pay less tax by
|  becoming greener, the software manufacturer would pay less tax
|  for producing cleaner code, [...]
| 
|  One could, I suppose, give rebates based on actual field experience:
|  Look at the number of security problems reported per year over a
|  two-year period and give rebates to sellers who have low rates.
| 
| And all of this completely ignores the $0 software market.  (I'm
| carefully not saying free, since that has too many other meanings,
| some of which have been perverted in recent years to mean just about
| the opposite of what they should.)  Who gets hit with tax when a bug
| is found in, say, the Linux kernel?  Why?
I'll answer this along my understanding of the lines of the proposal at
hand.  I have my doubts about the whole idea, for a number of reasons,
but if we grant that it's appropriate for for-fee software, it's easy
decide what happens with free software - though you won't like the
answer:  The user of the software pays anyway.  The cost is computed in
some other way than as a percentage of the price - it's no clear exactly
how.  Most likely, it would be the same tax as is paid by competing
non-free software with a similar security record.  (What you do when
there is no such software to compare against is an interesting problem
for some economist to work on.)

The argument the author is making is that security problems impose costs
on *everyone*, not just on the party running the software.  This is a
classic economic problem:  If a party can externalize its costs - i.e.,
dump them on other parties - its incentives become skewed.  Right now,
the costs of security problems for most vendors are externalized.
Where do they do?  We usually think of them as born by that vendor's
customers.  To the degree that's so, the customers will have an
incentive to push costs back on to the vendor, and eventually market
mechanisms will clean things up.  To some degree, that's happened to
Microsoft:  However effective or ineffective their security efforts,
it's impossible to deny that they are pouring large sums of money
into the problem.

To the degree that the vendors' customers can further externalize the
support onto the general public, however, they have no incentive to
push back either, and the market fails.  This is pretty much the case
for personal, as opposed to corporate, users of Microsoft's software.
Imposing a tax is the classic economic answer to such a market failure.
The tax's purpose is (theoretically) to transfer the externalized costs
back to those who are in a position to respond.  In theory, the cost
for security problems - real or simply possible; we have to go with
the latter because by the time we know about the former it's very
late in the game - should be born by those who develop the buggy
code, and by those who choose to use it.  A tax on the code itself
directly takes from the users of the code, indirectly from the
vendors because they will find it more difficult to compete with
vendors who pay lower tax rates, having written better code.  It's
much harder to impose the costs directly on the vendors.  (One way
is to require them to carry insurance - something we do with, say,
trucking companies).

In any case, these arguments apply to free software in exactly the same
way they do for for-fee software.  If I cars away for free, should I be
absolved of any of the costs involved if they pollute, or cause
accidents?  If I'm absolved, should the recipients of those cars also be
absolved?  If you decide the answer is yes, what you've just decided
is that *everyone* should pay a hidden tax to cover those costs.  In
what why is *that* fair?
-- Jerry
___
Secure Coding mailing list (SC-L) SC-L@securecoding.org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
___