Another big frustration: No-one seems to be making any real headway into the problem of actually measuring loss attributable to doing nothing - or, in other words, losses cradle to grave from operating insufficiently secure systems. People try to measure "ROI" from security, which is a ridiculous concept because it involves trying to measure a negative - i.e., this is how many times we DIDN'T lose $n - can't be done - or trying to measure how much competitive advantage only being hacked 20 vs. 50 times last year gave us as a company - or other such silly pseudo-measurements.
What I really want is: [1] Ability to measure the aggregate of losses attributable to a single degradation or failure in an ICT infrastructure (all layers) - not just immediate loss due to downtime or degraded performance, but all the costs involved in redirecting resources (i.e., to deal with incident response, forensics, restoring from backup, implementing COOP, etc.); implementing interim short and long-term workarounds, purchases and man-hours involved in achieving total recovery to a sustained acceptable working state (ideally the same or better state than pre-loss); investment in preemptiove actions, things, and extraordinary (not what I was already doing) risk management activities to prevent a recurrence; plus all the other things I've probably not thought of here that contribute to the WHOLE amount of loss (e.g., reputation loss, advertising and PR "reputation recovery" campaigns, legal fees, fines, preparations plus actual expenses involved in testifying in court and/or on Capitol Hill, ! additional tests and audits needed, etc.); [2] Ability to accurately determine which of my ICT-related losses can be attributed, in whole or in part (and, in the latter case, what %) to intentional malevolent actions by someone (direct or via supply chain or operational subversion or sabotage via malware, etc.) - and which losses can be attributed to stupid mistakes by someone. Once I can get a real grip on actual, complete loss amounts - not just the stuff that usually gets measured - I can then see if I really have struck the right balance between what I spend on security to avoid/prevent loss, and what I'm actually losing - so I can figure out if I need to adjust the equation. Also, being able to accurately identify all the "someones" involved in causing each loss - e.g., developers, integrators, users, administrators, etc. - while this level of attribution isn't necessary to quantify losses - would enable me not only to figure out if I'm spending the right amount, but if I'm spending the right amount on the right things. For example, if my losses are mainly down to crappy or subverted software, investment in mitigating end-user risk is going to be of less value than investment in correcting SLDC deficiencies. In short, every time I read about a new attempt to measure security, it's always either too granular or not granular enough, and I'm not seeing any credible efforts to apply analysis across all measurement data to actually build a COMPLETE picture not only of the current "security situation", but of the whole cost of security - what it is, and more importantly, what it should be. === Karen Mercedes Goertzel, CISSP Senior Lead Scientist Booz Allen Hamilton 703.698.7454 goertzel_ka...@bah.com "Answers are easy. It's asking the right questions which is hard." - The Doctor ________________________________________ From: Jeffrey Walton [noloa...@gmail.com] Sent: 07 July 2014 14:56 To: Goertzel, Karen [USA] Cc: Secure Code Mailing List Subject: Re: [SC-L] [External] Re: SearchSecurity: Medical Devices and Software Security > Ever since I read an article about the challenges of remote laser surgery > being done by doctors at the Naval Hospital in Bethesda, MD, via satellite > link on wounded soldiers in Iraq, I've been warning for years about the need > to apply software assurance principles to the development and testing - and > SCRM to the acquisition - of medical devices and their embedded software. https://en.wikipedia.org/wiki/Therac-25 FTW! > What I want to know is this: When is someone who can actually make a > difference going to FINALLY figure out the real potential hazards of the > Internet of Things. +1. Dr. Geer has already warned about it at http://www.lawfareblog.com/2014/04/heartbleed-as-metaphor/. Can you imagine the IoT, with medical devices and avionics packages, running around with little to no testing and little more that the browser security model. Clear the cache to erase the evidence!!! > Manufacturers of the latter need to stop trying so bloody hard to "improve" > products that no longer need improvement. This is a political problem rooted in software liability laws (or lack thereof). Too many carrots, not enough sticks.... As it stands, its cost effective to do nothing. The risk analysis equations need to be tipped in favor of the consumer or user. One it starts costing money to do nothing, doing nothing will no longer be economically feasible. The market will drive meaningful change (as opposed to the water downed legislation with no teeth bought and paid for by lobbyist and special interests). Jeff On Mon, Jul 7, 2014 at 10:52 AM, Goertzel, Karen [USA] <goertzel_ka...@bah.com> wrote: > Ever since I read an article about the challenges of remote laser surgery > being done by doctors at the Naval Hospital in Bethesda, MD, via satellite > link on wounded soldiers in Iraq, I've been warning for years about the need > to apply software assurance principles to the development and testing - and > SCRM to the acquisition - of medical devices and their embedded software. I'm > delighted to see someone with your influence start warning those who confuse > software correctness and safety with software security of the potential havoc > that can potentially be wrought by malevolent actors as these little widgets > become increasingly networked and even Internet-accessible. > > What I want to know is this: When is someone who can actually make a > difference going to FINALLY figure out the real potential hazards of the > Internet of Things. Certain physical systems and devices really should NEVER > be connected to the public Internet - e.g., most Industrial Control Systems, > all medical devices, any plane, train, or automobile. And others really never > NEED to be Internet-connected. I mean, do we really, REALLY need to be able > to access our refrigerators or washing machines over the Web? Aren't we all > growing obese enough without making things so bloody convenient that we > needn't even walk the 20 feet from the bedroom to the kitchen or laundry room > to program the coffee maker or start another rinse cycle? > > Manufacturers of the latter need to stop trying so bloody hard to "improve" > products that no longer need improvement. There does come a time when a > technology goes as far as it can go - and any further attempts to "improve" > it are either purely cosmetic, unnecessary, or dangerous. I wish all these > manufacturers who waste their times trying to invent a better toaster would, > instead, invent something entirely new to solve a problem that hasn't already > been solved quite adequately for many decades. No wonder American > manufacturing is no longer competitive. All they do is continually rearrange > deck chairs on the Titanic to improve the view as the boat sinks, instead of > inventing a new means of transportation that actually CANNOT be taken down by > an iceberg. > > > === > Karen Mercedes Goertzel, CISSP > Senior Lead Scientist > Booz Allen Hamilton > 703.698.7454 > goertzel_ka...@bah.com > > "Answers are easy. It's asking the right questions which is hard." > - The Doctor > > ________________________________________ > From: SC-L [sc-l-boun...@securecoding.org] on behalf of security curmudgeon > [jeri...@attrition.org] > Sent: 06 July 2014 01:21 > To: Gary McGraw > Cc: Chandu Ketkar; Secure Code Mailing List > Subject: [External] Re: [SC-L] SearchSecurity: Medical Devices and Software > Security > > On Mon, 30 Jun 2014, Gary McGraw wrote: > > : Chandu Ketkar and I wrote an article about medical device security based > : on a talk Chandu gave at Kevin Fu?s Archimedes conference in Ann Arbor. > : In the article, we discuss six categories of security defects that > : Cigital discovers again and again when analyzing medical devices for our > : customers. Have a look and pass it on: > : > : http://bit.ly/1pPH56p > : > : As always, your feedback is welcome. > > Per your request, my feedback: > > Why do so many security professionals think we need yet another article on > medical devices that give a high-level overview, that ultimately boils > down to "medical devices are not secure"? > > We see these every month or three, and have for a long time. Other than > medical vendors who are very resistent to the idea that their devices have > issues, who is this written for? Who exactly outside medical vendors think > that those devices are secure? > > These articles do nothing.. absolutely nothing, to fix problems. They are > bandwagon articles jumping on the 'medical security' wave that has some > attention right now. Everyone writing these articles seems to be > completely new to the medical arena. Most that write this crap that I have > talked to can't speak to any of the history of medical disclosures. Names > like Fu and Halperin are foreign to them, and the importance of 1985 in > the timeline of medical issues is lost on them. If you find yourself > Googling any of those, thanks for proving my point. > > This shit is not new. These articles are NOT advancing our field or the > medical field. Sure, you are getting a slice of attention for the issue, > but mostly in our echo chamber. > > Finally, your intro. "Since 1996 my company has analyzed hundreds of > systems..." Really? Hundreds? You might want to fix that, else you come > across as complete n00bz in the industry. I've done single engagements > that involved tends of thousands of machines. Perhaps you want to qualify > that to mean hundreds of vendors? Hundreds per months/year? > > To illustrate I am not the only one who feels this way: > https://twitter.com/attritionorg/status/485652525589086209 > > 1 minute later: > https://twitter.com/SteveSyfuhs/status/485652988044656640 > > Seriously, dare to evolve. _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates _______________________________________________
