Hi John, Thanks for the feedback. This is exactly what we were looking for. We've certainly sought simplicity in this model, even at the expense being incomplete. It's not necessarily aimed at the one man shop - it's aimed at any organization where secure software is just not an explicit top-level priority. It doesn't address any of the short-comings of any previous model because it's not an alternative to them. It's simply an observation of a seemingly natural - organic- progression of steps. I agree with you about its value. No organization matches this model completely - there are often additional steps, some that you mentioned, which one organization or another takes or where the order is slightly different than what we've outlined. You can think of the steps we've outlined as a line of best fit: the steps we've seen to be most common.
I'm often surprised to find security practitioners thinking they are way behind industry because they are are struggling to convince the lines of business to participate in security activities. One motivation for the model is to let those practitioners know that they're not alone. Case studies are a fantastic idea. We will add these to the model over time. We also want to be able to point to useful resources for people at each step, so if you (i.e. anyone reading this) has written relevant articles or whitepapers let me know. On Tue, Jul 19, 2011 at 4:43 PM, John Steven <jste...@cigital.com> wrote: > Paco, > > Thank you for cogently clarifying BSIMM. I'm a bit disappointed in the > community's ignorance regarding the model given it's both freely available > and Creative Commons licensed. Equally disappointing, to me, are positions > borne out of a "Just use [MyModel™: BSIMM || SAMM]" perspective. Rohit > asked: > > > If you're an actual practitioner who has lived through developing a > secure SDLC I'd love to hear your thoughts about the model's accuracy / > relevancy. > > Responses to this request would provide this mailing list's readership more > value. As one practitioner responsible for several SDL programs, I'll > respond ignoring Organic vs. BSIMM. I don't see much value in such a > comparison. > > [Is 'Organic' a model?] > > Yes. Paraphrasing one definition, a model is anything that abstracts a > system's factors in a way to helps its users quickly gain insight into the > subject's behavior. > > Inaccuracy isn't a fatal blow to a model, quoting Paul Wilmott's Manifesto > [BW1]: > > • I will remember that I didn't make the world and that > it doesn't satisfy my equations. > > • Though I will use models boldly to estimate value, I > will not be overly impressed by mathematics. > > • I will never sacrifice reality for elegance without > explaining why I have done so. Nor will I give the > people who use my model false comfort about its > accuracy. Instead, I will make explicit its > assumptions and oversights. > > • I understand that my work may have enormous effects > on society and the economy, many of them beyond my > comprehension. > > ...Navigators managed rather well with a "Flat Earth" hypothesis for some > time--no? So, we don't need over one hundred activities in our app. sec. > model in order to provide value. > > [Motivation] > Most of you know I respect Rohit a fair amount and so when I read his post, > you can imagine my thought, "In a world aware of BSIMM what is the value of > 'Organic'?" with honest curiosity, not disdain. I immediately guessed > 'Organic' was meant to address a common complaint regarding almost every > prior model: > > "I'm challenged applying <this> to smaller shops just > beginning their Application Security initiatives" > > Jim Bird has thoughtfully discussed the "one man shop" problem extensively > in his blog [JB1]. Rohit's own explanation mentions "no top down support" as > an indication of model applicability. > > [Accuracy] > 'Organic' ignores a lot of key components that even smaller shops already > have in place or care about improving. Three essential ones include 1) > measurement and iterative approach [JB2], 2) security policy [PC1], and 3) > security toolkits/frameworks [JB2][FM1]. While Rohit's post indicates > explicitly that things have been omitted, he focuses on having left out > architecture and related activities. > > To me, even if 'Organic' is designed to focus only on development > activities, ignoring a potential need for compliance to regulatory/security > policy, leveraging toolkits to make developers' jobs easier, or failing to > set up a measure-and-iterate loop are dire mistakes. I can point to small > organizations that have taken very different tacks and don't fit the model. > Some start with training. Others lean on SCR tools or security toolkits > before ever institutionalizing pen-testing. > > Perhaps it's inaccurate. Maybe it doesn't meet our industry need for > addressing "one man shop". So is it good-for-nothing? No. It's useful. > > [Value] > An immediate value that jumped out at me was the "Climb the Wall" element > [BS1]. I can again pull from experience organizations that don't fit the > model but the single most salient 'take home' from 'Organic', to me, is: > > You're going to need a champion, and until you "climb > the wall" you may hit barriers to progress. > > This concept seems to resonate with almost every security professional, > even if the form in which it comes can be contentious. > > Yet, often as vendors, internal champions, or <whatever> we don't always > make this requirement for progress explicit. We often underestimate its > cost, LoE, and impact. As a result things stall or slide backwards. > > Another key model elements that resonate with me is "Have a security > champion": > > > It's also worth pointing out that while the whole process grows > organically, at some stage it still requires a champion within the > organization to get the organization to move to the next step. That champion > usually has application security in their job description, but not always. > > BSIMM respondents handle this in consistent fashion [BS2] but Rohit's way > of explaining this is perhaps less foreboding to those getting their start > and anointing their first victim <cough> champion. > > There's also tremendous value (to the 'Organic' model) in admitting what I > think we all know implicitly: programs may have to "show need" (through > assessment) in order to progress. It's not shocking to see penetration > testing at the beginning of 'Organic'--" 'sploits always create splash" as I > say. > > [Conclusion] > To me, the 'Organic' model suffers from key inaccuracies due to omission. > As such, it doesn't particularly address the principal criticism of existing > models. Its value stems from simplicity and an potentially clear way to > drive its users through key tenets (my summary, not Rohit's): > > A) Anoint a champion > B) Show need > C) Educate execs > D) Drive assessment earlier in lifecycle > E) Bake assessment into BAU (QA) > > The ability to say these things succinctly to an organization starting its > Application Security journey provides value. I do not, however, believe that > we're going to see security assessment applied by business unit/product team > QA folk in a BAU scenario in the next 3-5 years, with the notable exception > of organizations like Microsoft. > > If 'Organic' was mine, I would attempt to amplify its positives by > converting it from a SDL Model to a method accompanied by case studies. I'd > drive it towards showing, backed by case study, how "climbing the wall" can > be accomplished most effectively. This is a tough nut and one worth > cracking. > > -jOHN > ---- > John Steven > Senior Director; Advanced Technology Consulting > Desk: 703.404.9293 x1204 Cell: 703.727.4034 > Key fingerprint = 4772 F7F3 1019 4668 62AD 94B0 AE7F EEF4 62D5 F908 > > twitter: @m1splacedsoul > Rss: http://feeds.feedburner.com/M1splacedOnTheWeb > web: http://www.cigital.com > Software Confidence. Achieved. > > > * [BW1] > http://www.businessweek.com/magazine/content/09_02/b4115059823953_page_2.htm > * [JB1] http://swreflections.blogspot.com/2009/04/opensamm-shows-way.html > * [JB2] > http://swreflections.blogspot.com/2011/06/safer-software-through-secure.html > * [PC1] https://www.pcisecuritystandards.org/security_standards/ > * [FM1] http://shiro.apache.org/ > * [BS1] Analogous to BSIMM SM1.3 "Educate Executives": > http://bsimm.com/online/governance/sm/?s=sm1.3#sm1.3 > * [BS2] http://www.informit.com/articles/article.aspx?p=1434903 > > On Jul 19, 2011, at 11:24 AM, Paco Hope wrote: > > > Jim, > > > > You're spot on. BSIMM is not a lifecycle for any company. Heck, it's not > even a set of recommendations. It's simply a way to measure what a firm > does. It's a model formulated from observations about how some firms' > implement software security in their lifecycles. You'll never catch us > calling the BSIMM a lifecycle. > > > > As for not translating into the SMB market, I don't understand that. > Unlike, say prescriptive standards which say "thou shalt do X" regardless of > how big you are, the BSIMM measures maturity of what a firm actually does. > There is no reason an SMB could not measure the maturity of their effort > using the BSIMM. > > > > Maturity is not a function of size. A team of 10 developers might score > higher on various criteria than a multi-national bank that has a whole team > of people dedicated to app sec. Maturity is a function of the depth to which > one takes a certain activity and their capability within that activity. > > > > This isn't Pac-Man, either. The goal is not to get the highest score and > an extra man. :) The goal is to put the right level of effort into the right > places. A firm can't do that until they know how much effort they're > spending on different activities. The BSIMM will illuminate the level of > effort. It allows a firm to decide to rebalance and spread the budget/people > around across the activities that make sense. Whether that's a team of 10 > developers or a team of 1000 developers, the principle is the same. The > execution varies. > > > > Here's another analogy. You can have a GPS and know your exact > coordinates, to within 3 meters, but not know how to get to the airport by > car. The BSIMM will tell you your coordinates at the present time. It does > not tell you the best way to the airport. It can tell you the crow-fly > distance to the airport, but it can't tell you that the airport is where you > want to be. > > > > Paco > > > > > > Paco, > > > > By your same logic I would not consider BSIMM a lifecycle either. It's > > a thermometer to measure an SDLC against what some some of the largest > > companies are doing. As others have noted, BSIMM does not translate > > well into the SMB market where most software is written. Don't get me > > wrong, BSIMM is very interesting data and is useful. But a > > comprehensive secure software lifecycle for every company it is not. > > > > - Jim Manico > -- Rohit Sethi SD Elements http://www.sdelements.com twitter: rksethi
_______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates _______________________________________________
