Hi all, Taking a look at this now. (Meant to yesterday but the weather and Duke Energy had different plans for me.)
I'm wondering if some of these stats are going to have the opposite effect than intended. What's the story we want to tell and what do we want to have come out of this meeting / what outcomes are we trying to prevent? We're emphasizing our independence and size with these stats which may be seen as impressive in many contexts but might be seen differently by folks who (and this is the assumption) aren't familiar with open source development or the ASF. 227m lines of code and 630K contributors may be seen as a mountain of problems rather than an impressive achievement. I'd also reverse the order - let's start with security handling + so forth and close with "About the ASF" for folks who are interested. (I snipped a sentence about "without the end user being aware" because it may be seen as alarming, but also it contradicts the section later that notes that distributors are required to notify end users due to section 4a of the Apache License.) On Mon, Jan 3, 2022 at 5:05 AM Mark J Cox <[email protected]> wrote: > > On Mon, Jan 3, 2022 at 2:58 AM David Nalley <[email protected]> wrote: >> >> On Sun, Jan 2, 2022 at 2:59 PM Sam Ruby <[email protected]> wrote: >> >> > On Sun, Jan 2, 2022 at 1:36 PM Mark J Cox <[email protected]> wrote: >> > > >> > > I'm really happy to see some great discussion on our response and the >> > work >> > > on the wiki so far. I felt there were some repeated parts >> > >> > There is an advantage at times to repetition, but at the moment, I >> > will suggest that an executive summary be added to the beginning. >> > >> > Shoot for three bullets (one sentence or so each), with a max of 5. >> > And try to make it so that if they only read the summary, they have >> > been exposed to the important points. >> > >> > The danger is that the decision maker hands this to somebody to >> > summarize. We want to be ones to provide the summary. >> > >> > I think we have the raw data. I'd suggest that you are in the best >> > position to pick out the 3-5 points to highlight. >> > >> > >> Agreed - I like the repetition. Sam and Mark this is great! >> >> I've done a little marketing copyediting to celebrate our status, and >> provide some awe-inspiring numbers, so that folks realize the importance of >> talking with us. I know that's counterintuitive to most things at the ASF, >> but the ASF isn't our audience. >> >> Joe: I've specifically copied you here to ask that you copyedit this too. > > > I've added a first attempt of an executive summary. > > Mark > > >> >> >> > > and it needed >> > > help with reordering, so I've created a copy as a sub-page with those >> > major >> > > edits and additional information about the security team role. This is, >> > I >> > > believe, just a submission we're making in advance of a pre-meeting so it >> > > doesn't need to be totally word-perfect, and I think it's still a level >> > too >> > > detailed (I left a lot of the existing text in); perhaps it needs more of >> > > the "why is the ASF important, influential, and relevant" kind of >> > elevator >> > > pitch. >> > >> > I added back in the link to www.apache.org. I think that should >> > suffice. It does a better job than we can do, and doesn't distract >> > from the content. >> > >> > > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530004 >> > > >> > > Regards, Mark >> > >> > - Sam Ruby >> > >> > > On Sat, Jan 1, 2022 at 8:25 PM Sam Ruby <[email protected]> wrote: >> > > >> > > > First read: this is awesome >> > > > >> > > > Second read: nope, not particularly controversial, and even the areas >> > > > that are potentially controversial are adequately marked as such >> > > > >> > > > Third read: this concept may not match the intended audience. I had >> > > > to look up SDLC, but I'm certainly familiar with the concept of a >> > > > Software Development Lifecycle, but the person we send this to may not >> > > > be - their area of intelligence may be elsewhere. Actually, strike >> > > > that - is undoubtedly elsewhere. >> > > > >> > > > So, whether this content is a gold mine or gobbledygook is in the eye >> > > > of the reader. And, all things considered, I'm entirely OK with that. >> > > > As long as we don't overwhelm them, we can try multiple approaches. >> > > > >> > > > I took a stab at concatenating the input to date, and the results are >> > > > here: >> > > > >> > https://cwiki.apache.org/confluence/display/COMDEV/White+House+Software+Security+Meeting%2C+January+13%2C+2022 >> > > > >> > > > I adjusted the formatting, fixes a few typos, and reworded a few first >> > > > person references. >> > > > >> > > > I'm actually very pleased with the results. In theory, the ASF links >> > > > at the top are all that one would need. In practice, I expect that >> > > > they will be skimmed and the reader will then proceed to the bullet >> > > > points and background reading which tends to be more focused on the >> > > > problem at hand. With that background, the links are re-introduced, >> > > > this time with relevant content surfaced to guide further reading. >> > > > >> > >> >> I don't think our intended audience is going to read our footnotes (the >> links). If we get them to read a solid page of text before our meeting, I'd >> consider that a huge accomplishment. >> >> >> > > > In any case, the content is now on a wiki - hopefully that will enable >> > > > greater participation. My only request is that anything beyond >> > > > wordsmithing and formatting be accompanied by a post to this mailing >> > > > list so that people tracking this content can follow along. >> > > > >> > > > - Sam Ruby >> > > > >> > > > On Sat, Jan 1, 2022 at 1:39 PM Phil Steitz <[email protected]> wrote: >> > > > > >> > > > > >> > > > > >> > > > > On 1/1/22 9:53 AM, Sam Ruby wrote: >> > > > > > Phil, I'm pleased to see that the links you curated are almost >> > exactly >> > > > > > the ones I picked: https://s.apache.org/3vkpr. >> > > > > Yep. I was happy to notice that too :) >> > > > > > And I'll add a link >> > > > > > that we both missed: https://apache.org/ >> > > > > > >> > > > > > I'm not of the opinion that we should be creating new content for >> > > > > > background material, nor do we have adequate time to do so. But I >> > do >> > > > > > take to heart the suggestion that these links should not be buried, >> > > > > > but rather put front and center. >> > > > > > >> > > > > > What do you think about adding the following >> > > > > > >> > > > > > --- >> > > > > > >> > > > > > The best place to get started to know what the ASF is is here: >> > > > > > https://apache.org/. That page is full of statistics, videos, and >> > > > > > links. >> > > > > > >> > > > > > The following links in particular are important background to this >> > > > discussion: >> > > > > > >> > > > > > * https://www.apache.org/foundation/how-it-works.html >> > > > > > * https://www.apache.org/dev/ >> > > > > > * https://www.apache.org/dev/pmc.html >> > > > > > * https://apache.org/security/ >> > > > > > * https://apache.org/theapacheway/index.html >> > > > > > >> > > > > > --- >> > > > > > >> > > > > > What still is missing is the concept of a "supply chain". Perhaps >> > > > > > something along the lines of the following, which applies to log4j >> > but >> > > > > > also applies to much of what the ASF produces: >> > > > > > >> > > > > > * Software developed at the ASF is made available at no cost and >> > > > > > without warranty >> > > > > > * Commercial products may include this software without entering >> > into >> > > > > > any form of contract with the ASF, or even notifying us >> > > > > > * End users may purchase these products but have little interest or >> > > > > > ability to apply fixes >> > > > > >> > > > > You are probably right, Sam, that we don't have time to develop and >> > > > > agree on "new content" for this. That is why I stopped short of >> > > > > volunteering. But this AM against my better judgement, I did some >> > > > > hacking. The idea is a bunch of little tl;drs decorated with links. >> > > > > The key supply chain concepts are in the how development works and >> > how >> > > > > the software is incorporated bits. I think there are really three >> > main >> > > > > suppy-chain related ideas that people need to get: >> > > > > >> > > > > 0) While I guess we may not all agree at this point that it works, >> > our >> > > > > biggest defense against vulnerabilities getting introduced is >> > eyeballs, >> > > > > which our open development process supports. Any interested human >> > (or >> > > > > bot) can be a QC inspector in our direct contribution part of the >> > supply >> > > > > chain. We have always welcomed this, especially when reports come >> > with >> > > > > patches. >> > > > > 1) There is a natural nesting that happens as software dependencies >> > > > > propagate through applications. Addressing vulnerabilities in base >> > > > > level components (e.g. log4j) has a cascading impact. Unless and >> > until >> > > > > all downstream systems have effectively automated build, test and >> > > > > deployment systems, this creates systemic risk which has nothing to >> > do >> > > > > with OSS per se. In his original post, markt called this out, along >> > with >> > > > > 2) End of life is an industry standard concept that we apply at the >> > > > > ASF. End of life means no more patches. End of life software >> > running >> > > > > in critical systems is another systemic risk. >> > > > > >> > > > > OK, here is what I hacked in the wee hours last night. Could be >> > best to >> > > > > just use as notes or ignore if others don't agree with the content. >> > > > > -------------------------- >> > > > > >> > > > > * Foundation organization and governance structure >> > > > > (https://www.apache.org/foundation/governance/) >> > > > > The ASF is an operating US 501(c)3 organization that supports >> > several >> > > > > hundred open source projects. The operating part of the ASF is >> > managed >> > > > > by the President, supported by the Treasurer, Vice Presidents, >> > > > > committees and third parties. The ASF board provides oversight for >> > > > > operating functions as well as ASF project communities. The Legal >> > > > > Affairs, Security and Treasurer departments report directly to the >> > > > > board, which means, for example that the Security team acts with the >> > > > > authority of the board. >> > > > > >> > > > > * How oversight works ( >> > > > https://www.apache.org/foundation/how-it-works.html) >> > > > > ASF projects are run by Project Management Committees (PMCs). PMC >> > > > > members review and cast binding votes on releases, nominate and vote >> > on >> > > > > new PMC members and collectively manage community, security, >> > trademarks >> > > > > or other problems. The PMCs act on behalf of the ASF when they cut >> > > > > releases. The PMC plays a key role in technical project oversight, >> > but >> > > > > not the only role. The full project community contributes by working >> > > > > (creating and reviewing) patches and contributing to discussion. >> > > > > >> > > > > All ASF community members, including PMC members, officers and the >> > > > > board, participate in our communities as individuals, not as >> > > > > representatives of companies or other organizations. Some community >> > > > > members are paid by their employers to contribute to ASF projects >> > but in >> > > > > their ASF work they are expected to act in the best interest of the >> > > > > community, exercising their own best judgement. Impact on project >> > > > > decisions is based on publicly earned merit which accrues to >> > > > > individuals, based only on their contributions at the ASF. >> > > > > >> > > > > * How development works (https://www.apache.org/dev/pmc.html) >> > > > > ASF projects don’t all work the same and the ASF does not prescribe >> > any >> > > > > kind of SDLC for our projects. What we do require is that everything >> > > > > happens in the open: >> > > > > All bug reports are visible to all interested parties (with >> > > > > exception of reported security vulnerabilities, which are kept >> > private >> > > > > until patched) >> > > > > All commits to project source code are tracked and visible to >> > all >> > > > > interested parties >> > > > > All project-related discussion that can be public is public >> > > > > ASF projects differ in how they do things, but most have documented >> > > > > process for working bug reports, introducing new features and >> > planning >> > > > > and executing releases. As new features are developed and bugs are >> > > > > addressed, code changes are made that will go into releases. >> > Typically, >> > > > > proposed changes are submitted as “pull requests” (PRs) that >> > explicitly >> > > > > describe how the source will change if they are applied. The >> > original >> > > > > source, the PR and any changes that result from applying it are all >> > > > > publicly visible. Any interested party can comment on proposed >> > changes >> > > > > and it frequently happens that performance, security or other issues >> > are >> > > > > discovered in the open development process that creates and modifies >> > ASF >> > > > > software. >> > > > > >> > > > > * How releases are versioned, validated and distributed >> > > > > (https://www.apache.org/dev/#releases) >> > > > > ASF software is distributed and meant to be consumed in the form of >> > > > > versioned releases. ASF PMCs have various ways of packaging software >> > > > > for distribution, but the core asset being released is always source >> > > > > code. Releases must be voted on by PMCs, who are responsible for >> > > > > validating release candidates (RCs). Most PMCs also make RCs >> > available >> > > > > to the public prior to or during release votes. Library or >> > > > > infrastructure projects sometimes make backward-incompatible changes >> > as >> > > > > they develop new versions of their software. Sometimes, a >> > > > > backward-compatible version of the software is added and maintained >> > in >> > > > > parallel, but it often happens that the actively supported version is >> > > > > not backward compatible with an older version. When library or >> > > > > infrastructure components make backward incompatible changes, >> > downstream >> > > > > users may have to make code changes to upgrade their applications. >> > > > > >> > > > > * How ASF code is incorporated in other software >> > > > > ASF software is used in running systems in basically 3 ways: >> > > > > 1. A distributed system includes a running ASF product (e.g. >> > Apache >> > > > > web server) >> > > > > 2. ASF software is integrated with other software to build >> > > > applications >> > > > > When applications created in 2. are themselves used to create other >> > > > > applications, nesting happens, causing the impact of a vulnerability >> > or >> > > > > incompatible change at the lowest level to have a cascading impact. >> > > > > When a new release is made available by an ASF project, downstream >> > users >> > > > > need to minimally make configuration changes and test their >> > systems. In >> > > > > case 2. above they typically have to rebuild and redeploy their >> > > > > applications. In nested situations, this often has to happen >> > > > sequentially. >> > > > > >> > > > > * How security reports are handled (https://www.apache.org/security/ >> > ) >> > > > > The ASF encourages responsible disclosure of security vulnerabilities >> > > > > discovered in software managed by ASF projects. The ASF security >> > team >> > > > > sets policy, maintains security contacts for PMCs and provides >> > support >> > > > > for projects responding to security issues. Reporters are encouraged >> > to >> > > > > use the designated security contacts to report vulnerabilities >> > > > > privately. PMCs are required to respond to security reports >> > promptly, >> > > > > working with reporters to investigate and if necessary develop >> > > > > patches. The ASF Security team may assist the PMC in publishing a >> > CVE >> > > > > once the vulnerability has been patched and a release containing the >> > > > > patch has been made available. >> > > > > >> > > > > * How end of support works >> > > > > Like all software products, ASF code lines go through a lifecycle the >> > > > > terminates in end of support. Typically, ASF PMCs communicate end of >> > > > > support dates for products or code lines with ample time (one year >> > plus) >> > > > > for users to plan upgrades or find alternatives. Once communicated >> > end >> > > > > of life dates arrive, the PMC is no longer expected or required to >> > > > > accept or apply patches to the end of life versions. This includes >> > > > > security patches against identified vulnerabilities. Therefore in >> > some >> > > > > cases, when users who are depending on no longer supported versions >> > of >> > > > > ASF software, a version upgrade will be required to obtain a fix of >> > an >> > > > > identified vulnerability and that may require code changes to >> > integrate. >> > > > > >> > > > > Phil >> > > > > >> > > > > > >> > > > > > - Sam Ruby >> > > > > > >> > > > > > On Fri, Dec 31, 2021 at 3:10 PM Phil Steitz <[email protected]> >> > wrote: >> > > > > >> Thanks, Sam. I think the bullets below are good for the meat, but >> > > > based >> > > > > >> on David's summary of the CISA conversation and various public >> > posts, >> > > > I >> > > > > >> think there is some important background / educational material >> > that >> > > > we >> > > > > >> need to communicate. The points that Mark makes about the lack of >> > > > > >> "upgrade agility" across the supply chain are not obvious to >> > people >> > > > who >> > > > > >> don't do this every day, which is pretty much 100% of the audience >> > > > > >> here. If what we are collectively trying to solve is a supply >> > chain >> > > > > >> issue, we really need a better level set on how exactly the supply >> > > > chain >> > > > > >> works. We have a fairly nicely encapsulated piece of it at the >> > ASF >> > > > and >> > > > > >> explaining how that piece works is good background and good >> > context >> > > > for >> > > > > >> how "help" can meaningfully applied in our communities. It also >> > sets >> > > > us >> > > > > >> up to ensure that the consequences of any external actions, regs, >> > etc. >> > > > > >> on our part of the supply chain are understood. >> > > > > >> >> > > > > >> So I would suggest that the meat below follows a brief >> > backgrounder on >> > > > > >> ASF structure, governance, release and security processes and >> > > > policies, >> > > > > >> emphasizing the points where we connect to different software >> > supply >> > > > > >> chains and how we accept and manage security issues. Probably a >> > lot >> > > > of >> > > > > >> that can come from existing web pages, e.g [0-3]. It is probably >> > > > also a >> > > > > >> good idea to ask Sally to help review any docs we provide that >> > are not >> > > > > >> just links to the web site. >> > > > > >> >> > > > > >> The key thing to keep reminding ourselves is that a) the WH and >> > other >> > > > US >> > > > > >> gov ppl want to act and b) their understanding of root causes and >> > > > > >> practical solutions likely contains large gaps that can only be >> > > > > >> addressed by getting a fuller understanding of how OSS is made, >> > > > > >> distributed, consumed and maintained. >> > > > > >> >> > > > > >> Phil >> > > > > >> >> > > > > >> [0] https://www.apache.org/security/ >> > > > > >> [1] https://www.apache.org/foundation/how-it-works.html >> > > > > >> [2] https://www.apache.org/theapacheway/index.html >> > > > > >> [3] https://www.apache.org/dev/ >> > > > > >> >> > > > > >> On 12/31/21 12:06 PM, Sam Ruby wrote: >> > > > > >>> In addition to the in-person meeting next month, we are invited >> > to >> > > > > >>> send read-ahead material and there will be a brief phone call. >> > Given >> > > > > >>> that the call will only last 30 minutes, I presume that call >> > will be >> > > > > >>> consumed by introductions, logistics, and perhaps questions >> > about the >> > > > > >>> read-ahead material. That call will be on Wednesday, so I would >> > like >> > > > > >>> to send any read-ahead material that we might have late morning >> > EST >> > > > on >> > > > > >>> Monday. >> > > > > >>> >> > > > > >>> My thoughts are to lead with a summary/bulletized version of >> > what has >> > > > > >>> been discussed recently on this list, followed by actual >> > pointers to >> > > > > >>> the original emails. Here's a first draft... >> > additions/corrections >> > > > > >>> welcome, both to the bullet points and new posts (preferably to >> > this >> > > > > >>> list) that should be added. >> > > > > >>> >> > > > > >>> Key bullet points >> > > > > >>> >> > > > > >>> * This will require collective action >> > > > > >>> o There are things we can do, both individually and >> > together, >> > > > to >> > > > > >>> reduce the number of vulnerabilities. >> > > > > >>> o There are things, such as SBOMs, that can help identify >> > what >> > > > is >> > > > > >>> affected once a vulnerability is found. >> > > > > >>> o Much of this is moot if patches are never applied. >> > > > > >>> * Volunteers/community/participation >> > > > > >>> o Out contributors tend to be seasoned software >> > professionals >> > > > > >>> whose employers include ASF releases in their commercial >> > > > products. >> > > > > >>> o Our communities are healthy, open, and transparent. >> > > > > >>> o Companies an government agencies that want to help don't >> > need >> > > > > >>> money or formal contracts to do so. Join our mailing >> > lists, >> > > > > >>> review our code, contribute fixes. >> > > > > >>> >> > > > > >>> Background reading: >> > > > > >>> >> > > > > >>> * EO - Mark Cox - https://s.apache.org/3nctr >> > > > > >>> * SBOM - David Nalley - https://s.apache.org/hccur >> > > > > >>> * Applying updates - Mark Thomas - https://s.apache.org/5jqab >> > > > > >>> * Collective action - Phil Steitz - https://s.apache.org/ljzn0 >> > > > > >>> * Volunteers - Sam Ruby - https://s.apache.org/3vkpr >> > > > > >>> * Contributors/maintenance - Dominik Psenner - >> > > > > >>> https://s.apache.org/3lrk1 >> > > > > >>> * CISA - David Nalley - https://s.apache.org/1gr1c >> > > > > >>> * Get Involved - >> > > > https://www.apache.org/foundation/getinvolved.html >> > > > > >>> >> > > > > >>> - Sam Ruby >> > > > > >>> >> > > > > >> >> > > > > >> >> > --------------------------------------------------------------------- >> > > > > >> To unsubscribe, e-mail: >> > > > [email protected] >> > > > > >> For additional commands, e-mail: >> > > > [email protected] >> > > > > >> >> > > > > > >> > --------------------------------------------------------------------- >> > > > > > To unsubscribe, e-mail: >> > > > [email protected] >> > > > > > For additional commands, e-mail: >> > > > [email protected] >> > > > > > >> > > > > >> > > > > >> > > > > --------------------------------------------------------------------- >> > > > > To unsubscribe, e-mail: >> > > > [email protected] >> > > > > For additional commands, e-mail: >> > > > [email protected] >> > > > > >> > > > >> > > > --------------------------------------------------------------------- >> > > > To unsubscribe, e-mail: >> > [email protected] >> > > > For additional commands, e-mail: >> > > > [email protected] >> > > > >> > > > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: >> > [email protected] >> > >> > -- Joe Brockmeier Vice President Marketing & Publicity [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
