Apologies for my subsequent cluelessness/over simplification. If I understand the thread correctly, the ASF has not been hitting the bar on every security issue, and Jonathan is going to move to a low-detail-zero day process followed 90 days later by the full details (though presumably that is due to projects in general rather than just the ASF's influence). Sounds like it's going to be very painful for all involved, but I understand the frustration leading to it.
Getting 200 semi-independent groups to follow the same process will always be challenging; but failing to do so also has repercussions. It feels like the only solution is more investment in managing each security incident (with huge kudos to the folk who do this today), possibly paid. I know we have the process for a security incident defined from a project's point of view. If we assume that someone central is managing each item, can we define the process from the central-management point of view to help scale the current folk more by making it a more clearly defined role? Hen On Fri, Feb 4, 2022 at 2:26 PM Mark J Cox <[email protected]> wrote: > On Fri, Feb 4, 2022 at 7:49 PM Jonathan Leitschuh < > [email protected]> wrote: > > > To the Apache Software Foundation Security Discussion Group, > > > > Hello, my name is Jonathan Leitschuh. I’m an OSS Security Researcher and > > recipient of the first-ever Dan Kaminsky Fellowship > > <https://www.humansecurity.com/blog/our-first-dan-kaminsky-fellow>. I’m > > reaching out today to discuss some issues I’ve had in the past with the > ASF > > regarding vulnerability disclosure and coordinating that disclosure. > > > > First off, I want to communicate that I clearly understand that in OSS, > > nobody is owed anything, including when engaging with a volunteer run > > foundation like the ASF. All software is provided as-is and OSS > maintainers > > are under no obligation to fix any bug or vulnerability identified by the > > wilder OSS community. > > > > In that same light, for a security researcher, the easiest and most > > straightforward way to get a vulnerability resolved is to publicly > disclose > > the vulnerability via full-disclosure. However, doing so allows attackers > > time to leverage these vulnerabilities to exploit end-users. In general, > I > > prefer to avoid this option if possible and would rather coordinate > > disclosure with the maintainers. > > > > Mark Thomas recently communicated to me: > > > > > As a volunteer org providing stewardship for 350+ projects and 200+ > > million lines of code we do not have the resources to implement a bespoke > > disclosure process for each reporter. > > > > On my side, I’m looking for vulnerabilities that are endemic to the OSS > > ecosystem. As a part of the Dan Kaminsky Fellowship, my mission for the > > next year is to work to eliminate entire classes of security > > vulnerabilities from OSS leveraging CodeQL and bulk PR generation. > > > > Since I’m trying to work on these endemic problems, I’m in an inverted > > situation at a similar scale. I’m looking at hundreds or thousands of > > vulnerabilities with a smattering of disclosure channels per > organization, > > between email, TalkTalk, Google Groups, Hacker1, BugCrowd, Google’s > > Disclosure Process via Monorail, ect... keeping track of all these > > disclosure channels and the various disclosure states is overwhelming. > As a > > result of my security research, I’m usually looking at hundreds or > > thousands of impacted OSS projects and I have to be selective about the > > projects I disclose to. > > > > As a result of my experience, not only with the ASF, but other > > organizations and OSS projects, I’ve decided to change my disclosure > > process to centrally locate my vulnerability disclosure. It is my > intention > > to establish automation around disclosure of vulnerabilities to > > consistently enforce my 90-day disclosure timeline. These new policies > can > > be found here: > > > > https://github.com/JLLeitschuh/security-research/blob/main/README.md > > > > These new policies I’ve established have already caused some conflict. I > > hope that this email helps explain that my new policies and practices are > > coming from a place of frustration with the current ASF process, not that > > I’m trying to intentionally upset anyone. I don’t hold any grudges > against > > anyone specifically at the ASF, but I have struggled enough with the ASF > > process that I’ve thrown up my hands and decided to forge my own > disclosure > > path. > > > > I’ve been told that the ASF has a “if it didn’t happen in email, it > didn’t > > happen” culture, which I understand and respect. I offer the ASF the > > following suggestion: set up a dummy GitHub account that gets > notifications > > sent to [email protected]. That way, The ASF can get a nice searchable > > papertrail which will work with your existing infrastructure. > > > > Here are the core set of issues I’ve experienced surrounding disclosure > > regarding the ASF: > > > > > > - > > > > Emails being sinkholed without any response for months > > - > > > > Lack of collaboration/communication with the original reporter > > - > > > > Not verifying fixes are sufficient with the original reporter > > - > > > > Lack of collaboration with the original reporter with respect to the > > vulnerability disclosure contents & CVE body > > - > > > > Disclosures lack important information like the signatures of methods > > that are impacted or REST endpoints that are vulnerable. > > - > > > > Disclosures lack links to the commits that actually resolve the issue. > > - > > > > Vulnerabilities not being handled within the previously communicated > > disclosure timeline. > > - > > > > The ASF security team siding with the ASF project maintainers over the > > researcher regarding vulnerability presence/impact when a > vulnerability > > is > > clearly present. > > > > > > > > The ASF has an established Vulnerability Handling Process: > > https://www.apache.org/security/committers.html > > > > However, looking through my email today I very quickly found examples of > > cases where the ASF had not followed this process. In particular: > > > > - > > > > CVE-2020-35451: Apache Oozie - Skipped steps 12 & 13 > > - > > > > CVE-2021-36151: Apache Goblin - Skipped steps 5, 6, 7, 12, & 13 > > > > In this case, the vulnerability wasn’t ever even confirmed by anyone. > > > > I did receive an apology from the ASF security team regarding these > issues, > > which I greatly appreciate, however they are good examples of the > problems > > I’ve experienced with the ASF. > > > > Net result from my previous interaction with the ASF: I feel disrespected > > and ignored and feel like my contributions aren't valued. This has led me > > to a feeling of resentment towards the ASF and an overall feeling of > > distrust towards the established ASF vulnerability handling process. > > > > The point of this email is to explain my perspective and feelings of > > frustration I’ve experienced with the ASF. Hopefully through this > dialogue > > a constructive outcome can be achieved. > > > > Respectfully, > > > > Jonathan Leitschuh > > > > Dan Kaminsky Fellowship > > > > Hi Jonathan; > > We certainly have appreciated your reports over the years which have led to > a number of security updates across our projects. One of the reasons we > set up this public list was so that we had a place we could work on process > and policy concerns in a public forum not just limited to the private > security teams. > > You called out a few examples of concerns in issues you’ve submitted; one > of which was prior to a change to our process last year specifically so > that issues sent to [email protected] get a confirmation now from the > security team rather than from the project team, and so ensuring that the > security team can be the point of contact if there is any break-down in > communication. We do deal with over 400 reports a year from researchers > and your experience isn’t what we strive to accomplish. We’ve already > passed on your comments to the projects mentioned. In general, where you > have concerns with the handling of any issue, the best way to raise the > concern is with a follow-up message on the email thread for that report, or > (after an issue is public) on the appropriate dev list so it reaches > everyone involved in the project development (Our projects don't all > monitor this list). > > An issue that affects a wide range of projects is always tricky to > co-ordinate, where we have to bring many different project teams all with > their own management, release cycles, and communities into the embargoed > discussion, that’s pretty hard to automate and requires a lot of manual > effort, so for these types of multiple-affected-project issues we can > certainly look at ways that will mutually work for both researchers and the > ASF projects without creating exceptions or a separate process for every > case. > > Regards, Mark >
