On Tue, May 19, 2020 at 5:53 AM Matthias van de Meent < matthias.vandeme...@cofano.nl> wrote:
> Hi Ryan, > > On Tue, 19 May 2020 at 00:47, Ryan Sleevi <r...@sleevi.com> wrote: > > > > Hi Matthias, > > > > We're aware of this. Could you explain what issue or issues this > > presents to you? > > One of the reasons I did this research was to check the track record > of CAs with regards to compliance and solving compliance issues. As > you might expect, this is quite difficult when the issues are not > updated regularly. I’m not sure I see the connection. For this scenario, it’s actually necessary to read the bugs. We do not use the bug metadata (e.g. opened, last updated, closed date) as part of our own analysis and processing, instead relying on reading the bugs themselves. The distinction closed / open is, (although skewed) a decent > indication for a CAs compliance track record and their readiness to > improve, especially when tracking open issues over time. When the > issue state is not linked to the actual solving of the compliance > issue, the skewed indication becomes even worse. Thanks. I don’t think we intend for it to be used as such, and so perhaps it’s not surprising that you find the current data unsatisfying. I would disagree that it’s a decent approximation, in part because that’s not something we guarantee, as you can see. I say this not to disagree that it “could” be useful, but it’s considerably more work and effort, and places the burden back on the Mozilla community to guarantee, for something that isn’t essential for our efforts or necessary for our tasks. For lack of a better comparison, it is a bit like asking why AIX or BeOS aren’t a tier 1 configuration ( https://firefox-source-docs.mozilla.org/build/buildsystem/supported-configurations.html ). The answer is because it’s significant burden for limited return. The MRSP section 2.4 asks the CA to promptly provide an incident > report, and regularly update this report until it is closed. My > opinion is that in this section Mozilla also has an implicit duty to > the CA - to mark issues as resolved when Mozilla and the CA agree that > the compliance issue has been resolved. Thanks. I don’t agree with that implicit duty assertion, and I think the only reason it comes up here is because it’s tied to a use case which is not presently supported. To use the previous metaphor, this is a bit like saying that if AIX/BeOS, then Mozilla has an implicit duty not to break the build. However, that ignores the notion of support tiers, and what guarantees, if any, are provided. Currently, I cannot see the forest for the trees due to so many issues > waiting to be closed, or having their next-update-by -windows long > passed, or just plain lack of communication about what is going on. > This makes it even more difficult to make informed decisions about > those CAs based on compliance track record. To close this out unambiguously: Using purely the bug metadata (i.e. the status, last modified date, etc) will not tell you this story. What you are trying to do is not presently supported. Would it be nice? Yes, I have no doubt. But that will require significantly more tooling and effort, on the part of Mozilla, and significant improvements by CAs. These are things that can be done, but are not zero cost, and are not on the most pressing of issues. It’s important to remember what this system replaced: threads on this mailing list. Threads which, by definition, are not “closed”, and which require reading with and examining in order to extract the timeline and structural data. The current use of Bugzilla is, for better or worse, a slight improvement on that system to help distinguish those threads and provide clearer signals to the CA with fewer messages, but it is not currently a full replacement for a structural, machine-readable timeline extraction. If you’d like to volunteer and contribute to the efforts to improve that, I’m happy to highlight areas for how additional tooling can help improve. This is certainly something we’ve discussed as a “nice to have”, but nowhere near the top of any of our priority lists. Patches to support this workflow would certainly be welcome. Hopefully that helps recalibrate expectations about what is currently supported, as it sounds like your use case is outside what’s supported. Also, hopefully it’s clearer that it’s not an unintentional decision, but an intentional one based on priorities and needs. Much like supporting an esoteric platform, there are non-trivial initial costs and startup costs, and as open source maintainers, sometimes the answer needs to be “No” ( https://blog.jessfraz.com/post/the-art-of-closing/ ). Again, if this is something that’s important to you, and you’re passionate about it, we’re happy to provide further details about how you can directly contribute to those efforts, so that at some point, in the future, it may be easier to support what you want it to be. > _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
