Re: Can you rely on an Audit?
On 30/12/08 06:45, Ben Bucksch wrote: On 28.12.2008 14:23, Ian G wrote: [1] disclosure, I work as an auditor So, Ian, what are you trying to tell us? We can't yank roots. We can't rely on audits. How are we supposed to restore and ensure proper operation of the system? Right. There are no easy solutions. I don't have them. What I would avoid however is following the old prescriptions without understanding them better. That will involve more work for all, papering over the cracks, and in the end, less security. This is what happened to the financial system... Obviously, just trusting CAs blindly and hoping for the best doesn't work. Not even an interested, security-conscious user can just walk into a CA and verify their operations, so they *have* to rely on us. In this I would differ somewhat :) In my time as an auditor, I have seen very little that needs to be secret, confidential or closed [1]. In particular, all of the verification processes are more or less available for open scrutiny already. As you yourself have shown, it was possible for you to read the CPS and the audit opinion, and work out that the reseller situation was as we found in the experiences. It's also possible for anyone to buy a cert and find out what it feels like, and today's discussion wasn't the first time this was done. The difference is between the belief that only an auditor can verify these things, and what you can really do. Consider that Mozo musters some 150 million end-users. Probably a million of those are interested parties. Probably 100k of those are technically savvy and interested in security. Of those, I suggest we can find 1000 who can read and comment on the CPS *and* review the verification processes of every CA under consideration. (I have seen this concept in other places. It works if you get enough people, enough openness in thinking, and enough value on the table. It works in open source for example.) So, where we would be heading is this: * reducing the scope of the audit to only those areas that cannot be opened up. * increasing the verification by the people over those areas that are open, either already, or easy to open. E.g., think about the current event. We now know far more about that verification than the audit ever told us. It's simple: just buy a few certs and report back. The cost of the certs is $10 - $100, and the cost of the audit is $100k--- ... It's even cheaper this way. Being able to yank roots, and relying on the auditor to verify and ensure that the actual, day-to-day operations follow the documented processes, and reading the process document to verify that it meets the requirements of our policy and our user's needs, is fundamental to the whole SSL thingy. Otherwise it's useless snake-oil, which harms users who rely on it - on *us* (Mozilla). Well, that is the old message from 1995. The message was constructed before we saw how it worked in practice. Things have changed a lot since then, especially we now know much more about open processes, and we know how the net really works. Instead, consider the reputational damage to the CA as the primary vector of harm. If a CA is criticised, this hurts their business [2]. If we had independent security researchers posting on their experiences, then this would enable a CHOICE style of approach to evaluating security [3]. (Yes, there are some who believe that all CAs must be equivalent, and all end-users are to be protected from any differences. I am pointing out that this very core principle is a root cause of all our troubles. It is unsustainable, and must break one day if security against a real attacker is to be the intent. There is no real security system that survives without end-to-end security, and the user is always the end.) Of course, these are just my opinions, and many do not agree to them! iang [1] Comodo provided some confirmation of this when they stated that much of what we were talking about they *could* open up but would only do it if all CAs were required to. E.g., it is closed for commercial reasons, not security or verification reasons. [2] This particular time it failed to reach the press in any big way (for reasons you can speculate on). [3] By CHOICE I mean the consumer magazine in some countries where the suppliers are kept at distance. This would be the exact opposite group to CABForum for example, where users are kept at distance. ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Can you rely on an Audit?
On 28/12/08 12:13, Kai Engert wrote: From my perspective, it's a CA's job to ensure competent [stuff]. OK. The auditing required for CAs is supposed to prove it. This might be a bit too strong. Let's look at that. What audits do is confirm criteria, and provide an opinion that the criteria are met, according to the opinion of the Auditor [1]. Now let's look at some of the limitations or caveats or bugs in that process: 1. It is up to you to read the criteria and decide if they are appropriate to your needs. 2. It is also up to you to decide whether the opinion letter is good enough whatever that means. 2.b. And, it is also up to you to investigate and understand the various languages and procedures that an Auditor uses, customarily, to weaken an apparently strong claim. 3. It is also up to you to check that the Audit verifies the Mozilla policy. I don't know, but my understanding was that Audits are generally done to WebTrust or ETSI criteria, and not to the policy. There would need to be a specific comment from the Auditor to say that the policy was included in scope. Has anyone here read the opinions and confirmed inclusion of the policy in the audit opinions? 4. It is also up to you to decide whether the Auditor has characteristics that speak highly of the opinion. Without getting into that debate, suffice to say, the singularity of the process is a weakness in and of itself. Where does this get us? It should be clear that Audits don't prove anything. At all. If you take an audit to prove something then *you have made a mistake* ; although we might agree that that this is a mistake that many people are comfortable for you to make, and they are unlikely to correct you in. The question is more likely placed as whether you can rely on an Audit. If you get through all the above, your answer is probably maybe. If you have a friend in the business, feel free to pose this question to them. Ask around, get some opinions. (And, if we accept the above, if you don't get through all of your above checks, how can your answer be any better than maybe ?) Perhaps more broadly, we could ask whether we as a community get any benefit from an audit at all, given the rather drastic nature of an individual maybe. I think the answer is yes, cautiously: it probably ensures that a CA is up to a known and reasonable level of practices [2]. So, at least we know where a CA is likely at, once they've passed their audit. Whereas before, we would be simply relying on advertising, which is hopeless. The good news is that we can actually cover a lot of these weaknesses by moving to more open disclosures. The Internet has given us wonderful opportunities to move more information more reliably, something that is not factored into current Audit thinking. So it may not be a huge issue that classical Audits have flaws (c.f., they are not perfect, nor prove what you want) as long as we look at where the flaws are found in practice and identify ways to use the open source approach to overcome those flaws [3]. iang [1] disclosure, I work as an auditor [2] I am ignoring costs analysis here, so it may be that the benefit described does not return on investment. [3] we sometimes call this open governance. ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: Can you rely on an Audit?
On 12/28/2008 03:23 PM, Ian G: [1] disclosure, I work as an auditor Since you are making a claim here of being an auditor - and specially in the context of WebTrust or similar criteria, can you please also disclose which formal training and titles you have? For which audit firm are you working currently and/or have you worked in the past? Thanks. -- Regards Signer: Eddy Nigg, StartCom Ltd. Jabber: start...@startcom.org Blog: https://blog.startcom.org ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: Can you rely on an Audit?
On 28/12/08 14:57, Eddy Nigg wrote: On 12/28/2008 03:23 PM, Ian G: [1] disclosure, I work as an auditor Since you are making a claim here of being an auditor - and specially in the context of WebTrust or similar criteria, OK, to answer the implied question here, the criteria are those written by David Ross, and are known sometimes as DRC. They are listed here: http://rossde.com/CA_review/ For the interested readers: DRC were developed following a review of WebTrust by David Ross, and his criteria cross-reference to WebTrust for ease of comparison. Those are my words, see the link for his own words. can you please also disclose which formal training and titles you have? BSc(hons) Computer Science, University of NSW. MBA, London Business School. For which audit firm are you working currently and/or have you worked in the past? Thanks. None. Interested readers might now relate this to point 9, 10 of the policy. http://www.mozilla.org/projects/security/certs/policy/ iang ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
