Re: Conceptual feedback on new ECC JEP
I received quite a bit of feedback that questions the value of the new provider, and the "branchless mode" in general. At the same time, nobody has told me that this mode would be useful to them. So I am removing it from the JEP. In short, I will only be doing the second bullet point from my last email below---SunEC will be modified to include modern implementations of these three curves. I want to stress that no change to the spec/description of SunEC will be made. In particular, the "EC" algorithms in SunEC will make no guarantees related to security against side-channel attacks. Because this is now an implementation-only change, and there is no new feature to document, there is no longer any need to make this change under a JEP. My plan is to continue this work under JDK-8208698[1] and withdraw the JEP. There is no API or behavior change, so there will be no CSR for this change. [1] https://bugs.openjdk.java.net/browse/JDK-8208698 On 9/25/2018 10:40 AM, Adam Petcher wrote: Thanks, everyone for your feedback on this JEP. I have incorporated this feedback (received on this mailing list and elsewhere) into the draft JEP[1]. Here is a summary of the current JEP and plan: *) A new provider (name TBD) will be developed to hold the new ECC implementation for the three curves. This provider will feature the interoperability-limiting restrictions on its API that were discussed at length on this mailing list. The new provider will be at the end of the list, so it won't be used by default. *) The operations of the new implementation will also be added to SunEC for the three curves. This means that the new implementation will be used by default, in a completely compatible way (without any restrictions on its API). Using the new implementation through SunEC will not provide the same level of security against side-channel attacks as using it through the new provider. *) We will add some tests that make sure that TLS still work when the new provider is used instead of SunEC. We may need to make some small changes to the TLS implementation in order to get these tests to pass. *) A couple of people asked me about whether we could modernize the implementation of more curves in the future. I added a section at the end of the JEP to discuss this. Of course, none of this is set in stone, and we still have some API details to work out in the CSR. I'll be doing the CSR next, and I'm happy to accept feedback at any time. [1] https://bugs.openjdk.java.net/browse/JDK-8204574 On 8/23/2018 1:50 PM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
On 9/25/2018 11:57 AM, Xuelei Fan wrote: On 9/25/2018 8:34 AM, Adam Petcher wrote: There will be nothing provider-dependent in the TLS implementation. The point of #3 is to say that we should test the TLS implementation to ensure that it will work with either "EC" provider. The only required changes to TLS code will be using PKCS8 private keys instead of BigInteger private keys. I read it as there is no need to change TLS implementation, right? The change from BigInteger private keys to PKCS8 private keys is for test only, right? What if we don't change test code as well? Can an existing application survive if it uses BigInteger private keys (okay, I this is a interop question)? I looked into the TLS code a bit, and the key stores (both PKCS12 and JKS) load the private keys in PKCS8 format and give them to the KeyFactory as a PKCS8EncodedKeySpec. From that point on, the TLS implementation uses the opaque keys. Based on this behavior, I don't expect there will be any need to change the TLS code in order for it to work with the new provider. An existing application cannot use BigInteger private keys if it wants to be branchless. If such an application attempts to use the new branchless provider, then the provider will throw an exception. If branchless behavior in the entire application is desired, then the solution is to change the application code so that private keys go directly from storage into the provider in PKCS8 format, in the same way that our TLS code behaves. If the import/export is allowed to branch, then the developer/configuration can use SunEC, which will allow branching in import/export, but will be branchless for the core EC operations (for three curves only). We may not want to have an impl to expose to side-channel attacks. Okay, let me ask the question in another way. Is it possible to have an interop-able impl without losing the quality of the new formula (side-channel attacks, etc)? How much effort will it take to make it possible (please consider even we have to update the BigInteger APIs as well)? Also, you asked a related question in another email: "Can we have the same security level impl in SunEC in some circumstances?" Yes, the implementation in SunEC will be interoperable and will have the same level of security against side-channel attacks when used in a particular way. Using this implementation in the new provider only ensures that it is used in the particular way that enables this level of security. I've thought more about modifying/subclassing BigInteger, and I don't believe it helps us. It only seems to move the problem around. For example, say getS() returns an internal BigInteger subclass that only supports a subset of the BigInteger operations. When an API client tries to use an operation that is not supported, it will get an exception or incorrect results. So we are back to having compatibility/interoperability issues, just in a different place in the code. Making a BigInteger subclass that implements the entire BigInteger API in constant time is impossible because the size of a BigInteger is unbounded.
Re: Conceptual feedback on new ECC JEP
On 9/25/2018 12:06 PM, Xuelei Fan wrote: On 9/25/2018 8:34 AM, Adam Petcher wrote: Yes, it is possible, at the expense of some assurance related to security against side-channel attacks. This interoperable implementation will be available by default in SunEC. A higher-assurance form of the same implementation will be available in the new provider. The additional effort required to put this implementation in both providers is expected to be relatively small. Can we have the same security level impl in SunEC in some circumstances? For example, when the key is not imported for the 4 named curves. Using a new provider means we force applications to choose between weak and interop, just because we cannot provide both at the same time. Thanks, Xuelei There's a hole in my desk from where I've been pounding my head on this subject. Basically, for the opaque operations - signing, verifying, deriving shared secrets - Adam can meet his branchless goal. For the non-opaque operations - importing and exporting the private key according to the interface specifications, he can't or thinks he can't. And even with the latter - with the use of PKCS12 as the standard key store format - the import of a private key from a key store (e.g. internally stored as PKCS8) should be able to be implemented branchless without too much effort. It's only when moving key material to/from his implementation where there are interoperability requirements that might involve branching (and not even then if he makes his own version of BigInteger to avoid those branches in the new() methods). Given that once the key leaves his implementation he has no control over whether it's used branchless, it's utterly disingenuous to claim that the process of exporting also must be branchless. So to clarify: all of his opaque code will be "high assurance" - and that's will be what's used about 99.999% of the time. The rest of this is just religious dogma that doesn't consider the actual use cases and costs related to moving keys to/from his implementation. Later, Mike
Re: Conceptual feedback on new ECC JEP
On 9/25/2018 8:34 AM, Adam Petcher wrote: Yes, it is possible, at the expense of some assurance related to security against side-channel attacks. This interoperable implementation will be available by default in SunEC. A higher-assurance form of the same implementation will be available in the new provider. The additional effort required to put this implementation in both providers is expected to be relatively small. Can we have the same security level impl in SunEC in some circumstances? For example, when the key is not imported for the 4 named curves. Using a new provider means we force applications to choose between weak and interop, just because we cannot provide both at the same time. Thanks, Xuelei
Re: Conceptual feedback on new ECC JEP
On 9/25/2018 8:34 AM, Adam Petcher wrote: On 9/25/2018 11:15 AM, Xuelei Fan wrote: I did not follow the discussion. But it does not sound right to me to have an application to be provider dependent (#3). There will be nothing provider-dependent in the TLS implementation. The point of #3 is to say that we should test the TLS implementation to ensure that it will work with either "EC" provider. The only required changes to TLS code will be using PKCS8 private keys instead of BigInteger private keys. I read it as there is no need to change TLS implementation, right? The change from BigInteger private keys to PKCS8 private keys is for test only, right? What if we don't change test code as well? Can an existing application survive if it uses BigInteger private keys (okay, I this is a interop question)? I was not confident that a new provider instead of updating the existing provider is a good idea. It might be a significant effort to update existing provider. However, if we don't do that, the cost to use the new provider is not minimal. As we discussed previous, lacking interop could face significant issues and result in complicated coding in practice. Thinking about SunPKCS11 and SunMSCAPI provider, and how many trouble we have had for them, and how many workaround we have patched for them. Unless it is not possible to have an interop-able implementation, I would suggest take more time to have an interop-able design and impl. Is it possible to have an interop-able impl? If it is possible, how much effort will it take? Yes, it is possible, at the expense of some assurance related to security against side-channel attacks. We may not want to have an impl to expose to side-channel attacks. Okay, let me ask the question in another way. Is it possible to have an interop-able impl without losing the quality of the new formula (side-channel attacks, etc)? How much effort will it take to make it possible (please consider even we have to update the BigInteger APIs as well)? Sorry for so much question, I did not take enough time for the new formula. So I depend on the questions to you so that I can have a better feel of the design. Thanks, Xuelei This interoperable implementation will be available by default in SunEC. A higher-assurance form of the same implementation will be available in the new provider. The additional effort required to put this implementation in both providers is expected to be relatively small.
Re: Conceptual feedback on new ECC JEP
On 9/25/2018 11:15 AM, Xuelei Fan wrote: I did not follow the discussion. But it does not sound right to me to have an application to be provider dependent (#3). There will be nothing provider-dependent in the TLS implementation. The point of #3 is to say that we should test the TLS implementation to ensure that it will work with either "EC" provider. The only required changes to TLS code will be using PKCS8 private keys instead of BigInteger private keys. I was not confident that a new provider instead of updating the existing provider is a good idea. It might be a significant effort to update existing provider. However, if we don't do that, the cost to use the new provider is not minimal. As we discussed previous, lacking interop could face significant issues and result in complicated coding in practice. Thinking about SunPKCS11 and SunMSCAPI provider, and how many trouble we have had for them, and how many workaround we have patched for them. Unless it is not possible to have an interop-able implementation, I would suggest take more time to have an interop-able design and impl. Is it possible to have an interop-able impl? If it is possible, how much effort will it take? Yes, it is possible, at the expense of some assurance related to security against side-channel attacks. This interoperable implementation will be available by default in SunEC. A higher-assurance form of the same implementation will be available in the new provider. The additional effort required to put this implementation in both providers is expected to be relatively small.
Re: Conceptual feedback on new ECC JEP
I did not follow the discussion. But it does not sound right to me to have an application to be provider dependent (#3). I was not confident that a new provider instead of updating the existing provider is a good idea. It might be a significant effort to update existing provider. However, if we don't do that, the cost to use the new provider is not minimal. As we discussed previous, lacking interop could face significant issues and result in complicated coding in practice. Thinking about SunPKCS11 and SunMSCAPI provider, and how many trouble we have had for them, and how many workaround we have patched for them. Unless it is not possible to have an interop-able implementation, I would suggest take more time to have an interop-able design and impl. Is it possible to have an interop-able impl? If it is possible, how much effort will it take? Thanks, Xuelei On 9/25/2018 7:40 AM, Adam Petcher wrote: Thanks, everyone for your feedback on this JEP. I have incorporated this feedback (received on this mailing list and elsewhere) into the draft JEP[1]. Here is a summary of the current JEP and plan: *) A new provider (name TBD) will be developed to hold the new ECC implementation for the three curves. This provider will feature the interoperability-limiting restrictions on its API that were discussed at length on this mailing list. The new provider will be at the end of the list, so it won't be used by default. *) The operations of the new implementation will also be added to SunEC for the three curves. This means that the new implementation will be used by default, in a completely compatible way (without any restrictions on its API). Using the new implementation through SunEC will not provide the same level of security against side-channel attacks as using it through the new provider. *) We will add some tests that make sure that TLS still work when the new provider is used instead of SunEC. We may need to make some small changes to the TLS implementation in order to get these tests to pass. *) A couple of people asked me about whether we could modernize the implementation of more curves in the future. I added a section at the end of the JEP to discuss this. Of course, none of this is set in stone, and we still have some API details to work out in the CSR. I'll be doing the CSR next, and I'm happy to accept feedback at any time. [1] https://bugs.openjdk.java.net/browse/JDK-8204574 On 8/23/2018 1:50 PM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
Thanks, everyone for your feedback on this JEP. I have incorporated this feedback (received on this mailing list and elsewhere) into the draft JEP[1]. Here is a summary of the current JEP and plan: *) A new provider (name TBD) will be developed to hold the new ECC implementation for the three curves. This provider will feature the interoperability-limiting restrictions on its API that were discussed at length on this mailing list. The new provider will be at the end of the list, so it won't be used by default. *) The operations of the new implementation will also be added to SunEC for the three curves. This means that the new implementation will be used by default, in a completely compatible way (without any restrictions on its API). Using the new implementation through SunEC will not provide the same level of security against side-channel attacks as using it through the new provider. *) We will add some tests that make sure that TLS still work when the new provider is used instead of SunEC. We may need to make some small changes to the TLS implementation in order to get these tests to pass. *) A couple of people asked me about whether we could modernize the implementation of more curves in the future. I added a section at the end of the JEP to discuss this. Of course, none of this is set in stone, and we still have some API details to work out in the CSR. I'll be doing the CSR next, and I'm happy to accept feedback at any time. [1] https://bugs.openjdk.java.net/browse/JDK-8204574 On 8/23/2018 1:50 PM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
On 9/19/2018 1:37 PM, Bernd Eckenfels wrote: Hello, I think I missed it, but where is the conversion on BigInteger branching on key material? Isn’t this only branching on effective constant values? Or are you concerned about Spectre-type problems? This is not for Spectre (etc.) issues, which effectively dump all memory contents, so there is nothing we can do to prevent it at this level. The proposed implementation defends against cache attacks like Flush+Reload[1], as well as timing attacks. Imagine an EC private key (i.e. a scalar) is stored in a fixed-length little-endian byte array in the key implementation class. I can perform the required EC operations directly on this byte array without branching on any part of its value. Now if I want to convert this key to an ECPrivateKeySpec using the KeyFactory, I need to convert this scalar to a BigInteger, which uses a variable-length representation. This conversion necessarily branches on the value of the private key, which we are trying to avoid. Branching on any non-secret value (including algorithm parameters and constants) is fine, but in the example above, we would be branching on the valud of the private key. [1] https://eprint.iacr.org/2013/448.pdf Besides that I totally agree on the idea of having a more secure implementation which can be activated by simply switching provider priorities. Gruss Bernd
Re: Conceptual feedback on new ECC JEP
On 9/19/2018 12:51 PM, Michael StJohns wrote: On 9/19/2018 11:45 AM, Adam Petcher wrote: My goal is for the new provider to be at least as interoperable as PKCS11 providers with non-exportable keys. Do all the PKCS11 providers that you have used allow importing private keys over JCA, or over some other API? Do you have to put the HSM in some sort of "import mode" in order for this import to be allowed? Is there any difference in the way that you can use keys that were loaded over this API vs keys that were generated on the device (or loaded securely)? Again - you seriously want to hang your hat on this?: 1) Yes. Over the JCA. (Umm.. PKCS11 provider is a JCA/JCE provider so of course this works). 2) No. In fact, the store operation of the PKCS11 keystore implementation does this just fine. 3) Depends. If you generate or load the HSM PKCS11 keys according to the JCA PKCS11 guidance (e.g. with the right set of attributes) using a direct PKCS11 driver, then it's seamless. Otherwise, the Sun PKCS11 keystore implementation mostly ignores them as it can't map them properly. I'd need to check the other two providers I'm aware of, but I think they implement the same scheme as the Sun provider. There's also a scheme for twiddling the attributes as part of the provider configuration, but that's a bit more painful. See https://docs.oracle.com/javase/7/docs/technotes/guides/security/p11guide.html and Appendix B. Thanks for the information. I'm having trouble believing that there is no situation in which a PKCS11 provider/token/HSM would reject a key that it is asked to import. But this is mostly a detour in the discussion. If the API of the new ECC implementation is, in some ways, more restrictive than PKCS11 with non-extractable keys, then that is acceptable to me.
Re: Conceptual feedback on new ECC JEP
Hello, I think I missed it, but where is the conversion on BigInteger branching on key material? Isn’t this only branching on effective constant values? Or are you concerned about Spectre-type problems? Besides that I totally agree on the idea of having a more secure implementation which can be activated by simply switching provider priorities. Gruss Bernd Gruss Bernd -- http://bernd.eckenfels.net Von: -1001833328m Auftrag von Gesendet: Mittwoch, September 19, 2018 7:16 PM An: security-dev@openjdk.java.net Betreff: Re: Conceptual feedback on new ECC JEP On 9/18/2018 4:24 PM, Michael StJohns wrote: > > Adam - > > Basically, the JCE is all about plugging in not about the > implementations. If this is truly an EC library, I should be able to > get the benefit of your library with very minimal changes - e.g. > specifying your provider in the various getInstance() calls. As it > stands, I doubt this will end up in anyone's "must use" category > because it will break existing code. Is this standard of being "truly an EC library" met by PKCS11 providers that don't export keys? > > You want folks to convince you that interoperability is a significant > problem when what we (or at least I) want is for you to convince us > that these interop breaks are warranted due to how wonderful your > approach is and that they're absolutely necessary due the secret sauce > of wonderfulness. You're not there yet. I don't agree with this logic. Different providers have different levels of interoperability, and this interoperability is a feature that requires effort to develop and maintain. I don't want to commit myself or this community to this effort unless I/we think it is worthwhile. Some of the proposals to increase interoperability for this effort (e.g. subclassing BigInteger) would require a significant amount of additional effort, and I think this requires justification. Of course, if you know of a way to have more interoperability without reducing assurance or significantly increasing effort, I would love to hear about it. For assurance, an important property is that the implementation should not branch on secrets after the developer has put it in branchless mode (by selecting the branchless provider or by using some other interface to switch modes). > > As for PKCS11 - there are exportable and non-exportable private keys > (e.g. PKCS11 with an accelerator vs an HSM for example). The > exportable ones show up as ECPrivateKeys, the non-exportable ones as > PrivateKeys (and I think with an underlying type of PKCS11Key...). So > it follows the model for exportable keys. And every PKCS11 provider > I've used at least has a way of IMPORTING ECPrivateKeys. My goal is for the new provider to be at least as interoperable as PKCS11 providers with non-exportable keys. Do all the PKCS11 providers that you have used allow importing private keys over JCA, or over some other API? Do you have to put the HSM in some sort of "import mode" in order for this import to be allowed? Is there any difference in the way that you can use keys that were loaded over this API vs keys that were generated on the device (or loaded securely)?
Re: Conceptual feedback on new ECC JEP
On 9/19/2018 11:45 AM, Adam Petcher wrote: On 9/18/2018 4:24 PM, Michael StJohns wrote: Adam - Basically, the JCE is all about plugging in not about the implementations. If this is truly an EC library, I should be able to get the benefit of your library with very minimal changes - e.g. specifying your provider in the various getInstance() calls. As it stands, I doubt this will end up in anyone's "must use" category because it will break existing code. Is this standard of being "truly an EC library" met by PKCS11 providers that don't export keys? This is what you want to hang your hat on? If you want to implement your code inside an HSM and follow the PKCS11 rules, feel free. Software libraries are not hardware libraries and the mapping between them is pretty good, but not perfect. And to be clear, it's not the provider that doesn't export the keys - its the underlying HSM and not for all keys. It's perfectly OK to generate an exportable key on an HSM, but the usual default is the roach motel model. The provider humors this reality by following the various interface contracts and only providing a PrivateKey object for non-exportable private keys, regardless of type. You want folks to convince you that interoperability is a significant problem when what we (or at least I) want is for you to convince us that these interop breaks are warranted due to how wonderful your approach is and that they're absolutely necessary due the secret sauce of wonderfulness. You're not there yet. I don't agree with this logic. Color me surprised. Different providers have different levels of interoperability, No - different providers mostly provide different suites of crypto, but build to a well-defined interface or set of interfaces. E.g. I really don't give a damn about what you do inside your own country, but you need to have the same type of passport as everyone else once you leave those boundaries. and this interoperability is a feature that requires effort to develop and maintain. I don't want to commit myself or this community to this effort unless I/we think it is worthwhile. Some of the proposals to increase interoperability for this effort (e.g. subclassing BigInteger) would require a significant amount of additional effort, and I think this requires justification. Implementing to a defined interface is somewhat different than "maintaining interoperability" - at least if you do it right. AFAICT, you're balking at providing the getS() method of the ECPrivateKey interface because you're afraid that conversion to/from BigInteger will leak something. Guess what... any time you border cross you leak something. Even moving key material from software to hardware will cause you to leak something. Of course, if you know of a way to have more interoperability without reducing assurance or significantly increasing effort, I would love to hear about it. For assurance, an important property is that the implementation should not branch on secrets after the developer has put it in branchless mode (by selecting the branchless provider or by using some other interface to switch modes). *bleah* This is not a useful discussion - you've drawn a line in the sand without actually understanding the cost/benefit of that line. And I gave you several approaches - simplest is to clone BigInteger and de-branch it for the methods you need. Actually, simplest is to use the existing BigInteger, because if the provider user is exporting the key material, they're probably going to be putting it into a whole different provider with "weaker" branchless protections. As for PKCS11 - there are exportable and non-exportable private keys (e.g. PKCS11 with an accelerator vs an HSM for example). The exportable ones show up as ECPrivateKeys, the non-exportable ones as PrivateKeys (and I think with an underlying type of PKCS11Key...). So it follows the model for exportable keys. And every PKCS11 provider I've used at least has a way of IMPORTING ECPrivateKeys. My goal is for the new provider to be at least as interoperable as PKCS11 providers with non-exportable keys. Do all the PKCS11 providers that you have used allow importing private keys over JCA, or over some other API? Do you have to put the HSM in some sort of "import mode" in order for this import to be allowed? Is there any difference in the way that you can use keys that were loaded over this API vs keys that were generated on the device (or loaded securely)? Again - you seriously want to hang your hat on this?: 1) Yes. Over the JCA. (Umm.. PKCS11 provider is a JCA/JCE provider so of course this works). 2) No. In fact, the store operation of the PKCS11 keystore implementation does this just fine. 3) Depends. If you generate or load the HSM PKCS11 keys according to the JCA PKCS11 guidance (e.g. with the right set of attributes) using a direct PKCS11 driver, then it's
Re: Conceptual feedback on new ECC JEP
On 9/18/2018 4:24 PM, Michael StJohns wrote: Adam - Basically, the JCE is all about plugging in not about the implementations. If this is truly an EC library, I should be able to get the benefit of your library with very minimal changes - e.g. specifying your provider in the various getInstance() calls. As it stands, I doubt this will end up in anyone's "must use" category because it will break existing code. Is this standard of being "truly an EC library" met by PKCS11 providers that don't export keys? You want folks to convince you that interoperability is a significant problem when what we (or at least I) want is for you to convince us that these interop breaks are warranted due to how wonderful your approach is and that they're absolutely necessary due the secret sauce of wonderfulness. You're not there yet. I don't agree with this logic. Different providers have different levels of interoperability, and this interoperability is a feature that requires effort to develop and maintain. I don't want to commit myself or this community to this effort unless I/we think it is worthwhile. Some of the proposals to increase interoperability for this effort (e.g. subclassing BigInteger) would require a significant amount of additional effort, and I think this requires justification. Of course, if you know of a way to have more interoperability without reducing assurance or significantly increasing effort, I would love to hear about it. For assurance, an important property is that the implementation should not branch on secrets after the developer has put it in branchless mode (by selecting the branchless provider or by using some other interface to switch modes). As for PKCS11 - there are exportable and non-exportable private keys (e.g. PKCS11 with an accelerator vs an HSM for example). The exportable ones show up as ECPrivateKeys, the non-exportable ones as PrivateKeys (and I think with an underlying type of PKCS11Key...). So it follows the model for exportable keys. And every PKCS11 provider I've used at least has a way of IMPORTING ECPrivateKeys. My goal is for the new provider to be at least as interoperable as PKCS11 providers with non-exportable keys. Do all the PKCS11 providers that you have used allow importing private keys over JCA, or over some other API? Do you have to put the HSM in some sort of "import mode" in order for this import to be allowed? Is there any difference in the way that you can use keys that were loaded over this API vs keys that were generated on the device (or loaded securely)?
Re: Conceptual feedback on new ECC JEP
On 9/18/2018 4:01 PM, Adam Petcher wrote: On 9/11/2018 11:07 AM, Adam Petcher wrote: I still haven't been convinced that this lack of interoperability is a significant problem. In the proposed design, the new KeyFactory will not support ECPrivateKeySpec, and the implementation will produce private keys that inherit from PrivateKey, but not ECPrivateKey. Specifically, what problems in JCE are introduced by this design? How are these interoperability issues different from the ones you encounter with a PKCS11 provider that doesn't export private keys? If the developer wants more interoperability, why not use SunEC? If we decide that we want the new implementation to have better interoperability in the future, does something prevent us from enhancing it? These questions are for anyone who can help me understand the objections that have been raised related to interoperability. A week has passed since I asked these questions to the mailing list, and I haven't gotten any answers. Without additional information, I'm not motivated to change the interoperability goals in the draft JEP. Adam - Basically, the JCE is all about plugging in not about the implementations. If this is truly an EC library, I should be able to get the benefit of your library with very minimal changes - e.g. specifying your provider in the various getInstance() calls. As it stands, I doubt this will end up in anyone's "must use" category because it will break existing code. You want folks to convince you that interoperability is a significant problem when what we (or at least I) want is for you to convince us that these interop breaks are warranted due to how wonderful your approach is and that they're absolutely necessary due the secret sauce of wonderfulness. You're not there yet. As for PKCS11 - there are exportable and non-exportable private keys (e.g. PKCS11 with an accelerator vs an HSM for example). The exportable ones show up as ECPrivateKeys, the non-exportable ones as PrivateKeys (and I think with an underlying type of PKCS11Key...). So it follows the model for exportable keys. And every PKCS11 provider I've used at least has a way of IMPORTING ECPrivateKeys. Later, Mike
Re: Conceptual feedback on new ECC JEP
On 9/11/2018 11:07 AM, Adam Petcher wrote: I still haven't been convinced that this lack of interoperability is a significant problem. In the proposed design, the new KeyFactory will not support ECPrivateKeySpec, and the implementation will produce private keys that inherit from PrivateKey, but not ECPrivateKey. Specifically, what problems in JCE are introduced by this design? How are these interoperability issues different from the ones you encounter with a PKCS11 provider that doesn't export private keys? If the developer wants more interoperability, why not use SunEC? If we decide that we want the new implementation to have better interoperability in the future, does something prevent us from enhancing it? These questions are for anyone who can help me understand the objections that have been raised related to interoperability. A week has passed since I asked these questions to the mailing list, and I haven't gotten any answers. Without additional information, I'm not motivated to change the interoperability goals in the draft JEP.
Re: Conceptual feedback on new ECC JEP
On 9/10/2018 7:49 PM, Xuelei Fan wrote: The motivation of the JEP is that some formulas may be more easier to expose attacks. It's true that the formulas impact the security level of the implementation. I was just wondering if the JEP proposed formulas have been well analyze. A standard or formal recommendation may indicate the good quality of the formulas. Thanks. I added this comment to the JEP under "Risks and Assumptions". As far as I know, the exact EC formulas that are usually implemented have never been standardized. The standards give a specification for point addition in affine coordinates, but that is not what is implemented. I admit that the novelty of the proposed formulas presents some additional risk, but the formulas in SunEC were novel when they were implemented, too. I'd like to hear more opinions about the perceived risk of using formulas from the academic literature, if anyone has them. It's worth noting that some of these academic formulas have already been implemented in other crypto/TLS libraries. It's a concern to me that interoperability is listed as "non-goals". In general, it may introduce a lot of problems in the current JCE framework. I don't know your detailed design yet, and not sure if you are able to mitigate the impact. Looks like the debate was mainly about the BigInteger. If the keys are used in the same provider, I don't think the BigInteger is a problem as you can extend a private BigInteger that you like. If the keys are use between two providers, add a thine clue to export/import BigInteger may be worthy for provider interoperability. I reworded that part of the JEP, because it was misleading. Now it more clearly indicates that interoperability with providers that only support BigInteger private keys is out of scope. I still haven't been convinced that this lack of interoperability is a significant problem. In the proposed design, the new KeyFactory will not support ECPrivateKeySpec, and the implementation will produce private keys that inherit from PrivateKey, but not ECPrivateKey. Specifically, what problems in JCE are introduced by this design? How are these interoperability issues different from the ones you encounter with a PKCS11 provider that doesn't export private keys? If the developer wants more interoperability, why not use SunEC? If we decide that we want the new implementation to have better interoperability in the future, does something prevent us from enhancing it? These questions are for anyone who can help me understand the objections that have been raised related to interoperability. I'm a little bit nervous about the two providers design. It simplify the initial crypto implementation, but left the complexity to developers and sustaining. I don't have a good sense about how to use them in JSSE. What if the proposed formulas is vulnerable in the future? I don't see any reason for a two-provider design to be more complex than a one-provider design. The two providers could even share code, but I don't think that is necessarily a good idea. Also, I'm not opposed to putting everything in one provider, with some sort of property to choose the behavior. But the provider system already exists, and it seems to work fine for this purpose, so I would rather use it. Here is a proposal for how we can make the two-provider design work in JSSE: 1) We add the new provider to the list, with lower priority than SunEC 2) We add a "Branchless" property to the provider 3) We modify SunJSSE so that it sends all EC private keys to the provider using an encoded key spec. PKCS8EncodedKeySpec would work, but perhaps we could add a new type for raw encoded private keys, if necessary. 4) We add some provider selection code to SunJSSE that is used for EC, ECDH, and ECDSA. This code will loop through the providers (in priority order), looking for the first "Branchless" provider that supports the desired curve. If it doesn't find one, it will use the first provider that supports the desired curve. I'm not sure how your question about vulnerability fits in here. If the formulas are vulnerable, we will fix them, if possible. Maybe if you restate the question, I'll have a better idea of what you are asking.
Re: Conceptual feedback on new ECC JEP
On 9/10/2018 1:23 PM, Adam Petcher wrote: The paper[1] containing the proposed formulas is referenced in the JEP. Thanks! As far as I know, they are not part of any standard. If you know of any standardized, complete EC point arithmetic formulas, then let me know. I don't know, either. I think it is okay to use these formulas as long as they produce the same result as the operations in Section 2.2 of SEC 1[2]. The motivation of the JEP is that some formulas may be more easier to expose attacks. It's true that the formulas impact the security level of the implementation. I was just wondering if the JEP proposed formulas have been well analyze. A standard or formal recommendation may indicate the good quality of the formulas. It's a concern to me that interoperability is listed as "non-goals". In general, it may introduce a lot of problems in the current JCE framework. I don't know your detailed design yet, and not sure if you are able to mitigate the impact. Looks like the debate was mainly about the BigInteger. If the keys are used in the same provider, I don't think the BigInteger is a problem as you can extend a private BigInteger that you like. If the keys are use between two providers, add a thine clue to export/import BigInteger may be worthy for provider interoperability. I'm a little bit nervous about the two providers design. It simplify the initial crypto implementation, but left the complexity to developers and sustaining. I don't have a good sense about how to use them in JSSE. What if the proposed formulas is vulnerable in the future? If you believe it is a better one, please consider to replace the current EC implementation. I know it may only support secp256r1/secp384r1/secp521r1 now, but we can use it to replace the implementation of the three curves for now. Thanks, Xuelei [1] https://eprint.iacr.org/2015/1060.pdf [2] http://www.secg.org/sec1-v2.pdf On 9/10/2018 2:23 PM, Xuelei Fan wrote: Can I have the links to the new formulas that you will be used? Are they part of any current standards? Thanks, Xuelei On 8/23/2018 10:50 AM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
The paper[1] containing the proposed formulas is referenced in the JEP. As far as I know, they are not part of any standard. If you know of any standardized, complete EC point arithmetic formulas, then let me know. I think it is okay to use these formulas as long as they produce the same result as the operations in Section 2.2 of SEC 1[2]. [1] https://eprint.iacr.org/2015/1060.pdf [2] http://www.secg.org/sec1-v2.pdf On 9/10/2018 2:23 PM, Xuelei Fan wrote: Can I have the links to the new formulas that you will be used? Are they part of any current standards? Thanks, Xuelei On 8/23/2018 10:50 AM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
Can I have the links to the new formulas that you will be used? Are they part of any current standards? Thanks, Xuelei On 8/23/2018 10:50 AM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
See inline below. It also occurred to me that we could use Provider properties here. We have the "ImplementedIn" property, and we can similarly add a standard "Branchless" property. I think my preferred solution is still to use separate providers, and perhaps use a read-only "Branchless" property to aid in provider selection. That is, JCA client code that supports the branchless API (that is, no BigInteger) could loop through the providers until it finds one with the "Branchless" property set. This same mechanism could be used to select a provider that allows branching in an environment where the highest-priority provider is branchless. We may also be able to set the "Branchless" property at runtime to influence the behavior of a single provider. But this would introduce a lot of complication, and I think it is starting to get away from the intended use of these properties. On 9/7/2018 7:07 PM, Anthony Scarpino wrote: On Sep 7, 2018, at 12:08 PM, Adam Petcher wrote: This is a good suggestion. I don't have particularly strong feelings about using separate providers vs a property in a single provider. I think the fundamental issues are the same, and this choice mostly affects API details. Do you think this should be a system property, security property, or something else? Should it be modifiable at any time? I would say a security property Perhaps it has to be in order to address Mike's desire to put the provider in "import/export mode". Would the property affect existing keys? Again, I think it would have to, so you can generate a key, turn off branchless mode, and then export it. My thought was the property would allow the branchless import/export for that java instance. Not turning it on and off during operations. I think the value of the branchless operation is in the usage of the algorithms. If an app wants to use a pure branchless provider the property can be set or never use the APIs with BigInteger in them. One could avoid the property completely by just documenting clearly the interfaces that branched and then leave it to the app developer, but from the email discussion with Mike it sounded like you preferred strong controls on the provider. The issues related to branching are too subtle, and I don't think we should expose any of it to the developer except "use this provider/property/algorithm to indicate that branching on secrets is unacceptable." Moreover, I'm intimately familiar with all of these issue, and yet I still don't have confidence in my own ability to switch SunJSSE over to using a branchless ECC implementation, unless I have strong enforcement in the provider. It would be too easy for me to simply miss an existing call to getS() or a use of ECPrivateKeySpec. If we only have a single provider, and a security property that has been set to indicate that branching is unacceptable in that provider, then I think that means that there is no way to change the value of the property and use private keys as BigInteger in that JVM process without using some other (third party) provider. Though maybe I'm misunderstanding something about your proposal. What about curves other than P256, P384, and P521? We can't do branchless operations in those curves, so any attempt to use them when this property is enabled would result in an exception. I wasn’t going to propose anything further than your existing curves, but it would be nice if branchful curves could be added to eventual replace SunEC. I know that isn’t the intent of this JEP and depends if you want to expand it. Sure, but for this JEP, we need to be concerned about how the API can be misused, and a branching implementation is obtained when a branchless one is expected. The developer shouldn't be expected to know that three of the curves have branchless implementations, and using any other curve will result in branching. So any global "don't branch" property would need to cause the use of any other curve to fail.
Re: Conceptual feedback on new ECC JEP
Hi, I have not had a chance to review this JEP yet. Personally, if possible, I would expect there is no public APIs update so that more applications can benefit from the enhancement, and SunJSSE could benefits from more crypto providers. I'm not sure if it is possible or not now, or how could we minimize the APIs update. I will see if I could be here next week. Please go ahead if you have an agreement before I look into this JEP. Thanks, Xuelei On 9/7/2018 12:08 PM, Adam Petcher wrote: This is a good suggestion. I don't have particularly strong feelings about using separate providers vs a property in a single provider. I think the fundamental issues are the same, and this choice mostly affects API details. Do you think this should be a system property, security property, or something else? Should it be modifiable at any time? Perhaps it has to be in order to address Mike's desire to put the provider in "import/export mode". Would the property affect existing keys? Again, I think it would have to, so you can generate a key, turn off branchless mode, and then export it. What about curves other than P256, P384, and P521? We can't do branchless operations in those curves, so any attempt to use them when this property is enabled would result in an exception. The questions above are for everybody, if you have thoughts on any of this, please share. My initial thoughts are that using a property may give us some additional flexibility, and may improve the interface, but the main cost is additional complexity in the implementation, since we'll need to implement some checks that would otherwise be accomplished by provider selection and having separate code. On 9/7/2018 1:53 PM, Anthony Scarpino wrote: Adam, I tend to agree with Mike that disallowing import/export of keys using BigInteger is not the value of a branchless implementation. As you point out in the JEP the provider is greatly hindered by this design choice. I feel it would be better to implementing the BigInteger parts and have a property to shut them off for a pure branchless implementation. That should allow the provider to be used in the default provider list and the ‘opt-in’ would be the property to turn off BigInteger or any other branching situation. I am concerned the desire for a purest provider will result in it being unused. Documentation can be clear about the import/export situation, the preference toward PKCS8EncodedKeySpec, and the property to lock it down. Tony On Aug 23, 2018, at 10:50 AM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
Hi, In general, System properties should be avoided, they add complexity to configurations and especially if they change sensitive behavior. In any case, the defaults should be secure-by-default; not the other way around. Perhaps two different providers, one secure and one more secure. $.02, Roger On 9/7/2018 3:08 PM, Adam Petcher wrote: This is a good suggestion. I don't have particularly strong feelings about using separate providers vs a property in a single provider. I think the fundamental issues are the same, and this choice mostly affects API details. Do you think this should be a system property, security property, or something else? Should it be modifiable at any time? Perhaps it has to be in order to address Mike's desire to put the provider in "import/export mode". Would the property affect existing keys? Again, I think it would have to, so you can generate a key, turn off branchless mode, and then export it. What about curves other than P256, P384, and P521? We can't do branchless operations in those curves, so any attempt to use them when this property is enabled would result in an exception. The questions above are for everybody, if you have thoughts on any of this, please share. My initial thoughts are that using a property may give us some additional flexibility, and may improve the interface, but the main cost is additional complexity in the implementation, since we'll need to implement some checks that would otherwise be accomplished by provider selection and having separate code. On 9/7/2018 1:53 PM, Anthony Scarpino wrote: Adam, I tend to agree with Mike that disallowing import/export of keys using BigInteger is not the value of a branchless implementation. As you point out in the JEP the provider is greatly hindered by this design choice. I feel it would be better to implementing the BigInteger parts and have a property to shut them off for a pure branchless implementation. That should allow the provider to be used in the default provider list and the ‘opt-in’ would be the property to turn off BigInteger or any other branching situation. I am concerned the desire for a purest provider will result in it being unused. Documentation can be clear about the import/export situation, the preference toward PKCS8EncodedKeySpec, and the property to lock it down. Tony On Aug 23, 2018, at 10:50 AM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
This is a good suggestion. I don't have particularly strong feelings about using separate providers vs a property in a single provider. I think the fundamental issues are the same, and this choice mostly affects API details. Do you think this should be a system property, security property, or something else? Should it be modifiable at any time? Perhaps it has to be in order to address Mike's desire to put the provider in "import/export mode". Would the property affect existing keys? Again, I think it would have to, so you can generate a key, turn off branchless mode, and then export it. What about curves other than P256, P384, and P521? We can't do branchless operations in those curves, so any attempt to use them when this property is enabled would result in an exception. The questions above are for everybody, if you have thoughts on any of this, please share. My initial thoughts are that using a property may give us some additional flexibility, and may improve the interface, but the main cost is additional complexity in the implementation, since we'll need to implement some checks that would otherwise be accomplished by provider selection and having separate code. On 9/7/2018 1:53 PM, Anthony Scarpino wrote: Adam, I tend to agree with Mike that disallowing import/export of keys using BigInteger is not the value of a branchless implementation. As you point out in the JEP the provider is greatly hindered by this design choice. I feel it would be better to implementing the BigInteger parts and have a property to shut them off for a pure branchless implementation. That should allow the provider to be used in the default provider list and the ‘opt-in’ would be the property to turn off BigInteger or any other branching situation. I am concerned the desire for a purest provider will result in it being unused. Documentation can be clear about the import/export situation, the preference toward PKCS8EncodedKeySpec, and the property to lock it down. Tony On Aug 23, 2018, at 10:50 AM, Adam Petcher wrote: I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
Re: Conceptual feedback on new ECC JEP
Adam, I tend to agree with Mike that disallowing import/export of keys using BigInteger is not the value of a branchless implementation. As you point out in the JEP the provider is greatly hindered by this design choice. I feel it would be better to implementing the BigInteger parts and have a property to shut them off for a pure branchless implementation. That should allow the provider to be used in the default provider list and the ‘opt-in’ would be the property to turn off BigInteger or any other branching situation. I am concerned the desire for a purest provider will result in it being unused. Documentation can be clear about the import/export situation, the preference toward PKCS8EncodedKeySpec, and the property to lock it down. Tony > On Aug 23, 2018, at 10:50 AM, Adam Petcher wrote: > > I'm starting work on yet another ECC JEP[1], this time with the goal of > developing improved implementations of existing algorithms, rather than > implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, > 384-, and 521-bit NIST prime curves. The new implementation will be all Java, > and will resist side-channel attacks by not branching on secrets. It will go > in a new provider which is not in the provider list in the java.security file > by default. So it will need to be manually enabled by changing the > configuration or putting the new provider name in the code. It will only > support a subset of the API that is supported by the implementation in SunEC. > In particular, it will reject any private keys with scalar values specified > using BigInteger (as in ECPrivateKeySpec), and its private keys will not > return scalar values as BigInteger (as in ECPrivateKey.getS()). > > Please take a look and send me any feedback you have. I'm especially looking > for suggestions on how this new implementation should fit into the API. I > would prefer to have it enabled by default, but I can't think of a way to do > that without either branching on secrets in some cases (converting a > BigInteger private key to an array) or breaking compatibility (throwing an > exception when it gets a BigInteger private key). > > [1] https://bugs.openjdk.java.net/browse/JDK-8204574 >
Re: Conceptual feedback on new ECC JEP
I'm only going to respond to one of your points in detail, see inline below. For the other points, I don't think we are going to agree. You want the implementation to be more interoperable, and I have stated that the level of interoperability that you want is not a goal of the JEP. This is supposed to be a more secure software ECC implementation, with corresponding restrictions on its API. I will be happy with it if the API is just functional enough to allow us to use it in SunJSSE, with some non-trivial modifications to the SunJSSE code. In the initial release of this new provider, it should only be enabled by users who are willing to accept this tradeoff between security and usability/interoperability, and who are willing to modify existing code to make the new provider work. On 9/7/2018 12:43 PM, Michael StJohns wrote: Simple way to do this is to extend BigInteger and fix these perceived problems. You may have to replace/replicate pretty much everything, but as long as it has a BigInteger signature when output, you're good to go. Actually, do that - clone BigInteger.java as SafeBigInteger.java, have extend java.math.BigInteger and change out the things that bother you. I checked - BigInteger is not a final class so this should work. Thanks for the suggestion. I agree that this might work, but it would greatly increase the scope/effort of this JEP. There might also be other solutions that require less effort (e.g. new types of keys/specs that hold values as integers, but not BigInteger) that should be considered. I've updated the JEP to record this suggestion, and to indicate that it is out of scope. We can always enhance the interface/implementation later.
Re: Conceptual feedback on new ECC JEP
On 9/6/2018 9:53 AM, Adam Petcher wrote: On 9/5/2018 5:53 PM, Michael StJohns wrote: BigInteger is not required to encode a PKCS8 key, but its trivial to translate from BigInteger to PKCS8 and vice versa. Note that internally, the current BigInteger stores the magnitude as an array of int in big endian order and stores the sign separately. The BigInteger (int sigNum, byte[] magnitude) constructor mostly just copies the magnitude bytes over (and converts 4 bytes to an integer) for positive integers. While the current BigInteger doesn't have a byte[] BigInteger.getMagnitude() call, it would be trivial to subclass BigInteger to copy the magnitude bytes (without branching) out. In any event, for NewEC to OldEc - you do use the sign/magnitude call to create the BigInteger - no leaking on the part of NewEC, and once it gets to OldEC its not your problem. For OldEc to NewEc you copy the BigInteger to NewBigInteger (trivial wrapper class) and do a getMagnitude(). See the code for BigInteger[1]. The sign/magnitude constructor calls stripLeadingZeroBytes (line 405) on the magnitude to convert it to an int array. This method compares the leading values in the byte array to zero in a short-circuit expression in a for loop (line 4298). So this constructor branches on the value that is supplied to it, and it cannot be used in a branchless implementation. I don't know how to write this branchless getMagnitude method that you are proposing. I would need to convert from a variable-length int array to a fixed-length byte array. In doing this, I can't branch on the length of the variable-length array, because doing so would leak whether the high-order bits of the key are zero. If you know how to do this, then please provide some code to illustrate how it is done. Simple way to do this is to extend BigInteger and fix these perceived problems. You may have to replace/replicate pretty much everything, but as long as it has a BigInteger signature when output, you're good to go. Actually, do that - clone BigInteger.java as SafeBigInteger.java, have extend java.math.BigInteger and change out the things that bother you. I checked - BigInteger is not a final class so this should work. E.g. as long as *I* can call the functions I need to call on the object you exported, I'm fine with it. In any event, you're still missing the point here. You're EXPORTING the key from your provider to other providers which probably already don't care that much about the one in 256 (approx - slightly larger) time where exporting the key might leak the fact of leading zero bits. [1] http://hg.openjdk.java.net/jdk/jdk/file/805807f15830/src/java.base/share/classes/java/math/BigInteger.java While the contract for getEncoded() explicitly allows for a null result, getS() of ECPrivateKey does not. This more than any other reason appears to be why the PKCS 11 provider represents the ECPrivateKey as a simple PrivateKey object. Good point. There is no need for the private keys for this provider to implement ECPrivateKey, since there is only one method that is effectively unimplemented. We should use a new implementation class (not ECPrivateKeyImpl) that implements PrivateKey, not ECPrivateKey. *pounds head on table* Sure. And then you can't use any translation features and then you break all the code. So implementing your provider requires other providers to change? Because? Do you expect BouncyCastle and NSS to change as well? I think the situation is improved by not having the new private keys implement ECPrivateKey. Then the JDK ECKeyFactory code does not need to change, and neither do other providers, assuming they behave in a similar way. Though I think it is acceptable if other providers don't behave this way, and therefore cannot translate private keys from this new provider. I've updated the JEP to indicate that interoperability with these providers is a non-goal. OK. At this point you're no longer calling this an EC key. Make sure you catch all the places where ECPrivateKey is used instead of PrivateKey. I think you're incredibly short sighted and not a lot of folks will implement this, but go for it. I think its a *really* bad idea and that you're fixing the wrong problems. The only way to get private keys in or out of the new provider is through a PKCS#8 encoding. Moving keys to/from another provider that supports ECPrivateKeySpec but not PKCS#8 encoding can be accomplished by translating the specs---possibly with the help of a KeyFactory that supports both, like the one in SunEC. *pounds head against table* Bits are bits are bits. Creating a PKCS8EncodedKeySpec gets you a byte array which you can convert to a BigInteger by decoding the PKCS8 structure and taking the PKCS8 PrivateKey octets and doing new BigInteger (1, privateKeyOctets). That doesn't require ASN1 integer representation (e.g. leading byte is zero if high bit
Re: Conceptual feedback on new ECC JEP
On 9/5/2018 5:53 PM, Michael StJohns wrote: BigInteger is not required to encode a PKCS8 key, but its trivial to translate from BigInteger to PKCS8 and vice versa. Note that internally, the current BigInteger stores the magnitude as an array of int in big endian order and stores the sign separately. The BigInteger (int sigNum, byte[] magnitude) constructor mostly just copies the magnitude bytes over (and converts 4 bytes to an integer) for positive integers. While the current BigInteger doesn't have a byte[] BigInteger.getMagnitude() call, it would be trivial to subclass BigInteger to copy the magnitude bytes (without branching) out. In any event, for NewEC to OldEc - you do use the sign/magnitude call to create the BigInteger - no leaking on the part of NewEC, and once it gets to OldEC its not your problem. For OldEc to NewEc you copy the BigInteger to NewBigInteger (trivial wrapper class) and do a getMagnitude(). See the code for BigInteger[1]. The sign/magnitude constructor calls stripLeadingZeroBytes (line 405) on the magnitude to convert it to an int array. This method compares the leading values in the byte array to zero in a short-circuit expression in a for loop (line 4298). So this constructor branches on the value that is supplied to it, and it cannot be used in a branchless implementation. I don't know how to write this branchless getMagnitude method that you are proposing. I would need to convert from a variable-length int array to a fixed-length byte array. In doing this, I can't branch on the length of the variable-length array, because doing so would leak whether the high-order bits of the key are zero. If you know how to do this, then please provide some code to illustrate how it is done. [1] http://hg.openjdk.java.net/jdk/jdk/file/805807f15830/src/java.base/share/classes/java/math/BigInteger.java While the contract for getEncoded() explicitly allows for a null result, getS() of ECPrivateKey does not. This more than any other reason appears to be why the PKCS 11 provider represents the ECPrivateKey as a simple PrivateKey object. Good point. There is no need for the private keys for this provider to implement ECPrivateKey, since there is only one method that is effectively unimplemented. We should use a new implementation class (not ECPrivateKeyImpl) that implements PrivateKey, not ECPrivateKey. So implementing your provider requires other providers to change? Because? Do you expect BouncyCastle and NSS to change as well? I think the situation is improved by not having the new private keys implement ECPrivateKey. Then the JDK ECKeyFactory code does not need to change, and neither do other providers, assuming they behave in a similar way. Though I think it is acceptable if other providers don't behave this way, and therefore cannot translate private keys from this new provider. I've updated the JEP to indicate that interoperability with these providers is a non-goal. The only way to get private keys in or out of the new provider is through a PKCS#8 encoding. Moving keys to/from another provider that supports ECPrivateKeySpec but not PKCS#8 encoding can be accomplished by translating the specs---possibly with the help of a KeyFactory that supports both, like the one in SunEC. *pounds head against table* Bits are bits are bits. Creating a PKCS8EncodedKeySpec gets you a byte array which you can convert to a BigInteger by decoding the PKCS8 structure and taking the PKCS8 PrivateKey octets and doing new BigInteger (1, privateKeyOctets). That doesn't require ASN1 integer representation (e.g. leading byte is zero if high bit is set) and doesn't cause a branch. See above for my explanation of why this solution does, in fact, branch.
Re: Conceptual feedback on new ECC JEP
On 9/4/2018 3:19 PM, Adam Petcher wrote:
I think what you are suggesting is that the implementation should
convert between BigInteger and the internal representation when
necessary. The problem with this approach is that it is too easy to
inadvertently supply a BigInteger to the implementation, and this
would result in a branch. I understand that this branch may be
acceptable in some circumstances, but we would need something in the
API to tell the implementation whether it is okay to branch or not. I
think the simplest way to do that is to have a provider that never
branches. If branching is okay, then SunEC can be used.
Basically yes.
But I don't understand what you mean by "inadvertently supply a
BigInteger"? AFAICT, you "supply" a BigInteger in an ECPrivateKeySpec
and you retrieve one when you call getEncoded() or getS(). Your
implementation would convert between the BigInteger and internal
representation during the use of the engineGeneratePrivate() method of
the KeyFactorySpi and would convert from your internal representation
when exporting S, or encoding something as PKCS8.
Again - you're wrongly conflating interface requirements with
implementation requirements.
And how do you expect to move key material between SunEC and this
implementation? See below for my commentary on that.
That's essentially the plan. Calling PrivateKey::getEncoded will
return null, which is a convention for non-extractable keys. Trying to
convert from/to an ECPrivateKeySpec using the KeyFactory in the new
provider will result in an exception---so you won't have an object to
call getS() on.
That's not what PKCS11 does - it just gives you a "PrivateKey" object
with an internal type of sun.security.pkcs11.P11Key. While that object
is not type safe exactly, it is provider safe.
You're still wanting to use the various EC classes of java.security,
java.security.spec, and java.security.interfaces, but you're unwilling
to actually meet their contracts for some really suspect reasons.
To create the key from stored information, the best way is to
construct a PKCS8EncodedKeySpec using the encoded key. If you are
starting with a BigInteger, and if branching is acceptable, you can
use the KeyFactory from SunEC to convert an ECPrivateKeySpec to
PrivateKey to get the encoded value.
Umm... what?
If you were doing NewEC -> SunEC manually (getEncoded() -> KeySpec) -
you'll need to end up emitting a PKCS8 blob using RFC5915, which -
unsurprisingly has BigEndian INTEGERs (yes, its an octet string, but
the encoding is specified by RFC3447 as pretty much the big endian
encoding of an integer). E.g. it may look opaque from Java's point of
view, but it's not really opaque. (See below)
Or have you got a different way of encoding the PKCS8 blob for the new
provider? E.g. point me at a specification please.
My head hurt when I tried to work through the various cases of
translating a private key from your provider to SunEC or to BouncyCastle
and vice versa. Basically, if you don't support the getS() call, then
KeyFactory.translateKey() will fail. (Below from
sun.security.ec.ECKeyFactory.java - the SunEC provider's implementation).
private PrivateKey implTranslatePrivateKey(PrivateKey key)
throws InvalidKeyException {
if (key instanceof ECPrivateKey) {
if (key instanceof ECPrivateKeyImpl) {
return key;
}
ECPrivateKey ecKey = (ECPrivateKey)key;
return new ECPrivateKeyImpl(
ecKey.getS(),
ecKey.getParams()
);
} else if ("PKCS#8".equals(key.getFormat())) {
return new ECPrivateKeyImpl(key.getEncoded());
} else {
throw new InvalidKeyException("Private keys must be instance "
+ "of ECPrivateKey or have PKCS#8 encoding");
}
4.1 I2OSP
I2OSP converts a nonnegative integer to an octet string of a
specified length.
I2OSP (x, xLen)
Input:
xnonnegative integer to be converted
xLen intended length of the resulting octet string
Output:
Xcorresponding octet string of length xLen
Error: "integer too large"
Steps:
1. If x >= 256^xLen, output "integer too large" and stop.
2. Write the integer x in its unique xLen-digit representation in
base 256:
x = x_(xLen-1) 256^(xLen-1) + x_(xLen-2) 256^(xLen-2) + ...
+ x_1 256 + x_0,
where 0 <= x_i < 256 (note that one or more leading digits will be
zero if x is less than 256^(xLen-1)).
3. Let the octet X_i have the integer value x_(xLen-i) for 1 <= i <=
xLen. Output the octet string
X = X_1 X_2 ... X_xLen.
Re: Conceptual feedback on new ECC JEP
On 9/4/2018 2:01 PM, Michael StJohns wrote: Below *buzz* wrong answer. Sorry. The internal storage of the key can be anything you want it to be if you want to prevent a non-constant-time issue for normal calculation. But the import/export of the key really isn't subject to the cargo cult "must not branch" dogma - hint - I'm moving across a security boundary line anyway. So if I do a "getEncoded()" or a "getS()" on an ECPrivateKey object or provide a BigInteger on an ECPrivateKeySpec there is no additional threat over the presence of the private key bits in public space. I think what you are suggesting is that the implementation should convert between BigInteger and the internal representation when necessary. The problem with this approach is that it is too easy to inadvertently supply a BigInteger to the implementation, and this would result in a branch. I understand that this branch may be acceptable in some circumstances, but we would need something in the API to tell the implementation whether it is okay to branch or not. I think the simplest way to do that is to have a provider that never branches. If branching is okay, then SunEC can be used. If you believe this to be such a problem, then I'd suggest instead updating BigInteger to allow for BigEndian or LittleEndian encoding input/output and fix the constant time issue there inside the implementation rather than breaking public APIs. BigInteger wasn't designed for this sort of thing, and changing it so that supports constant-time encoding/decoding is a massive undertaking, if it is even possible. It would be more reasonable to add a new public type for integers that supports constant-time operations, but I don't think that helps us here. Alternately, I guess you could throw some sort of exception if someone tries to call getS() or getEncoded(). Or you could do what PKCS11 does for non-extractable private keys and only class the private key as a PrivateKey - make it opaque. But then how do you create the key from stored information? That's essentially the plan. Calling PrivateKey::getEncoded will return null, which is a convention for non-extractable keys. Trying to convert from/to an ECPrivateKeySpec using the KeyFactory in the new provider will result in an exception---so you won't have an object to call getS() on. To create the key from stored information, the best way is to construct a PKCS8EncodedKeySpec using the encoded key. If you are starting with a BigInteger, and if branching is acceptable, you can use the KeyFactory from SunEC to convert an ECPrivateKeySpec to PrivateKey to get the encoded value.
Re: Conceptual feedback on new ECC JEP
Below On 9/4/2018 8:57 AM, Adam Petcher wrote: On 9/1/2018 2:03 PM, Michael StJohns wrote: On 8/23/2018 1:50 PM, Adam Petcher wrote: It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Um... why? EC Private keys are integers I've said this multiple times and - with the single exception of EDDSA keys because of a very idiosyncratic (and IMHO short-sighted) RFC specification - all of the EC private keys of whatever curve can be expressed as integers. The explanation is in the JEP: "The existing API for ECC private keys has some classes that specify private scalar values using BigInteger. There is no way to get a value out of a BigInteger (into, for example, a fixed-length array) without branching." There is no problem with making private keys integers in the API. The problem is specifically with BigInteger and its implementation. BigInteger stores the value in the shortest int array possible. To access the value, you need to branch on the length of the array, which leaks whether the high-order bits of the private key are 0. *buzz* wrong answer. Sorry. The internal storage of the key can be anything you want it to be if you want to prevent a non-constant-time issue for normal calculation. But the import/export of the key really isn't subject to the cargo cult "must not branch" dogma - hint - I'm moving across a security boundary line anyway. So if I do a "getEncoded()" or a "getS()" on an ECPrivateKey object or provide a BigInteger on an ECPrivateKeySpec there is no additional threat over the presence of the private key bits in public space. If you believe this to be such a problem, then I'd suggest instead updating BigInteger to allow for BigEndian or LittleEndian encoding input/output and fix the constant time issue there inside the implementation rather than breaking public APIs. Alternately, I guess you could throw some sort of exception if someone tries to call getS() or getEncoded(). Or you could do what PKCS11 does for non-extractable private keys and only class the private key as a PrivateKey - make it opaque. But then how do you create the key from stored information? Later, Mike
Re: Conceptual feedback on new ECC JEP
On 9/1/2018 2:03 PM, Michael StJohns wrote: On 8/23/2018 1:50 PM, Adam Petcher wrote: It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Um... why? EC Private keys are integers I've said this multiple times and - with the single exception of EDDSA keys because of a very idiosyncratic (and IMHO short-sighted) RFC specification - all of the EC private keys of whatever curve can be expressed as integers. The explanation is in the JEP: "The existing API for ECC private keys has some classes that specify private scalar values using BigInteger. There is no way to get a value out of a BigInteger (into, for example, a fixed-length array) without branching." There is no problem with making private keys integers in the API. The problem is specifically with BigInteger and its implementation. BigInteger stores the value in the shortest int array possible. To access the value, you need to branch on the length of the array, which leaks whether the high-order bits of the private key are 0.
Re: Conceptual feedback on new ECC JEP
On 8/23/2018 1:50 PM, Adam Petcher wrote: It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Um... why? EC Private keys are integers I've said this multiple times and - with the single exception of EDDSA keys because of a very idiosyncratic (and IMHO short-sighted) RFC specification - all of the EC private keys of whatever curve can be expressed as integers. Mike
Conceptual feedback on new ECC JEP
I'm starting work on yet another ECC JEP[1], this time with the goal of developing improved implementations of existing algorithms, rather than implementing new ones. The JEP will re-implement ECDH and ECDSA for the 256-, 384-, and 521-bit NIST prime curves. The new implementation will be all Java, and will resist side-channel attacks by not branching on secrets. It will go in a new provider which is not in the provider list in the java.security file by default. So it will need to be manually enabled by changing the configuration or putting the new provider name in the code. It will only support a subset of the API that is supported by the implementation in SunEC. In particular, it will reject any private keys with scalar values specified using BigInteger (as in ECPrivateKeySpec), and its private keys will not return scalar values as BigInteger (as in ECPrivateKey.getS()). Please take a look and send me any feedback you have. I'm especially looking for suggestions on how this new implementation should fit into the API. I would prefer to have it enabled by default, but I can't think of a way to do that without either branching on secrets in some cases (converting a BigInteger private key to an array) or breaking compatibility (throwing an exception when it gets a BigInteger private key). [1] https://bugs.openjdk.java.net/browse/JDK-8204574
