Just catching up on emails. I agree about PKCS11 as well, but vast amounts of people use it and I don’t think it is going away anytime soon.
This is a great discussion. So Michael, I think you are saying we should be able to use a combination of KeyAgreement and Cipher to implement a KEM in the JCA. I think that works in practice for any existing Key Agreement or Cipher (that is what we essentially use in our IETF draft for turning existing Key agreement or Cipher algorithms into a KEM). https://datatracker.ietf.org/doc/draft-ounsworth-pq-composite-kem/ However, for a pure KEM like Kyber I don’t think you can just assume you will be able to break up the encapsulation() procedure which returns a CipherText and Shared-Secret given a Kyber public key. It defines the encapsulation and decapsulation operations (see https://datatracker.ietf.org/doc/draft-cfrg-schwabe-kyber/). A Cipher encrypt can’t model the Encapsulation operation as it just returns a byte[]. I suppose a Cipher Decrypt could model the Decapsulation() operation with the result being the shared secret, but that is only half the picture…. Kyber has internal functions that do encrypt and decrypt like operations which could be modeled as a Cipher in JCA (but the message would have to be some type of specially formatted structure containing the m and cpaSeed values, or if a keywrap the public key would have to be structured along with those required values). So even if we did that, how do you propose the rest of the Kyber KEM operations fit in the current Java JCA? It seems you would need to somehow split up the components of the algorithm across different parts of the JCA and would it be possible to hide the complexity as simply as adding a KEM JCA object type with an encapsulate() method and a decapsulate() method? From https://datatracker.ietf.org/doc/draft-cfrg-schwabe-kyber/ 11.2. Encapsulation Kyber encapsulation takes a public key and a 32-octet seed and deterministically generates a shared secret and ciphertext for the public key as follows. 1. Compute 1. m = H(seed) 2. (Kbar, cpaSeed) = G(m || H(pk)) 3. cpaCipherText = Kyber.CPAPKE.Enc(m, publicKey, cpaSeed) 2. Return 1. cipherText = cpaCipherText 2. sharedSecret = KDF(KBar || H(cpaCipherText)) 11.3. Decapsulation Kyber decapsulation takes a private key and a cipher text and returns a shared secret as follows. 1. Split privateKey into 1. A 12*k*n/8-octet cpaPrivateKey 2. A 12*k*n/8+32-octet cpaPublicKey 3. A 32-octet h 4. A 32-octet z 2. Compute 1. m2 = Kyber.CPAPKE.Dec(cipherText, cpaPrivateKey) 2. (KBar2, cpaSeed2) = G(m2 || h) 3. cipherText2 = Kyber.CPAPKE.Enc(m2, cpaPublicKey, cpaSeed2) 4. K1 = KDF(KBar2 || H(cipherText)) 5. K2 = KDF(z || H(cipherText)) 3. In constant-time, set K = K1 if cipherText == cipherText2 else set K = K2. 4. Return 1. sharedSecret = K It can *sort of* fit with a KeyAgreement if you do this, but its kludgy: On sending side: KeyAgreement kem = KeyAgreement.getInstance(“Kyber”); Kem.init(null, KEMparameters) - I’m generating the CipherText and shared-secret for 1 other person, I don’t have their private key and its not multi-party KEMCipherTextKey = Kem.doPhase(Key kemPublicKey, true) byte[] sharedSecret = generateSecret() The KEMCipherTextKey contains the CipherText that just happens to implements the Key interface. It is very weird, but we something to carry the cipher text. Send KEMCipherTextKey to the receiver: On receiving side: KeyAgreement kem = KeyAgreement.getInstance(“Kyber”); Kem.init(KEMCipherTextKey, KEMParameters); -> The CipherTExt is the KEMCipherTExtKey null = Kem.doPhase(KemPrivateKey, true) -> Shared secret is generated from CipherText and PrivateKey, but a Key object is not returned byte[] sharedSecret = generateSecret() So it can work, but it is kludgy. The placement of the keys could be reversed (the public and private keys could be passed in via init, then null in the first doPhase, and the CipherTextKey in the second doPhase. I don’t know which is better as it could work either way. This just shows how it doesn’t fit cleanly… In the openSSL-OQS port which is in C, they have KEM’s defined simply as follows: https://github.com/open-quantum-safe/liboqs/blob/main/src/kem/kem.h Which follows which NIST outlines in https://csrc.nist.gov/CSRC/media/Projects/Post-Quantum-Cryptography/documents/example-files/api-notes.pdf Obviously we can make it better in Java while keeping it simple. For example: KEMEncapsulation = KEM.encapsulate(publicKey); byte[] ss = KEM.decapsulate(privateKey, CipherText); KEMEncapsualtion simply contains the shared secret (ss) and CipherText… Or to fit with init() pattern the rest of them use: KEM.init(publicKey); KEMEncapsulation = KEM.encapsulate() byte[] cipherText = KEMEncapsulation.getCipherText(); And then KEM.init(privateKey); byte[] ss = KEM.decapsulate(cipherText) or maybe even better: KEM.init(publicKey); byte[] cipherText = KEM.encapsulate() byte[] sharedSecret = KEM.getSharedSecret() And then KEM.init(privateKey); byte[] ss = KEM.decapsulate(cipherText) Cheers, John Gray From: David Hook <d...@cryptoworkshop.com> Sent: Sunday, August 21, 2022 10:51 PM To: Michael StJohns <mstjo...@comcast.net>; John Gray <john.g...@entrust.com> Cc: security-dev@openjdk.org Subject: [EXTERNAL] Re: Is there a KEM (Key Encapsulation Mechanism) architecture being proposed for the JCA? WARNING: This email originated outside of Entrust. DO NOT CLICK links or attachments unless you trust the sender and know the content is safe. ________________________________ I'd have to agree about PKCS 11. One more thing about the PQC KEMs - the KDF step is built in. As you've mentioned, previously there's been a lot of possible combinations with key agreement, with PQC KEMs this has changed (of course, you could still use a KDF too, but the original reasons for doing so no longer apply). Regards, David On 21/8/22 13:52, Michael StJohns wrote: On 8/20/2022 2:08 PM, David Hook wrote: Hi Michael, I don't know anything about bureaucrats, I am an engineer. You may need to consult someone else on bureaucrats. I apologize for my apparent deficiencies in this area, but would you mind explaining how Cipher.wrap() is either supposed to take a public key and create an encapsulation based on it and return a secret key implicitly in one clean move, or why it even makes sense to do so. The method was never conceived as providing the functionality for what a KEM actually does, and when I did the initial PKCS11 implementation at Eracom in the late 90's and the team at Sun added the wrap/unwrap functions to support it, this is definitely not was intended either - it was for explicit key wrapping based on the key that was passed to Cipher.init(). First - PKCS11 is a 40 year old API that probably needs to be retired. I spent the better part of 2 years working with the PKCS11 Oasis group trying to get them to properly support master secrets and KDFs and failed utterly. You should not use PKCS11 as an example that the JCE should use as a goal. At the base, a java class is a collection of objects. A Cipher object Let's build a non-parameterized ECIES-KEM which implicitly uses AES256 bit keys to key a GCM cipher, and a KDF based on SP800-108 counter mode with SHA256 as the underlying hash, and with a well known label and context for the KDF since there is a new key for every wrap. 1) Implement CipherSpi - public class EciesKemCipher extends CipherSpi { private KeyAgreement ka; private Cipher gcm; private KeyPair kp; private KeyPairGenerator kg; EciesKemCipher() { ka = KeyAgreement.getInstance("ECDH"); kpg = KeyPairGenerator.getInstance ("EC"); gcm = Cipher.getInstance ("AES/GCM/NoPadding"); } // implement a single example @override protected void engineInit (int opMode, Key key, SecureRandom dontcare) { switch (opMode) { Cipher.MODE_WRAP: initWrap((ECPublicKey) key); break; default: // unimpl } } private void initWrap (ECPublicKey k) { ECParameterSpec spec = k.getParams(); kpg.initialize(spec); kp = kpg.genKeyPair(); ka.init (kp.getPrivate()); ka.doPhase (k, true); byte[] sharedSecret = ka.generateSecret(); byte[] keyStream = kdf(sharedSecret, 32 + 12); // output 44 bytes for Key and IV SecretKeySpec gcmKey = new SecretKeySpec (keyStream, 0, 32, "AES"); IvParameterSpec gcmIv = new IVParameterSpec(keyStream, 32, 12); gcm.init (Cipher.MODE_ENCRYPT, gcmKey, gcmIv); // all ready to go } protected byte[] engineWrap (Key k) { ByteBuffer outData = ByteBuffer.allocate (k.getEncoded().length + kp.getPublic().getEncoded().length) + 16; // Place a copy of the ephemeral public key I generated in init here for the use of the receiver. outData.put (kp.getEncoded()); // One s outData.put (gcm.doFinal(k.getEncoded()); outData.flip(); byte[] result = outdata.remaining(); outData.get(result); // kp = null; clear cipher if it hasn't already been cleared, clear ka if necessary (e.g. un-init) return result; } ... and unwrap and kdf function } 2) Implement a provider and add the above. On BC's part, we've already implemented RFC 5990/SP 800-56B in Java and the experience has, at best, been awkward. The new algorithms have moved awkward to inappropriate. With the new algorithms, there's no longer only one case of this, it's not an outlier, there should be a general way of supporting KEMs in the API that doesn't involve over engineering KeyGenerator and Cipher. There's a big difference between the API and your underlying implementation. Everything you want to do can be done using the current APIs. As I said before, Cipher.wrap/unwrap are the appropriate APIs for this as they meet the contract requirements you need. Most Ciphers require some extra data -e.g. IVs - that have to either be carried or implicitly derived. In this case, what needs to be carried in addition to the encrypted key material is at least the ephemeral public key the wrapper creates. I used a very simple encoding scheme above and this assumes that both ends know exactly what "ECIES-KEM" means. Obviously, there are 100s of possible combinations of parameters and KDFs and key wrap algorithms. What I would suggest is heading over to LAMPS at the IETF and proposing a data encoding scheme for carrying the parameters. Once you have that done, then come here and map JCE names against parameter sets to close the loop. It won't require an API change. I work with a team that has had to implement all of them and had to make them fit into the JCA. We have done so. Like John, I am simply relaying our experience. In about 18 months these algorithms are going to become mandatory, what all of us think is irrelevant. We, for our part, already have a solution, but we both realize it's not "the solution" - we recognize that the JVM is uniquely positioned to provide leadership on this and provide a universal way of doing it. Then suggest an API and we'll start knocking it around. I personally don't think its necessary at this time and will add to API bloat. Of course, if it's felt that these algorithms should be ignored, it's not my place to revolt, although I do feel obliged to argue. I will simply try and do the best by my users, as I have no doubt will John. Both of us have simply offered our comments in good faith and to alert the community that things have changed and that with these new algorithms there is room for a new approach. The ambiguity about how these algorithms can be implemented and the excessive need to fallback on propritary classes for them does suggest that there are some additions to the JCA which would help. I appreciate to understand this statement does involve actually understanding what these algorithms do and may require some additional reading. As I said, I'm an engineer, my users will be able to use these algorithms properly, my team will ensure that, as I have no doubt will John's. What John and myself, apparently mistakenly, care about is that our users should also be able to use these algorithms portably. Are you saying portability is no longer a consideration? I have no idea where you got that idea. Regards, David On 21/8/22 02:23, Michael StJohns wrote: Hi David/John - I would submit that you're trying too hard to make your life simple! :-) Cipher.wrap/unwrap are the correct methods. For example: Cipher kem = Cipher.getInstance ("ECIES/GCM-128-64/KDF-SP800-108-COUNTER-SHA256"); kem.init (Cipher.WRAP_MODE, pubkey); byte[] opaqueEncapsulatedKey = kem.wrap (someOtherKey); The "opaqueEncapsulatedKey" would contain the data needed by the unwrap function - specifically a) the ecies ephemeral public key, b) the fact that the derived key is a GCM key of length 128 and that the GCM tag is 64 bytes long, c) the KDF, d) (optional) any mixins other than defaults required by the KDF - which would be passed in a parameter blob during init. Cipher would NOT return the underlying generated secret used to wrap the key. Just the public part of the key pair used to do the ECDH operation against the passed in public key. In the RSA case, the wrapped encrypting secret would be an opaque data blob and would be part of the data passed to the unwrap function. If you want a key generated for other purposes, then the right thing is using a KDF and a Key agreement function in tandem. Strangely the KDF appears in the javacard API for 3.1, but not in the JCE/JDK API. "What's the difference between a bureaucrat and an engineer? A bureaucrat takes small solvable pieces and combines them into one insoluble mass." In this case, Java provides a number of flexible primitives that can be combined as needed. In this case, the underlying Cipher implementation would wrap key agreement and kdf and cipher (GCM) instances. It should return UnsupportedOperationException for all operations execept wrap/unwrap and the appropriate init methods. Later, Mike On 8/19/2022 6:38 PM, David Hook wrote: Hi Mike, KEMs can be used for key wrapping - we've actually implemented support for this too. But they are not actually key wrapping ciphers. Here's a simple example of using Kyber for key wrapping in BC: SecretKey key = new SecretKeySpec(keyBytes, "AES"); w1.init(Cipher.WRAP_MODE, kp.getPublic(), new KEMParameterSpec("AES-KWP")); byte[] data = w1.wrap(key); Cipher w2 = Cipher.getInstance(algorithm, "BCPQC"); w2.init(Cipher.UNWRAP_MODE, kp.getPrivate(), new KEMParameterSpec("AES-KWP")); Key k = w2.unwrap(data, "AES", Cipher.SECRET_KEY); The behavior in this case is in line with what is given in RFC 5990 for the RSA KEM. How it works is by using the key generated by the KEM to create an AES-KWP key, which is then used to wrap keyBytes. The shortcoming is it means you have to generate the secret key separately. This is the problem though - a KEM can actually be used to generate a secret key for other purposes. For example, where someone is trying to implement a hybrid KAS scheme. But there is currently no mechanism in the Java APIs for being able to take advantage of this directly, hence our use of the KeyGenerator class and other people's attempts to make use of the KeyAgreement class. The Cipher.wrap() returns a byte[] - to be used with a KEM for secret generation it would also have to return the generated secret (I would probably also argue that passing a public key to wrap in order to generate an encapsulation of a generated encrypted secret was not the correct use of the API either, but the fact remains a byte[] is not really going to cut it). If you have any further questions, please feel free to ask. For what it is worth, I have been developing providers for the JCE/JCA since the late 90's and am actually one of the people responsible for the introduction of the existing wrap/unwrap API in the Cipher class. Thanks, David On 20/8/22 07:53, Mike StJohns wrote: Hi This implemented as part of Javax.crypto.Cipher. See the Java doc for the wrap and unwrap methods. Mike Sent from my iPad On Aug 19, 2022, at 12:56, John Gray <john.g...@entrust.com><mailto:john.g...@entrust.com> wrote: We are starting to make use of the new PQ algorithms adopted by NIST for prototyping and development of standards. In particular we are working on a composite KEM standard: See: https://datatracker.ietf.org/doc/draft-ounsworth-pq-composite-kem/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ounsworth-pq-composite-kem/__;!!FJ-Y8qCqXTj2!aq1XN3bBGrr6HdA54EJyCj9B6UftE-mPYHo2CGe5mErDXCLDYCYk4RGeUwPvz2ZJcvumylSDBp5YsbmdIO4$> However, there is no KEM interface in the JCA (which make sense because these are new algorithms, although RSA-KEM has been out since 2010). I can add one into our toolkit (and I think David may have already added on into BC), but I assume at some point there will be an official one added in Java and likely it won't be identical to what we do even if it is very close, which would cause backwards compatibility pain... Perhaps we could collaborate on extending the JCA to support KEM? Essentially it requires methods. ss, ct := encapsulate(PublicKey) ss := decapsulate(PrivateKey, ct) -ss is a shared secret (could come back as a Java SecretKey if you wanted as it would usually be used to derive something like an AES afterwards) -ct is a Cipher Text (a byte array would make sense) -Public and Private Keys would use the regular public and private key interface. -An object holding the ss and ct from the encapsulate() method could be returned, with accessor methods to get the ss and ct. It could be called 'EncapsulatedKEMData' for example. Likely you would want a new type of KEM crypto object (like you have for Signature, MessageDigest, Cipher, Mac, SecureRandom, KeyAgreement.. etc). Calling it KEM would seem to make sense. 😊 It could also use similar calling patterns and have a KEM.initKEM(keypair.getPublic()) or KEM.initKEM(keypair.getPrivate()), and then you would just call KEM.encapsulate() or KEM.decapsulate(ct). Then algorithms could be registered in providers as usual: put("KEM.Kyber","com.blah.Kyber") put("KEM.compositeKEM","com.entrust.toolkit.crypto.kem.compositeKEM") Then the above methods (encapsulate and decapsulate) could be defined in that new object type. Then we would be able to make use of it and not have to worry about incompatibility issues down the road... Cheers, John Gray Any email and files/attachments transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. If this message has been sent to you in error, you must not copy, distribute or disclose of the information it contains. Please notify Entrust immediately and delete the message from your system.
