Re: Draft design for Key Derivation API

[Adam Petcher]

On 11/19/2017 3:15 PM, Michael StJohns wrote:



That behavior all sounds reasonable, I just have doubts that this 
belongs in the spec. Are you expecting KeyDerivation to contain the 
logic in your last paragraph? Something like this:





KDFs are somewhat problematic in that *_they may not necessarily be 
producing objects from their own provider_*. This unfortunately isn't 
obvious, but let me try and explain.





Your response didn't contain a direct answer to my question above. If I 
am interpreting your response correctly, then your answer is "Yes, and 
we may need some additional information in DerivationParameterSpec (or 
elsewhere) that controls this logic." Though I'm not sure I am 
interpreting this correctly, so please let me know.


To be clear: I don't object to including the method that returns an 
Object produced by a KDF. I'm specifically asking about the requirement 
that this class of objects has a (byte[] int) constructor, and how that 
constructor is expected to be used.

Re: Draft design for Key Derivation API

[Michael StJohns]

Sorry - I've been on travel for the last few days... comments below.

On 11/17/2017 10:48 AM, Adam Petcher wrote:

On 11/17/2017 10:10 AM, Michael StJohns wrote:


On 11/16/2017 1:29 PM, Adam Petcher wrote:

On 11/8/2017 6:50 PM, Michael StJohns wrote:

What is the motivation behind this constructor that takes a byte 
array? It seems like this constructor wouldn't actually help in a 
hardware implementation. Would it be better to leave the 
construction of this object to the implementation?


This is a reasonable point, but misses a few things.   If you're 
calling the hardware implementation from software, you need to be 
able to pass data from the software domain to hardware domain.  If 
the KDF and the Object are both in hardware, then the provider 
implementation doesn't actually externalize the byte array from the 
KDF - it just returns the final pointer to the object.


The hardware/software boundary has some unique challenges - mostly 
these are handled OK in the JCA.  For this particular model, you need 
to be able to move bits from software to hardware which is the point 
of the constructor as specified. For hardware to hardware it happens 
under the hood.  For hardware to software it may be prohibited (e.g. 
you can't actually externalize the bits of the key stream), but if 
its permitted then you need a simple way of translating key stream 
bytes into an object.


That behavior all sounds reasonable, I just have doubts that this 
belongs in the spec. Are you expecting KeyDerivation to contain the 
logic in your last paragraph? Something like this:


class KeyDerivation{
  Object deriveObject() {
    try {
  return spi.deriveObject();
    } catch (...) {
  Class clazz = // get the class from the parameters
  return clazz.newInstance(deriveData(), length); // shorthand for 
getting the right ctor and calling it

    }
  }
}

I would expect something like that to happen in the KeyDerivationSpi 
implementation instead, in which case it could construct the object 
any way it wants. So the spec would not need to place any requirements 
on the class of objects returned by deriveObject.




KDFs are somewhat problematic in that *_they may not necessarily be 
producing objects from their own provider_*. This unfortunately isn't 
obvious, but let me try and explain.


A KDF is basically a keyed pseudo-random number generator.  From the 
input key (and mixin context and label stuff) it produces a stream of 
bits.   Once that's done, the stream of bits is assigned to various 
output objects - secret keys, private keys, a byte array[] or a 
cryptographic object of some sort (cf TLS exporters for an example of 
this).  The current draft has an implicit assumption that the Key based 
objects will be formed from the current provider.  The byte array output 
is a providerless java.lang object.  The last type provides a model to 
allow for the production of objects not within the current provider.   
You *could* just punt on this and assume that you take the output of the 
deriveData() call and feed it to a factory of the other provider, but 
that means that the derivation production will necessarily be in the 
clear because the stream data will pass through the JVM.


Here's where it gets even trickier:

A given provider has a given security domain.  E.g. most software 
providers share the JVM memory domain.  HSM providers have a security 
domain representing pointers to objects within the HSM secure 
perimeter.  Mostly now, HSM providers do not share the same domains - 
but there are some cases where this might be possible and desirable 
(different providers leveraged on top the same HSM implementation, two 
SMs with a trusted path between them - TPMs and TEEPs for example).  I'd 
*really* like it if there is some way to keep  data from two different 
providers sharing the same or compatible security domains from having to 
pass their key stream data through the unsecure JVM memory domain.


Maybe there's a different way to do this - perhaps by changing the 
DeriviationParameterSpec to include the output provider?  But then you 
still need a way of generating non-key secure cryptographic objects I 
think


Mike

Re: Draft design for Key Derivation API

[Adam Petcher]

On 11/17/2017 10:10 AM, Michael StJohns wrote:


On 11/16/2017 1:29 PM, Adam Petcher wrote:

On 11/8/2017 6:50 PM, Michael StJohns wrote:

What is the motivation behind this constructor that takes a byte 
array? It seems like this constructor wouldn't actually help in a 
hardware implementation. Would it be better to leave the construction 
of this object to the implementation?


This is a reasonable point, but misses a few things.   If you're 
calling the hardware implementation from software, you need to be able 
to pass data from the software domain to hardware domain.  If the KDF 
and the Object are both in hardware, then the provider implementation 
doesn't actually externalize the byte array from the KDF - it just 
returns the final pointer to the object.


The hardware/software boundary has some unique challenges - mostly 
these are handled OK in the JCA.  For this particular model, you need 
to be able to move bits from software to hardware which is the point 
of the constructor as specified.  For hardware to hardware it happens 
under the hood.  For hardware to software it may be prohibited (e.g. 
you can't actually externalize the bits of the key stream), but if its 
permitted then you need a simple way of translating key stream bytes 
into an object.


That behavior all sounds reasonable, I just have doubts that this 
belongs in the spec. Are you expecting KeyDerivation to contain the 
logic in your last paragraph? Something like this:


class KeyDerivation{
  Object deriveObject() {
    try {
  return spi.deriveObject();
    } catch (...) {
  Class clazz = // get the class from the parameters
  return clazz.newInstance(deriveData(), length); // shorthand for 
getting the right ctor and calling it

    }
  }
}

I would expect something like that to happen in the KeyDerivationSpi 
implementation instead, in which case it could construct the object any 
way it wants. So the spec would not need to place any requirements on 
the class of objects returned by deriveObject.

Re: Draft design for Key Derivation API

[Michael StJohns]

On 11/16/2017 1:29 PM, Adam Petcher wrote:

On 11/8/2017 6:50 PM, Michael StJohns wrote:


On 11/3/2017 4:59 PM, Jamil Nimeh wrote:
Add a .deriveData() with a return class of byte[].   This gets a 
portion of the derived data stream in the clear. E.g. an IV.


Add a .deriveObject() with a return class of Object.  The returned 
object may not be an instance of java.security.Key.  This takes 
the derived data stream and converts it into the object type 
specified by the derivation parameter.  In a hardware security 
module, this might be a reference to a secured set of data or even 
an confidential IV.
Again, just want to make sure I understand fully: So in a case 
where I want a given output to be an Object, I would provide a 
DerivationParameterSpec with an alg of..."Object" (?), a byte 
length, and Object-specific parameters provided through the 
"params" argument to the DPS?


Working this through, but it should be a Class  being specified with 
a constructor of a byte array plus a length.


What is the motivation behind this constructor that takes a byte 
array? It seems like this constructor wouldn't actually help in a 
hardware implementation. Would it be better to leave the construction 
of this object to the implementation?


This is a reasonable point, but misses a few things.   If you're calling 
the hardware implementation from software, you need to be able to pass 
data from the software domain to hardware domain.  If the KDF and the 
Object are both in hardware, then the provider implementation doesn't 
actually externalize the byte array from the KDF - it just returns the 
final pointer to the object.


The hardware/software boundary has some unique challenges - mostly these 
are handled OK in the JCA.  For this particular model, you need to be 
able to move bits from software to hardware which is the point of the 
constructor as specified.  For hardware to hardware it happens under the 
hood.  For hardware to software it may be prohibited (e.g. you can't 
actually externalize the bits of the key stream), but if its permitted 
then you need a simple way of translating key stream bytes into an object.


Mike

Re: Draft design for Key Derivation API

[Adam Petcher]

On 11/8/2017 6:50 PM, Michael StJohns wrote:


On 11/3/2017 4:59 PM, Jamil Nimeh wrote:
Add a .deriveData() with a return class of byte[].   This gets a 
portion of the derived data stream in the clear. E.g. an IV.


Add a .deriveObject() with a return class of Object.  The returned 
object may not be an instance of java.security.Key.  This takes the 
derived data stream and converts it into the object type specified 
by the derivation parameter.  In a hardware security module, this 
might be a reference to a secured set of data or even an 
confidential IV.
Again, just want to make sure I understand fully: So in a case where 
I want a given output to be an Object, I would provide a 
DerivationParameterSpec with an alg of..."Object" (?), a byte 
length, and Object-specific parameters provided through the "params" 
argument to the DPS?


Working this through, but it should be a Class  being specified with a 
constructor of a byte array plus a length.


What is the motivation behind this constructor that takes a byte array? 
It seems like this constructor wouldn't actually help in a hardware 
implementation. Would it be better to leave the construction of this 
object to the implementation?

Re: Draft design for Key Derivation API

[Michael StJohns]

On 11/7/2017 8:17 PM, Jamil Nimeh wrote:


Hi Mike, thank you for your comments and feedback.  I have a few 
comments and questions inline:




(I have a little bit more time before my flight than i thought so 
see inline).



On 11/06/2017 05:25 PM, Michael StJohns wrote:

On 11/3/2017 4:59 PM, Jamil Nimeh wrote:

Hello all,

This is a review request for the draft of a new Key Derivation API.  
The goal of this API will be to provide a framework for KDF 
algorithms like HKDF, TLS-PRF, PBKDF2 and so forth to be publicly 
accessible.  We also plan to provide an SPI that let 3rd parties 
create their own implementations of KDFs in their providers, rather 
than trying to force them into KeyGenerators, SecretKeyFactories and 
the like.


Rather than stuff this email full of the specification text (since 
it is likely to get quite a few iterations of comments and 
comments-to-comments), I have placed the API both in simple text 
form and as a Javadoc at the following locations:


spec: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/kdfspec.01.txt

javadoc: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/javadoc.01/

They're both the same content, just use whichever is friendlier for 
your eyes.


In addition, I have opened up the JEP as well:

https://bugs.openjdk.java.net/browse/JDK-8189808

Thanks to those who have contributed to very early internal drafts 
of this so far, and thanks in advance to those who will be 
contributing comments going forward.


--Jamil


Most of the following suggestions (and please take them as such 
regardless of any directive language) represent things I've had to do 
manually that I'd really prefer to do in a real key derivation API.  
A few are related to how to keep things securely stored in an HSM.


Add a .reset() method to KeyDerivation.  Call this to clear the state 
of the KDF.


Add an .initialize(List, SecretKey 
masterSecret) method.  Remove the argument to deriveKey and 
deriveKeys.  This plays with the stuff to follow, but basically, a 
KDF may need all of the per-key derivation input to calculate the 
total length of the output key stream as an internal input to the KDF 
before ever emitting a single key.   Also - how exactly were you 
planning on keying the KDF? I guess you could pass that in in the 
KeyDerivation.getInstance() call or as part of the algorithmParameter 
but probably makes more sense to keep the KDF instance key-free 
to allow for reuse.
Well, let's get the easy one out of the way.  As you suspected I 
planned to pass the SecretKey in via AlgorithmParameterSpec.  The 
three classes unfortunately didn't show that.  Maybe on the next 
iteration I can put an HkdfParameterSpec in there just as a sample so 
folks can see that where the key comes in.  The reason I went that way 
was because the goal was to provide all algorithm paramters at 
instantiation time, and the SecretKey was just another input.  I don't 
know if just making the KDF key-free would be enough for reuse, at 
least not for all cases.  Thinking about HKDF and TLS 1.3 for 
instance, the key is the same for a collection of keys (like the 
client and server app traffic master keys that come from the master 
secret, for instance) - what changes are the other inputs to HKDF.


Yup - but that's easily handled through the new initialization call - 
which again matches the way Cipher, Signature and KeyAgreement do 
things.   Simplifying (??) the interface just to make one use case 
easier is probably not a great tradeoff.





One issue that came up on an early internal rev of the API was that we 
didn't want to separate instantiation and initialization, so all the 
inputs to the KDF now come in at getInstance time through 
AlgorithmParameterSpecs, rather than doing getInstance/init/... like 
KeyAgreement does.  I wonder if it would be OK to still have an init 
(and a reset as you wanted) method so we can provide new inputs 
top-to-bottom into the KDF object.  All the getInstance forms would 
stay more or less the same, so there's no way to make a KDF object 
without it being in an initialized state.  But when you need new 
inputs you don't have to make a new object.  I like being able to 
reuse the object and reset it to its starting state.  I don't know if 
the folks that brought up the instance/init issue would have a problem 
with that.  I think we're still adhering to the spirit of what they 
wanted to see since getInstance still gives you a fully initialized 
object.


As I noted in my other email, that's not the general form of a contract 
in the JCA.




That's a bit different than what you're talking about with your 
initialize method, I kinda birdwalked a bit.  Let me ask a couple 
questions: When you proposed initialize(), were you envisioning that 
applications would always need to call it before derive*?


Yes, init would always need to be called before you begin derives. A KDF 
call would require an instantiation (where you pass the parameters of 
the mechanism - 

Re: Draft design for Key Derivation API

[Mike StJohns]

Hi - I on a plane tomorrow so it may be a few days before I can completely 
answer the email, but I wanted to hit this section.


Think of KeyDerivation as something that has a pattern similar to Cipher or 
KeyAgreement or Signature. The instantiation phase sets up the mechanism of the 
crypto object, but allows you to use that object with many keys through an 
init/reset (implicit or explicit) cycle.   That model works well with both 
software and hardware implementations.   In literally no case can I see it 
making sense to irrevocably bind the key to the keyderivation object and every 
reason not to do it.  Consider for a minute the server side of TLS.  Wouldn’t 
it make sense to instantiate a single KeyDerivation object and throw different 
master keys at it as they are negotiated?  Especially if the KeyDerivation 
object is an abstract for an HSM implementation.  

None of the other crypto main classes feed in the key during instantiation.  I 
don’t think it’s a good idea to vary the pattern without a better reason than 
you’ve given.  

more later -Mike



Sent from my iPad

> On Nov 7, 2017, at 17:17, Jamil Nimeh  wrote:
> 
> One issue that came up on an early internal rev of the API was that we didn't 
> want to separate instantiation and initialization, so all the inputs to the 
> KDF now come in at getInstance time through AlgorithmParameterSpecs, rather 
> than doing getInstance/init/... like KeyAgreement does.  I wonder if it would 
> be OK to still have an init (and a reset as you wanted) method so we can 
> provide new inputs top-to-bottom into the KDF object.  All the getInstance 
> forms would stay more or less the same, so there's no way to make a KDF 
> object without it being in an initialized state.  But when you need new 
> inputs you don't have to make a new object.  I like being able to reuse the 
> object and reset it to its starting state.  I don't know if the folks that 
> brought up the instance/init issue would have a problem with that.  I think 
> we're still adhering to the spirit of what they wanted to see since 
> getInstance still gives you a fully initialized object.

Re: Draft design for Key Derivation API

[Jamil Nimeh]

Hi Mike, thank you for your comments and feedback.  I have a few 
comments and questions inline:



On 11/06/2017 05:25 PM, Michael StJohns wrote:

On 11/3/2017 4:59 PM, Jamil Nimeh wrote:

Hello all,

This is a review request for the draft of a new Key Derivation API.  
The goal of this API will be to provide a framework for KDF 
algorithms like HKDF, TLS-PRF, PBKDF2 and so forth to be publicly 
accessible.  We also plan to provide an SPI that let 3rd parties 
create their own implementations of KDFs in their providers, rather 
than trying to force them into KeyGenerators, SecretKeyFactories and 
the like.


Rather than stuff this email full of the specification text (since it 
is likely to get quite a few iterations of comments and 
comments-to-comments), I have placed the API both in simple text form 
and as a Javadoc at the following locations:


spec: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/kdfspec.01.txt

javadoc: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/javadoc.01/

They're both the same content, just use whichever is friendlier for 
your eyes.


In addition, I have opened up the JEP as well:

https://bugs.openjdk.java.net/browse/JDK-8189808

Thanks to those who have contributed to very early internal drafts of 
this so far, and thanks in advance to those who will be contributing 
comments going forward.


--Jamil


Most of the following suggestions (and please take them as such 
regardless of any directive language) represent things I've had to do 
manually that I'd really prefer to do in a real key derivation API.  A 
few are related to how to keep things securely stored in an HSM.


Add a .reset() method to KeyDerivation.  Call this to clear the state 
of the KDF.


Add an .initialize(List, SecretKey 
masterSecret) method.  Remove the argument to deriveKey and 
deriveKeys.  This plays with the stuff to follow, but basically, a KDF 
may need all of the per-key derivation input to calculate the total 
length of the output key stream as an internal input to the KDF before 
ever emitting a single key.   Also - how exactly were you planning on 
keying the KDF?  I guess you could pass that in in the 
KeyDerivation.getInstance() call or as part of the algorithmParameter 
but probably makes more sense to keep the KDF instance key-free to 
allow for reuse.
Well, let's get the easy one out of the way.  As you suspected I planned 
to pass the SecretKey in via AlgorithmParameterSpec.  The three classes 
unfortunately didn't show that.  Maybe on the next iteration I can put 
an HkdfParameterSpec in there just as a sample so folks can see that 
where the key comes in.  The reason I went that way was because the goal 
was to provide all algorithm paramters at instantiation time, and the 
SecretKey was just another input.  I don't know if just making the KDF 
key-free would be enough for reuse, at least not for all cases.  
Thinking about HKDF and TLS 1.3 for instance, the key is the same for a 
collection of keys (like the client and server app traffic master keys 
that come from the master secret, for instance) - what changes are the 
other inputs to HKDF.


One issue that came up on an early internal rev of the API was that we 
didn't want to separate instantiation and initialization, so all the 
inputs to the KDF now come in at getInstance time through 
AlgorithmParameterSpecs, rather than doing getInstance/init/... like 
KeyAgreement does.  I wonder if it would be OK to still have an init 
(and a reset as you wanted) method so we can provide new inputs 
top-to-bottom into the KDF object.  All the getInstance forms would stay 
more or less the same, so there's no way to make a KDF object without it 
being in an initialized state.  But when you need new inputs you don't 
have to make a new object.  I like being able to reuse the object and 
reset it to its starting state.  I don't know if the folks that brought 
up the instance/init issue would have a problem with that.  I think 
we're still adhering to the spirit of what they wanted to see since 
getInstance still gives you a fully initialized object.


That's a bit different than what you're talking about with your 
initialize method, I kinda birdwalked a bit.  Let me ask a couple 
questions: When you proposed initialize(), were you envisioning that 
applications would always need to call it before derive*?  Or did you 
really mean "may" and an implementation would have to go back and 
generate more material if they exhausted everything they knew about?  
Given your changes to deriveKey(s) it looked more like you intended to 
know the total length up-front, since there's no other way to say some 
arbitrary next key is of a specific length with no argument to deriveKey[s].


If you did want the total length of all keys/data/objects to be supplied 
before derivation, what if we were to supply that to the getInstance 
calls?  A similar idea was put forth internally, but we decided to hold 
off on it and wait for some feedback from the 

Re: Draft design for Key Derivation API

[Michael StJohns]

On 11/3/2017 4:59 PM, Jamil Nimeh wrote:

Hello all,

This is a review request for the draft of a new Key Derivation API.  
The goal of this API will be to provide a framework for KDF algorithms 
like HKDF, TLS-PRF, PBKDF2 and so forth to be publicly accessible.  We 
also plan to provide an SPI that let 3rd parties create their own 
implementations of KDFs in their providers, rather than trying to 
force them into KeyGenerators, SecretKeyFactories and the like.


Rather than stuff this email full of the specification text (since it 
is likely to get quite a few iterations of comments and 
comments-to-comments), I have placed the API both in simple text form 
and as a Javadoc at the following locations:


spec: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/kdfspec.01.txt

javadoc: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/javadoc.01/

They're both the same content, just use whichever is friendlier for 
your eyes.


In addition, I have opened up the JEP as well:

https://bugs.openjdk.java.net/browse/JDK-8189808

Thanks to those who have contributed to very early internal drafts of 
this so far, and thanks in advance to those who will be contributing 
comments going forward.


--Jamil


Most of the following suggestions (and please take them as such 
regardless of any directive language) represent things I've had to do 
manually that I'd really prefer to do in a real key derivation API.  A 
few are related to how to keep things securely stored in an HSM.


Add a .reset() method to KeyDerivation.  Call this to clear the state of 
the KDF.


Add an .initialize(List, SecretKey 
masterSecret) method.  Remove the argument to deriveKey and deriveKeys.  
This plays with the stuff to follow, but basically, a KDF may need all 
of the per-key derivation input to calculate the total length of the 
output key stream as an internal input to the KDF before ever emitting a 
single key.   Also - how exactly were you planning on keying the KDF?  I 
guess you could pass that in in the KeyDerivation.getInstance() call or 
as part of the algorithmParameter but probably makes more sense to 
keep the KDF instance key-free to allow for reuse.


Rename DerivedKeyParameterSpec to DeriviationParameterSpec and provide 
an algorithm name for "IV" or "Cleartext".  See below for .deriveData()


deriveKey() emits the next key in the sequence using the data stream to 
key conversion rules.


deriveKeys() emits as many keys left in the stream to the next data 
derivation or the defined end of stream based on the input specs.  
deriveKeys(int num) derives the next num keys.


Add a .deriveData() with a return class of byte[].   This gets a portion 
of the derived data stream in the clear. E.g. an IV.


Add a .deriveObject() with a return class of Object.  The returned 
object may not be an instance of java.security.Key.  This takes the 
derived data stream and converts it into the object type specified by 
the derivation parameter.  In a hardware security module, this might be 
a reference to a secured set of data or even an confidential IV.


All of the derive methods throw an InvalidParameterSpecException if the 
next derivation parameter doesn't match the calling method (e.g. trying 
to deriveData when the parameter spec says emit a key).


In KeyDerivation, change the output class of the deriveKey to 
java.security.Key; similar for deriveKeys change the output to 
List.   Basically, its possible to use the output of a KDF stream 
to derive private keys and this should be supported. It's occasionally 
helpful (but not very often) for two devices to share a key pair that 
they create through a key agreement process (e.g. two HSMs acting as 
backup to each other).  Alternately, consider adding a "public KeyPair 
deriveKeyPair()" method.


Consider adding a marker interface  javax.crypto.MasterSecret (subclass 
of javax.crypto.SecretKey) and using that as class for the initialize 
call argument.


I'm happy to provide an edited .java file with these proposed changes - 
but not until at least next Monday; I'm on travel.


Mike

Draft design for Key Derivation API

[Jamil Nimeh]

Hello all,

This is a review request for the draft of a new Key Derivation API.  The 
goal of this API will be to provide a framework for KDF algorithms like 
HKDF, TLS-PRF, PBKDF2 and so forth to be publicly accessible.  We also 
plan to provide an SPI that let 3rd parties create their own 
implementations of KDFs in their providers, rather than trying to force 
them into KeyGenerators, SecretKeyFactories and the like.


Rather than stuff this email full of the specification text (since it is 
likely to get quite a few iterations of comments and 
comments-to-comments), I have placed the API both in simple text form 
and as a Javadoc at the following locations:


spec: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/kdfspec.01.txt

javadoc: http://cr.openjdk.java.net/~jnimeh/reviews/kdfspec/javadoc.01/

They're both the same content, just use whichever is friendlier for your 
eyes.


In addition, I have opened up the JEP as well:

https://bugs.openjdk.java.net/browse/JDK-8189808

Thanks to those who have contributed to very early internal drafts of 
this so far, and thanks in advance to those who will be contributing 
comments going forward.


--Jamil