Re: [PHP-DEV] hash_hkdf() signature and return value

[Yasuo Ohgaki]

Hi Dan,

Sorry for keep posting broken English.

I shouldn't difficult.
>

It shouldn't be difficult.

Looking forward more than handful, useful and common hash_hkdf()
application
examples for PHP that justify the API. If you would not like to spend time
for working
code, just ideas are OK also.

Regards,

Yasuo

Re: [PHP-DEV] hash_hkdf() signature and return value

[Yasuo Ohgaki]

Hi Dan,

I appreciate your feedback, regardless of your opinion towards this issue.

On Wed, Sep 6, 2017 at 8:04 PM, Dan Ackroyd  wrote:

> On 6 September 2017 at 02:15, Yasuo Ohgaki  wrote:
> > What should we do for this?
>
> Not us, you.
>

OK. It is recorded that you think current API is totally valid.
I'm not sure who is "us". Please reply and record your opinion.


>
> You should start listening to other people's feedback.
>
> You continually refuse to accept any feedback that doesn't agree with
> your world-view, not only on the subject of hkdf, but on validation
> and other things you think are "MANDATORY"
>

Well, my thoughts are not totally my inventions.

For HKDF, it came from the RFC 5689 basically.

For input validations, it is originated in Defensive Programming. Defensive
Programming is referred in early 90's AFAIK, it is called secure programming
or secure coding now. I believe the basic idea was developed 60's computer
scientists who researched program execution correctness verification
methods.

You should respect RFC votes and stop bringing up the same discussions
> over and over again. This is incredibly tedious.
>

Did you really read the RFC 5689?
Please mention what is wrong with my statements if you think my statements
are totally wrong.  It should be easy to point it out, since I provided
many points.
This is technical list, not political list nor religious list after all.

I should note that no valid code example was presented that justifies
current API.

Not a single valid code example, yet.

This fact infers what you say "us" have concrete reason(s) that supports
current API validity.

Please provide undisclosed valid code example that would be more common
than CSRF token and API token derivations. There are even more URL singing,
etc in
my PHP RFC.


> In particular your suggestion about hash_hkdf was rejected
> unanimously, apart from your own vote
> https://wiki.php.net/rfc/improve_hash_hkdf_parameter and so probably
> shouldn't be brought up for discussion, except if you can bring new
> facts for discussion.
>

If any one of you provided example usages that would be common, valid
(and optimal, it should be optimal since HKDF is designed for the best
possible derived key with HMAC), I would not raise this issue again.


>
> "Not liking the result of the vote" is not a new fact to discuss.
>

Sorry  but, I'm not liking the vote result.
I'm frustrating the fact there is no code example(s) justifies current API
design
that is insecure and inconsistent.


Additionally though, your ideas about adding validation/filter
> functions as a C extension, rather than implementing them in PHP have
> also been resoundingly rejected,
> https://wiki.php.net/rfc/add_validate_functions_to_filter and yet you
> continue to promote the idea. This is also tedious.
>

It's totally new module.
I already replied to your comments in other thread.
Apparently, you are ignoring single responsibility principle.
Please take into the principle into your thoughts.

This pattern of behaviour, continually refusing to accept that other
> people have opinions that don't align with yours, and continually
> bringing up the same topics for discussion over, and over, and over
> again is not productive. It does not make people want to engage you in
> discussion, as it is a waste of their time. This is not something
> other people can fix for you.
>

As I wrote _MANY_ times in past mails.
I would have stopped discussion, if there are example codes that
justify the API.

Sorry for repeating requests, but this is technical list and no evidence
is provided.

If you really think my statements are wrong. Please comment
each line by replying "wrong" so that your idea becomes more clear.
I shouldn't difficult.

I provided more than handful valid use cases in my PHP RFC.
The technical evidence should not be difficult.
It's just example(s).

I'm waiting.

Regards,

-- 
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature and return value

[Dan Ackroyd]

On 6 September 2017 at 02:15, Yasuo Ohgaki  wrote:
> What should we do for this?

Not us, you.

You should start listening to other people's feedback.

You continually refuse to accept any feedback that doesn't agree with
your world-view, not only on the subject of hkdf, but on validation
and other things you think are "MANDATORY"

You should respect RFC votes and stop bringing up the same discussions
over and over again. This is incredibly tedious.

In particular your suggestion about hash_hkdf was rejected
unanimously, apart from your own vote
https://wiki.php.net/rfc/improve_hash_hkdf_parameter and so probably
shouldn't be brought up for discussion, except if you can bring new
facts for discussion.

"Not liking the result of the vote" is not a new fact to discuss.

Additionally though, your ideas about adding validation/filter
functions as a C extension, rather than implementing them in PHP have
also been resoundingly rejected,
https://wiki.php.net/rfc/add_validate_functions_to_filter and yet you
continue to promote the idea. This is also tedious.

This pattern of behaviour, continually refusing to accept that other
people have opinions that don't align with yours, and continually
bringing up the same topics for discussion over, and over, and over
again is not productive. It does not make people want to engage you in
discussion, as it is a waste of their time. This is not something
other people can fix for you.

cheers
Dan
Ack

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] hash_hkdf() signature and return value

[Yasuo Ohgaki]

Hi all,

This is the last recommendation for hash_hkdf[1]. In fact,
this would be the last chance to fix because we'll have 7.2 soon.
The issue is secure usage and API consistency.

Currently hash_hkdf() has following signature:

 hash_hkdf(string $algo , string $ikm [, int $length = 0 [,
string $info = '' [, string $salt = '' ]]] )

These are rationals behind recommendation. There are many, but
please read on.

=== Parameter Order ===

HKDF[2] algorithm is:
 1. General purpose key derivation function as per RFC 5869
 2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned
RFC 5869
 3. "$salt" (or preshared key) is very strongly recommended for security
season as per RFC 5869
 4. Supplying salt that the same length of input key does not affect
performance as per RFC 5969
 5. "$info" is what makes HKDF useful, but it's less important as described
in RFC 5869
 6. "$length" is truly an optional parameter for very specific encryption
algorithm or usage.

Rationale for change:
 1. Key derivations without secondary key ($salt) does not make sense when
 secondary key could be available. HKDF is designed for best possible
key
 security with the key. Not using secondary key ($salt) simply
downgrades
 key security without no reason. i.e. HKDF performance is the same
 when $salt has the same as hash is set.
 2. HKDF is based on HMAC. When $info has no use, HMAC would be the best
 choice for it. i.e. $newkey = hash_hmac($ikm, $key);
 3. It does not make sense violating RFC recommendations for a RFC
implementation.

>From these facts and reasons, $salt, $info and $length parameter order and
requirement should be changed from

string $algo , string $ikm [, int $length = 0 [, string $info = '' [,
string $salt = '' ]]]

to

string $algo , string $ikm , string $salt, string $info = '' [, int $length
= 0 ]
Note: Users can set empty string if they really don't need $salt and/or
$info.

Conclusion:
This way, users would have better chances to use hash_hkdf() more securely
and
properly.

[1] http://php.net/hash_hkdf
[2] http://www.faqs.org/rfcs/rfc5869.html

=== Return Value and Output Option ===

The most common HKDF usage with PHP would be:
 1. CSRF token generation that is specific to a request with expiration
time.
 (HEX return value would be appropriate, not BINARY)
 2. API access token generation that does not transmit "The API Key", but
 derived key by HKDF. It also should have expiration time.
 (HEX return value would be appropriate, not BINARY)

Consistency with other hash_*() functions:
 1. All of other hash_*()  returns HEX string hash value.
 2. hash_hkdf() is the only one returns BINARY hash value.

Conclusion:
hash_hkdf() should return HEX by default, not BINARY.
Optional [, bool $raw_output = false ] should be added just like other
hash_*().


=== Compatibility ===

IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0
in the first place. The mess could be resolved by 7.2.

Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are
using it yet. If we change API with 7.2 release, there would be least
possible confusions. (We may remove it from 7.1 to avoid confusions, too)

Our choices:
 - Keep the current insecure/inconsistent API forever.
 - Change the API to have secure/consistent API forever.

Conclusion:
No conclusion for this. There would be conflicting options.


I strongly think it is not worth to keep this insecure/inconsistent API
forever.
I prefer to change the API to what it should be.

What should we do for this?
Comments?



P.S.

Nikita, you've said following during HKDF discussion:
 - HKDF for CSRF tokens/etc does not make sense at all.
 - Current parameter order and return value makes perfect sense
   and has valid/common usages.
 - Everyone one this list, shouldn't listen to me because I'm insane and
   totally misunderstood what the HKDF is.

Phrases are not the exact, but it should be correct enough. Some part
is my fault, w/o reading mail and/or poor English. I blindly assumed
you've understand RFC 5869 and possible common usages with PHP. I
apologized for the confusion and tried to explain why w/o success.

If you still believe what you've said is correct, I don't request you to
take these back, show us at least one common/reasonable hash_hkdf()
usage example for current API, and point out what's wrong my recommendations
and rationales above.

If not, I request you to take it back. I respect your contributions much,
but
the last one you've said is out of tolerance.

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi Andrey,

On Thu, Feb 9, 2017 at 7:59 PM, Andrey Andreev  wrote:

>
>> I suppose most developers will use 'length' for shorter length.
>> i.e. Weaker output keys. If it's not too short, shorter key length works.
>>
>>
> Shows that you shouldn't be trusted with anything related to cryptography
> either.
> I may be no cryptographer myself, but one thing I know for sure is that
> shorter than required key lengths are *never* ok, and most encryption
> algorithms have a *fixed* key length.
>
> Also, this is the last time I reply to you on this topic. You're just
> impossible to reason with.
>

I think I finally understand what you don't understand.
Please read "Current Status" section of the draft PHP RFC.
https://wiki.php.net/rfc/improve_hash_hkdf_parameter#current_status

hash_hkdf() is simple hash_hmac() extension, why should not hash_hkdf() have
compatible signature with hash_hmac()? Aside from it, $salt is "key" in
many cases.
There is no reason "key" to be the last optional parameter.

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi Scott and all,

On Thu, Feb 9, 2017 at 10:52 PM, Scott Arciszewski 
wrote:

>
> ​
>> ​HKDF relies on PRK being cryptographically strong.
>> ​
>> ​
>
>
>
> ​Yes, b​ut not for the reasons you might suspect.
>
> ​The main use case of HKDF is to completely prevent related-key attacks,
> while splitting a single key into several for domain-specific purposes.​
> Figuring out a subkey (by sheer luck) should also not give an attacker
> enough information to recreate the IKM.
>
> HKDF isn't meant for key stretching (e.g. use an 80-bit secret in an
> application that requires a 256-bit secret). That's what a password hashing
> function is for. (Bcrypt, Argon2, etc.)
>
> ​
>> ​You should understand users _must_ make sure either "salt" or "IKM" is
>
> strong for HKDF to work. Since hash_hkdf() is generic function, you never
>
> can make sure IKM to be strong always.​
>
> ​​
>
> Not quite. IKM has to be strong. The salt? Not so much.
>
> In cryptography protocols, salts are considered public information (even
> if you don't go out of your way to publish it).
>
> If you have a weak IKM but a strong salt (32 bytes from random_bytes()),
> you should still consider the OKM weak.
> ​
>
>> Salt is the most important for both input and output key security.
>> Salt is mandatory and/or can be used for almost always with PHP.
>> Salt usage results in better design/security.
>> Salt is often used as final key as combined key.
>
>
>
> Don't get me wrong: Having secure random salts is useful (serves as a
> useful way to enlarge the effective nonce in a symmetric encryption
> protocol).
>
> But the cryptographic secret in this protocol is IKM. Not the salt.
>

I agree that your description matches many typical crypto
encryptions/authentication/etc.
As you would know, there are some exceptions that encryption key is not
really a random
(Not cryptographically strong) If user could use strong IKM, they should
use it, then
salt could be for additional security. (HKDF IKM could be weak, salt could
be secret, but
you mentioned typical use case and best practice in crypto)

The most typical HKDF application with PHP is access token key derivations
with timestamp
such as CSRF and Object Access token. For these applications, salt usage is
strongly
recommended, info(context) is recommended, length is OK with hash default.

e.g.
Generating CSRF token keys
$expire = time()+900; // Even, non secret, low entropy salt makes HKDF OKM
significantly stronger.
$csrf_token = bin2hex(hash_hkdf('sha256', $_SESSION['CSRF_TOKEN_SEED'], 0,
'', $expire));
then, send $expire and $csrf_token as access key.

This could be done with hash_hmac() already.
$csrf_token = hash_hmac('sha256', $_SESSION['CSRF_TOKEN_SEED'], $expire));

What's really good about HKDF is it can add additional non secret optional
context(domain) safely, as you mentioned.
// Make CSRF token only valid to 'Admin' features.
$csrf_token = bin2hex(hash_hkdf('sha256', $_SESSION['CSRF_TOKEN_SEED'], 0,
'Admin', $expire));
// Should be OK, but this is risky compare to above.
$csrf_token = hash_hmac('sha256', $_SESSION['CSRF_TOKEN_SEED'], $expire.
'-Admin'));

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Scott Arciszewski]

On Thu, Feb 9, 2017 at 7:51 AM, Yasuo Ohgaki  wrote:

> Hi Andrey,
>
> How the manual page would be. Would it be
>
>   "Even though 'slat' is the last optional parameter that users may omit
> easily
> by mistake, users _must_ set strong salt for weak $ikm. This is
> mandatory
> requirement for HKDF to work. In addition, it is advices to set salt
> whenever
> possible as RFC 5689 recommends"
>
> It can't make sense for new function.
>
> On Thu, Feb 9, 2017 at 7:59 PM, Andrey Andreev  wrote:
>
>>
>> The fact that you continue to talk about passwords and other low-entropy
>> data as IKM shows, yet again, that you don't understand HKDF.
>> It is simply not a password-based KDF; if you want that - use PBKDF2.
>> Please read Section 4* of the spec: https://tools.ietf.org/html/rf
>> c5869.html#section-4
>>
>
> PHP would not be a language that builds low level crypto library/feature.
> I'm not sure what you mean by this. What sentences you refer as I don't
> understand?
>
> I assumed you would like to derive human typable password because
> you prefer to have the most priority for "length". Do you have any typical
> "length" use with PHP?
>
> HKDF relies on PRK being cryptographically strong.
>
> Extract step
>PRK = HMAC-Hash(salt, IKM)
>
> You should understand users _must_ make sure either "salt" or "IKM" is
> strong for HKDF to work. Since hash_hkdf() is generic function, you never
> can make sure IKM to be strong always.
>
> With this fact alone, salt must have the most priority.
>
>
>> In fact, at one point I suggested the function be named just 'hkdf()' as
>> a way of discouraging such misuse, because a lot of people associate the
>> word "hash" with password handling.
>>
>
>> And this ...
>>
>>
>>> I suppose most developers will use 'length' for shorter length.
>>> i.e. Weaker output keys. If it's not too short, shorter key length
>>> works.
>>>
>>>
>> Shows that you shouldn't be trusted with anything related to cryptography
>> either.
>> I may be no cryptographer myself, but one thing I know for sure is that
>> shorter than required key lengths are *never* ok, and most encryption
>> algorithms have a *fixed* key length.
>>
>
> Are you assuming converting crypto keys to be other length would be
> the typical HKDF usage with PHP?
> Then, I have to say your assumption is wrong.
>
> It would be most used for CSRF key generation, object access
> control, e.g. AWS S3 presigned URL, or anything related to web
> app, not low level crypto.
>
> Discussion so far would not justify use of vulnerable, i.e. PRK could
> be weak very easily, signature.
>
> And you don't answer questions to you:
>
> 
> Why you recommends following usage?
> hash_hkdf('sha256', $weak_ikm, 9); // We can generate strong key easily,
> Nice! <= Must not do this.
> 
>
> 
> What's the reason why you trying to disregard RFC 5869 strongly recommends?
> 
>
> 
> Parameter order must reflect
>  - importance of the parameter
>  - likelihood to be used
>
> Salt is the most important for both input and output key security.
> Salt is mandatory and/or can be used for almost always with PHP.
> Salt usage results in better design/security.
> Salt is often used as final key as combined key.
>
> Does 'length' and/or 'info' achieve or designed for aboves? I don't think
> so.
> 
>
> 
> BTW, I don't think of typical PHP application that requires
> human typable short password with HKDF. What kind of
> application would be?
> 
>
>
>
> Regards,
>
> --
> Yasuo Ohgaki
> yohg...@ohgaki.net
>

​Hi,

​
> ​HKDF relies on PRK being cryptographically strong.
> ​
> ​



​Yes, b​ut not for the reasons you might suspect.

​The main use case of HKDF is to completely prevent related-key attacks,
while splitting a single key into several for domain-specific purposes.​
Figuring out a subkey (by sheer luck) should also not give an attacker
enough information to recreate the IKM.

HKDF isn't meant for key stretching (e.g. use an 80-bit secret in an
application that requires a 256-bit secret). That's what a password hashing
function is for. (Bcrypt, Argon2, etc.)

​
> ​You should understand users _must_ make sure either "salt" or "IKM" is

strong for HKDF to work. Since hash_hkdf() is generic function, you never

can make sure IKM to be strong always.​

​​

Not quite. IKM has to be strong. The salt? Not so much.

In cryptography protocols, salts are considered public information (even if
you don't go out of your way to publish it).

If you have a weak IKM but a strong salt (32 bytes from random_bytes()),
you should still consider the OKM weak.
​

> Salt is the most important for both input and output key security.
> Salt is mandatory and/or can be used for almost always with PHP.
> Salt usage results in better design/security.
> Salt is often used as final key as combined key.



Don't get me wrong: 

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi Andrey,

How the manual page would be. Would it be

  "Even though 'slat' is the last optional parameter that users may omit
easily
by mistake, users _must_ set strong salt for weak $ikm. This is
mandatory
requirement for HKDF to work. In addition, it is advices to set salt
whenever
possible as RFC 5689 recommends"

It can't make sense for new function.

On Thu, Feb 9, 2017 at 7:59 PM, Andrey Andreev  wrote:

>
> The fact that you continue to talk about passwords and other low-entropy
> data as IKM shows, yet again, that you don't understand HKDF.
> It is simply not a password-based KDF; if you want that - use PBKDF2.
> Please read Section 4* of the spec: https://tools.ietf.org/html/
> rfc5869.html#section-4
>

PHP would not be a language that builds low level crypto library/feature.
I'm not sure what you mean by this. What sentences you refer as I don't
understand?

I assumed you would like to derive human typable password because
you prefer to have the most priority for "length". Do you have any typical
"length" use with PHP?

HKDF relies on PRK being cryptographically strong.

Extract step
   PRK = HMAC-Hash(salt, IKM)

You should understand users _must_ make sure either "salt" or "IKM" is
strong for HKDF to work. Since hash_hkdf() is generic function, you never
can make sure IKM to be strong always.

With this fact alone, salt must have the most priority.


> In fact, at one point I suggested the function be named just 'hkdf()' as a
> way of discouraging such misuse, because a lot of people associate the word
> "hash" with password handling.
>

> And this ...
>
>
>> I suppose most developers will use 'length' for shorter length.
>> i.e. Weaker output keys. If it's not too short, shorter key length works.
>>
>>
> Shows that you shouldn't be trusted with anything related to cryptography
> either.
> I may be no cryptographer myself, but one thing I know for sure is that
> shorter than required key lengths are *never* ok, and most encryption
> algorithms have a *fixed* key length.
>

Are you assuming converting crypto keys to be other length would be
the typical HKDF usage with PHP?
Then, I have to say your assumption is wrong.

It would be most used for CSRF key generation, object access
control, e.g. AWS S3 presigned URL, or anything related to web
app, not low level crypto.

Discussion so far would not justify use of vulnerable, i.e. PRK could
be weak very easily, signature.

And you don't answer questions to you:


Why you recommends following usage?
hash_hkdf('sha256', $weak_ikm, 9); // We can generate strong key easily,
Nice! <= Must not do this.



What's the reason why you trying to disregard RFC 5869 strongly recommends?



Parameter order must reflect
 - importance of the parameter
 - likelihood to be used

Salt is the most important for both input and output key security.
Salt is mandatory and/or can be used for almost always with PHP.
Salt usage results in better design/security.
Salt is often used as final key as combined key.

Does 'length' and/or 'info' achieve or designed for aboves? I don't think
so.



BTW, I don't think of typical PHP application that requires
human typable short password with HKDF. What kind of
application would be?




Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Andrey Andreev]

Hi Yasuo,

The fact that you continue to talk about passwords and other low-entropy
data as IKM shows, yet again, that you don't understand HKDF.
It is simply not a password-based KDF; if you want that - use PBKDF2.
Please read Section 4* of the spec:
https://tools.ietf.org/html/rfc5869.html#section-4

In fact, at one point I suggested the function be named just 'hkdf()' as a
way of discouraging such misuse, because a lot of people associate the word
"hash" with password handling.

And this ...


> I suppose most developers will use 'length' for shorter length.
> i.e. Weaker output keys. If it's not too short, shorter key length works.
>
>
Shows that you shouldn't be trusted with anything related to cryptography
either.
I may be no cryptographer myself, but one thing I know for sure is that
shorter than required key lengths are *never* ok, and most encryption
algorithms have a *fixed* key length.

Also, this is the last time I reply to you on this topic. You're just
impossible to reason with.

Cheers,
Andrey.

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi all and Scott,

On Wed, Feb 8, 2017 at 11:22 PM, Scott Arciszewski 
wrote:

> On Wed, Feb 8, 2017 at 6:27 AM, Andrey Andreev  wrote:
>
>> Hi,
>>
>> On Tue, Feb 7, 2017 at 10:22 PM, Scott Arciszewski 
>> wrote:
>>
>>>
>>>
>>> Although the RFC itself says that salts are optional, the argument to
>>> make them required in PHP's implementation has merit. The only downside is:
>>> If you're integrating with an implementation that doesn't require salts,
>>> and the application doesn't use salts, you're out of luck. Is that enough
>>> of a downside to dismiss an argument for better security? Maybe.
>>>
>>>
>> I said this in another thread already, but I'll re-iterate here ... Using
>> a salt should be recommended, but the suggestion here takes that out of
>> context and misses a key point - that the Length and Info params are not of
>> any less importance overall.
>>
>> - Length is effectively the key size and is in fact not optional in
>> RFC5869*. You have to really know what you're doing if you don't use it,
>> and choose the hash function very carefully. I'd argue that if you know
>> this well enough, you'd also know what to do with Salt.
>> - Info is technically optional, but pretty much the feature why you'd
>> choose HKDF over other KDFs. It's what makes HKDF's more useful in that it
>> enables derivation of multiple OKMs from a single IKM. If you don't need
>> Info, chances are you don't need HKDF.
>>
>> One could use Salt for the purposes of Info, but that would be misusing
>> the algorithm.
>> Over-focusing on Section 3.1 of the specification, while at the same time
>> ignoring Section 3.2 - arbitrarily making OptionalParamX mandatory because
>> the note about it mentions the word "security". Noble, but misguided.
>>
>> I'd have no problem if ALL of the params were made non-optional - I'm all
>> for that, but either do that or leave it untouched.
>>
>> * RFC 5869: https://tools.ietf.org/html/rfc5869
>>
>> Cheers,
>> Andrey.
>>
>
>
> Given everything discussed in this sub-thread so far, ​I would be in favor
> of making everything non-optional.
>

I'm OK with making 'salt' and 'info' required. Users should
consider 'info'(key context) could be used almost always before
omitting it.

I suppose most developers will use 'length' for shorter length.
i.e. Weaker output keys. If it's not too short, shorter key length works.
I'm not sure if shorter output key length, that would weaken the key,
does any good in general .
Note: longer output key does not make output key stronger. It may
hide used hash function, though.

Most use cases would be AWS S3 pre-signed URL like
usage that users do not have to care long keys. With such
use cases, users are better to use key as is. i.e. Use default
hash output length.

There may be vote option for length, but I'm in favor of leave
it as optional. I do care about parameter order, but what to be
required is not too much concern to me.

Regards,

BTW, I don't think of typical PHP application that requires
human typable short password with HKDF. What kind of
application would be? Thank you.

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi Andrey,

On Wed, Feb 8, 2017 at 8:27 PM, Andrey Andreev  wrote:

> On Tue, Feb 7, 2017 at 10:22 PM, Scott Arciszewski 
> wrote:
>
>>
>>
>> Although the RFC itself says that salts are optional, the argument to
>> make them required in PHP's implementation has merit. The only downside is:
>> If you're integrating with an implementation that doesn't require salts,
>> and the application doesn't use salts, you're out of luck. Is that enough
>> of a downside to dismiss an argument for better security? Maybe.
>>
>>
> I said this in another thread already, but I'll re-iterate here ... Using
> a salt should be recommended, but the suggestion here takes that out of
> context and misses a key point - that the Length and Info params are not of
> any less importance overall.
>

>From the HKDF algorithm, importance is clear. Salt >>> info > length.

Salt is the most important to secure input key. You seem to think following
is the most important.

 - Salt could be optional _only_ when input key is cryptographically strong.

What this means is

 - Salt is absolute _mandatory_ parameter unless input key is
absolutely strong key.

- Length is effectively the key size and is in fact not optional in
> RFC5869*. You have to really know what you're doing if you don't use it,
> and choose the hash function very carefully. I'd argue that if you know
> this well enough,
>
- Info is technically optional, but pretty much the feature why you'd
> choose HKDF over other KDFs. It's what makes HKDF's more useful in that it
> enables derivation of multiple OKMs from a single IKM. If you don't need
> Info, chances are you don't need HKDF.
>

There is huge difference between "Required to be implemented" and
"Required/important as parameter".
Implementation requirement is _nothing_ to do with parameter
requirement/importance.

'salt' could be omitted the application that input key is always strong.
This cannot be applied to
generic API like hash_hkdf() that would be used by countless applications.

Why you recommends following usage?
hash_hkdf('sha256', $weak_ikm, 9); // We can generate strong key easily,
Nice! <= Must not do this.

One could use Salt for the purposes of Info, but that would be misusing the
> algorithm.
>

I totally agree.
HKDF is specific about 'salt' and 'info' task. Salt is entropy to make
input/output
key strong/stronger. Info is for output key context.

Over-focusing on Section 3.1 of the specification, while at the same time
> ignoring Section 3.2 - arbitrarily making OptionalParamX mandatory because
> the note about it mentions the word "security". Noble, but misguided.
>

What's the reason why you trying to disregard RFC 5869 strongly recommends?

   "designers of applications are therefore encouraged to provide salt
values to
   HKDF if such values can be obtained by the application."

It seems you're mixed up with "Implementation requirement" and "Parameter
requirement/importance".
After all, we are implementing RFC 5869, why we should not follow
recommendation?


> I'd have no problem if ALL of the params were made non-optional - I'm all
> for that, but either do that or leave it untouched.
>
> * RFC 5869: https://tools.ietf.org/html/rfc5869
>

While I don't mind requiring both 'salt' and 'info', I strongly disagree
with the current parameter order and 'length' is pure optional parameter.
'info' is important but optional. 'salt' can only be optional when input
key
is absolutely cryptographically strong.

Parameter order must reflect
 - importance of the parameter
 - likelihood to be used

Salt is the most important for both input and output key security.
Salt is mandatory and/or can be used for almost always with PHP.
Salt usage results in better design/security.
Salt is often used as final key as combined key.

Does 'length' and/or 'info' achieve or designed for aboves? I don't think
so.

hash_hkdf() is generic API. $ikm cannot be strong key always.
Assuming $ikm to be strong is simply wrong assumption.

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Scott Arciszewski]

On Wed, Feb 8, 2017 at 4:16 AM, Yasuo Ohgaki  wrote:

> Hi Scott,
>
> There are applications that do not require salt. In this case, all users
> has to do is
> $salt = NULL
> to omit $salt.
>


Great.

On Wed, Feb 8, 2017 at 6:27 AM, Andrey Andreev  wrote:

> Hi,
>
> On Tue, Feb 7, 2017 at 10:22 PM, Scott Arciszewski 
> wrote:
>
>>
>>
>> Although the RFC itself says that salts are optional, the argument to
>> make them required in PHP's implementation has merit. The only downside is:
>> If you're integrating with an implementation that doesn't require salts,
>> and the application doesn't use salts, you're out of luck. Is that enough
>> of a downside to dismiss an argument for better security? Maybe.
>>
>>
> I said this in another thread already, but I'll re-iterate here ... Using
> a salt should be recommended, but the suggestion here takes that out of
> context and misses a key point - that the Length and Info params are not of
> any less importance overall.
>
> - Length is effectively the key size and is in fact not optional in
> RFC5869*. You have to really know what you're doing if you don't use it,
> and choose the hash function very carefully. I'd argue that if you know
> this well enough, you'd also know what to do with Salt.
> - Info is technically optional, but pretty much the feature why you'd
> choose HKDF over other KDFs. It's what makes HKDF's more useful in that it
> enables derivation of multiple OKMs from a single IKM. If you don't need
> Info, chances are you don't need HKDF.
>
> One could use Salt for the purposes of Info, but that would be misusing
> the algorithm.
> Over-focusing on Section 3.1 of the specification, while at the same time
> ignoring Section 3.2 - arbitrarily making OptionalParamX mandatory because
> the note about it mentions the word "security". Noble, but misguided.
>
> I'd have no problem if ALL of the params were made non-optional - I'm all
> for that, but either do that or leave it untouched.
>
> * RFC 5869: https://tools.ietf.org/html/rfc5869
>
> Cheers,
> Andrey.
>


Given everything discussed in this sub-thread so far, ​I would be in favor
of making everything non-optional.

Scott Arciszewski
Chief Development Officer
Paragon Initiative Enterprises ​

Re: [PHP-DEV] hash_hkdf() signature

[Andrey Andreev]

Hi,

On Tue, Feb 7, 2017 at 10:22 PM, Scott Arciszewski 
wrote:

>
>
> Although the RFC itself says that salts are optional, the argument to make
> them required in PHP's implementation has merit. The only downside is: If
> you're integrating with an implementation that doesn't require salts, and
> the application doesn't use salts, you're out of luck. Is that enough of a
> downside to dismiss an argument for better security? Maybe.
>
>
I said this in another thread already, but I'll re-iterate here ... Using a
salt should be recommended, but the suggestion here takes that out of
context and misses a key point - that the Length and Info params are not of
any less importance overall.

- Length is effectively the key size and is in fact not optional in
RFC5869*. You have to really know what you're doing if you don't use it,
and choose the hash function very carefully. I'd argue that if you know
this well enough, you'd also know what to do with Salt.
- Info is technically optional, but pretty much the feature why you'd
choose HKDF over other KDFs. It's what makes HKDF's more useful in that it
enables derivation of multiple OKMs from a single IKM. If you don't need
Info, chances are you don't need HKDF.

One could use Salt for the purposes of Info, but that would be misusing the
algorithm.
Over-focusing on Section 3.1 of the specification, while at the same time
ignoring Section 3.2 - arbitrarily making OptionalParamX mandatory because
the note about it mentions the word "security". Noble, but misguided.

I'd have no problem if ALL of the params were made non-optional - I'm all
for that, but either do that or leave it untouched.

* RFC 5869: https://tools.ietf.org/html/rfc5869

Cheers,
Andrey.

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Forgot to mention the most important.

On Wed, Feb 8, 2017 at 6:16 PM, Yasuo Ohgaki  wrote:

> Is that enough of a downside to dismiss an argument for better security?
> Maybe


I suppose so.
It is very clear that "salt" is the most important for derived key security,
"info" is next, then "length".

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi Scott,

On Wed, Feb 8, 2017 at 5:22 AM, Scott Arciszewski 
wrote:

>
> One such real-world use case: Defuse v1 used HKDF without a salt.
>
> https://github.com/defuse/php-encryption/blob/
> b87737b2eec06b13f025cabea847338fa203d1b4/Crypto.php#L157-L170
> https://github.com/defuse/php-encryption/blob/
> b87737b2eec06b13f025cabea847338fa203d1b4/Crypto.php#L358
>
> In version 2, we included a 32-byte random salt for each encryption, which
> was stored next to the AES-256-CTR nonce in the ciphertext. (Both the nonce
> and HKDF-salt, as well as the version information header, are covered by
> the HMAC of the ciphertext.)
>
> The end result: Instead of having to worry about birthday collisions after
> you've seen 2^64 AES outputs (because 128-bit randomly generated nonce),
> now you need 2^192 before you have a useful collision.
>
> Although the RFC itself says that salts are optional, the argument to make
> them required in PHP's implementation has merit. The only downside is: If
> you're integrating with an implementation that doesn't require salts, and
> the application doesn't use salts, you're out of luck. Is that enough of a
> downside to dismiss an argument for better security? Maybe.
>

There are applications that do not require salt. In this case, all users
has to do is
$salt = NULL
to omit $salt.

Current signature has $length as the first optional parameter which would
be set
to "0" in many cases if not most. i.e. Just deriving new key from strong
$ikm would
not be typical.

Requiring "$salt = NULL" to omit salt would not hurt API use much compare
to
current signature that requires "$length = 0" for most cases, I suppose.

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] hash_hkdf() signature

[Tom Worster]

On 2/7/17 3:22 PM, Scott Arciszewski wrote:

One such real-world use case: Defuse v1 used HKDF without a salt.

https://github.com/defuse/php-encryption/blob/b87737b2eec06b13f025cabea847338fa203d1b4/Crypto.php#L157-L170
https://github.com/defuse/php-encryption/blob/b87737b2eec06b13f025cabea847338fa203d1b4/Crypto.php#L358

In version 2, we included a 32-byte random salt for each encryption, 
which
was stored next to the AES-256-CTR nonce in the ciphertext. (Both the 
nonce
and HKDF-salt, as well as the version information header, are covered 
by

the HMAC of the ciphertext.)

The end result: Instead of having to worry about birthday collisions 
after
you've seen 2^64 AES outputs (because 128-bit randomly generated 
nonce),

now you need 2^192 before you have a useful collision.


In this situation shouldn't you either use a longer random IKM or not 
use HKDF at all?


If your IKM is so weak that it needs a salt then shouldn't you use an 
iterated hash instead of HKDF?


Tom

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] hash_hkdf() signature

[Scott Arciszewski]

On Tue, Feb 7, 2017 at 2:35 PM, Yasuo Ohgaki  wrote:

> Hi Nikita, Andrey and all,



​​
>



Regards,
>
> P.S. I'll be more careful, but I become very sloppy mail reader sometimes.
> I appreciate if you could let know via private email. Thank you!
>
> --
> Yasuo Ohgaki
> yohg...@ohgaki.net
>

​Hi,​

Although there are HKDF usage without salt, many HKDF applications with PHP
> require or are better with salt. e.g. Previous per-session encryption.
> Developers
> will develop better application if they consider how salt could be used.
> Therefore,
> salt is better to be required parameter and omit it only when salt cannot

be used.​
>
>
​---​

One such real-world use case: Defuse v1 used HKDF without a salt.

https://github.com/defuse/php-encryption/blob/b87737b2eec06b13f025cabea847338fa203d1b4/Crypto.php#L157-L170
https://github.com/defuse/php-encryption/blob/b87737b2eec06b13f025cabea847338fa203d1b4/Crypto.php#L358

In version 2, we included a 32-byte random salt for each encryption, which
was stored next to the AES-256-CTR nonce in the ciphertext. (Both the nonce
and HKDF-salt, as well as the version information header, are covered by
the HMAC of the ciphertext.)

The end result: Instead of having to worry about birthday collisions after
you've seen 2^64 AES outputs (because 128-bit randomly generated nonce),
now you need 2^192 before you have a useful collision.

Although the RFC itself says that salts are optional, the argument to make
them required in PHP's implementation has merit. The only downside is: If
you're integrating with an implementation that doesn't require salts, and
the application doesn't use salts, you're out of luck. Is that enough of a
downside to dismiss an argument for better security? Maybe.

Scott Arciszewski
Chief Development Officer
Paragon Initiative Enterprises ​

[PHP-DEV] hash_hkdf() signature

[Yasuo Ohgaki]

Hi Nikita, Andrey and all,

My apologies, I misread mails by super sloppy reading.
I'll explain basis by my idea clearly and properly this time.
This mail is long.

Basis of my idea is
 - Salt is made to optional only for applications that such value is not
available. (From RFC 5869)
 - Omitting salt could lead to security disaster. i.e. password leak.
 - Combined key, output key and salt, as final key(combined key) is common
in many use cases.
 - Many HKDF applications with PHP must/should have salt for better
implementation.
 - API should encourage "salt" use by its signature. (From RFC 5869)

Ref: https://tools.ietf.org/html/rfc5869

Before restart discussion, there should be rationale for others.

In theory, cryptographic hashes are cryptographically secure. Therefore,
following operations should be considered as secure by definition.

$new_key = sha1('some original key' . 'strong salt' . 'some info');
$signature = sha1('some data' . 'strong key');

However, in real world, people come up with better idea for cryptographic
hashes. Sometimes people invent efficient way to attack cryptographic
hashes.

HMAC is known and proven method to generate more secure signature.
Therefore,
$signature = hash_hmac('sha1', 'some data', 'some key');
is secure even when cryptographic hash had minor defect(s).

HKDF is made to generate secure new keys from existing key suitable
for required operations by using HMAC.
HKDF inputs
 - IKM, input key which may be weak or strong
 - salt, some entropy which makes HKDF stronger overall, may be secret or
non secret/weak or strong.
 - info, which specifies HKDF contexts that are non secret usually. e.g. a
protocol number, algorithm identifiers, user identities, etc.
 - length(L), output key length

HKDF calculates output key(OKM) as follows

Extract step
   PRK = HMAC-Hash(salt, IKM)

This step is designed to make strong output key(OKM) and PRK always.
OKM to be secure, either IKM or salt must be strong.

Expand step
   N = ceil(L/HashLen)
   T = T(1) | T(2) | T(3) | ... | T(N)
   OKM = first L octets of T

   where:
   T(0) = empty string (zero length)
   T(1) = HMAC-Hash(PRK, T(0) | info | 0x01)
   T(2) = HMAC-Hash(PRK, T(1) | info | 0x02)
   T(3) = HMAC-Hash(PRK, T(2) | info | 0x03)

Note: OKM is output key material that is return value from HKDF.

This step is designed to make derived key have the length(L) from
strong key(PRK) generated by Extract step. Key context(info) is
distinguished by this step also.


Both salt and info is optional, but RFC 5869 states differently.

  "HKDF is defined to operate with and without random salt.  This is
   done to accommodate applications where a salt value is not available.
   We stress, however, that the use of salt adds significantly to the
   strength of HKDF, ensuring independence between different uses of the
   hash function, supporting "source-independent" extraction, and
   strengthening the analytical results that back the HKDF design."

This statement implies salt is almost mandatory parameter for HKDF
when salt can be used. In contrast, info is described as pure optional
parameter for key context.

  "While the 'info' value is optional in the definition of HKDF, it is
   often of great importance in applications.  Its main objective is to
   bind the derived key material to application- and context-specific
   information.  For example, 'info' may contain a protocol number,
   algorithm identifiers, user identities, etc."

With regard to mandatoriness parameters, strong salt is mandatory to derive
cryptographically strong key when input key is weak, while info/length is
optional always.

In addition, it is common that salt being used as a part of combined keys.
Salt is mandatory for such applications. There are many authentication
implementations that use
 - key which does not disclose original key by hashing
 - nonce(salt)

Access permission with timeout is another typical usage that requires
timestamp as salt and combined key.

HKDF can produce secure key, which protects input key(IKM) and
generates strong output key (OKM), but this is true only when
IKM or salt is strong. Use of weak IKM and salt could lead password leak.


For above reasons, I'm proposing change
   string hash_hkdf(string $hash, string $ikm [, int $length=0 [, string
$info='' [, string $salt='']]])
to
   string hash_hkdf(string $hash, string $ikm, string $salt [, string
$info='' [, int $length=0 ]])
 - To omit salt, $salt=NULL. $salt='' raise exception.


On Mon, Jan 16, 2017 at 8:08 PM, Nikita Popov  wrote:

> Making the salt required makes no sense to me.
>
> HKDF has a number of different applications:
> a) Derive multiple strong keys from strong keying material. Typical case
> for this is deriving independent encryption and authentication keys from a
> master key. This requires only specification of $length. A salt is neither
> necessary nor useful in this case, because you start with strong
> cryptographic keying material.
>