Vote results: remove -crypt option from passwd and C source output options

2020-11-11 Thread Dr Paul Dale
Vote: Drop from 3.0 support for the '-crypt' option from the passwd application

For: 6, Against: 0, Abstain: 0, Didn’t vote: 1

The vote passes.


Vote: Drop from 3.0 support for the C code output options from the applications 
where the use of deprecated APIs is required

For: 4, Against: 0, Abstain: 2, Didn’t vote: 1

The vote passes.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: [OTC VOTE PROPOSAL] Approve behavior change for `pkey -[pub]check`

2020-11-11 Thread Dr Paul Dale
An OMC vote deeming that adding error checks like this are or are not 
considered breaking changes.

My view is that detecting an error condition and returning an error code is a 
bug fix rather than a breaking change.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 11 Nov 2020, at 7:10 pm, Matt Caswell  wrote:
> 
> I have no problem  with the proposal or the vote text. I only wonder
> whether, as a breaking change an OMC vote is required? Or is an OTC vote
> sufficient?
> 
> Matt
> 
> 
> On 10/11/2020 16:15, Nicola Tuveri wrote:
>> ## Background
>> 
>> As part of the discussion in [issue #8435], it was highlighted that both
>> in 1.1.1 and master we are missing tests to validate decoded SM2 keys.
>> The `openssl pkey -check` (or `pkey -pubcheck` to validate only the
>> public key component) command allows to explicitly execute the
>> validation checks, while on regular operations (e.g., using `pkeyutl` or
>> `dgst`) no validation of the input keys is normally done (probably by
>> design).
>> 
>> In [PR 13359] I am adding new tests, using `openssl pkey -check` to
>> validate specific key corner cases, for P-256 as an exemplar for EC keys
>> and for SM2 keys.
>> Unfortunately `openssl pkey -check` behavior currently shows the result
>> of the validation only in the text output, so parsing of `stdout` would
>> require measuring the test results.
>> In the PR I am proposing to change the behavior of `openssl pkey` so
>> that an early exit is triggered if `-check` or `-pubcheck` validation
>> fails, exiting with a failure exit status: [apps/pkey.c changes].
>> 
>> This is a breaking change in the behavior of `openssl pkey` and the
>> reason why I am planning to raise a vote to approve this change.
>> 
>> Note that during our OTC meeting today it was proposed as an alternative
>> to have a more generic vote on approving this kind of change in general
>> for all similar situations across all the apps.
>> While I am not opposed to such a decision, I am afraid having such a
>> generic vote might be quite difficult:
>> 
>> - I am not sure I can express in a clear and unambiguous what are the
>>  conditions to fall within "similar situations", making such a
>>  decision hard to execute in practical terms;
>> - I personally cannot commit to execute such vote across all apps and
>>  all relevant conditions: doing so requires extensive review of the
>>  apps logic in parsing the CLI switches, of conformity with existing
>>  documentation and an exploration of existing use patterns in the user
>>  codebase to see what are the expectations and estimate the impact of
>>  such changes. It would also require writing an extensive battery of
>>  tests to ensure we behave as documented/expected across apps and CLI
>>  switches.
>> - The amount of work to which we would commit after such a generic
>>  decision, and the impact it could have on the behavior consistency
>>  w.r.t. 3.0 and 1.1.1, would make me think such a decision is more in
>>  the realm of strategic objectives, for which an OMC vote would be more
>>  appropriate.
>> 
>> For these reasons, at this time, I am opting to propose an OTC vote on
>> the single case of the behavior change of `pkey -check`/`pkey -pubcheck`
>> rather than a more generic one, and I will let others decide if a more
>> generic vote is also required/appropriate and if it can be framed
>> exclusively within the technical prerogatives of the OTC decision
>> process.
>> 
>> ## Proposed vote text
>> 
>>Change the behavior of `pkey -check`
>>and `pkey -pubcheck` in 3.0 to trigger an
>>early exit if validation fails, returning a
>>failure exit status to the parent process.
>> 
>> ---
>> 
>> Please leave feedback relevant to the proposed vote text and the scope
>> of the vote.
>> In absence of feedback I plan to open the actual vote in 24h.
>> 
>> 
>> 
>> Cheers,
>> 
>> Nicola
>> 
>> ---
>> 
>> [issue #8435]: 
>> <https://urldefense.com/v3/__https://github.com/openssl/openssl/issues/8435__;!!GqivPVa7Brio!Iiq8eSK65cKttwL_M-jL1JwQw-V9roGNZaxYAPw08zcLEiuEMvGtYPDvrBSYf2E$
>>  
>> <https://urldefense.com/v3/__https://github.com/openssl/openssl/issues/8435__;!!GqivPVa7Brio!Iiq8eSK65cKttwL_M-jL1JwQw-V9roGNZaxYAPw08zcLEiuEMvGtYPDvrBSYf2E$>
>>  >
>> [PR 13359]: 
>> <https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/13359__;!!GqivPVa7Brio!Iiq8eSK65cKttwL_M-jL1Jw

Proposed OMC by law vote

2020-11-05 Thread Dr Paul Dale
Proposed vote text: Accept the by law changes proposed in #207.

Comment: https://github.com/openssl/web/pull/207 
<https://github.com/openssl/web/pull/207>


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Proposed OMC vote to remove C source output from apps

2020-11-03 Thread Dr Paul Dale
Proposed vote text:

Remove -C option from the dhparam, dsaparam, ecparam and x509 apps.

This is a breaking change and requires OMC approval.  The rationale is that it 
is easier to not support this using the EVP calls and there is some doubt that 
the options are used anymore.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Proposed OMC vote: drop -crypt option from passwd app

2020-11-03 Thread Dr Paul Dale
Proposed vote text:

Remove the -crypt option from the passwd app.

The rationale behind this is that this is a very old, long broken algorithm and 
that supporting it is difficult using non-deprecated calls.  This is a breaking 
change and requires OMC approval.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Project direction

2020-11-01 Thread Dr Paul Dale
This came in privately from Michael Richardson and is forwarded with permission:

> If you don't grow the user base, you decline.
> As long as the old APIs do not go away too quickly, or the new APIs are
> obvious refactors, then it won't be such a problem.
> 
> In general, I think that there are way too many APIs, and the documentation
> is poorly organized by what it does rather than what things it does it to
> (more OOP).
> I tried to collect notes at one point, I had other priorities.
> 
> And I still have a pull request which is waiting for me to validate that it
> works on VAX/VMS.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 30 Oct 2020, at 9:43 am, Dr Paul Dale  wrote:
> 
> At the OTC call on Tuesday Tim raise a point about the future direction of 
> the project.  I was tasked with bringing this to the OMC for consideration.
> 
> The question was should we design our APIs to ease the pain existing users of 
> OpenSSL or should we be trying to attract new users.
> The idea being that supporting existing users means not changing the existing 
> API, whereas catering to new users means working towards a new fresh 
> consistent API.
> 
> This is all in the context of function naming, argument ordering, cleanup for 
> beta 1.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 



Project direction

2020-10-29 Thread Dr Paul Dale
At the OTC call on Tuesday Tim raise a point about the future direction of the 
project.  I was tasked with bringing this to the OMC for consideration.

The question was should we design our APIs to ease the pain existing users of 
OpenSSL or should we be trying to attract new users.
The idea being that supporting existing users means not changing the existing 
API, whereas catering to new users means working towards a new fresh consistent 
API.

This is all in the context of function naming, argument ordering, cleanup for 
beta 1.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Hacktoberfest

2020-10-20 Thread Dr Paul Dale
There has been a request in #13172 
<https://github.com/openssl/openssl/pull/13172> to tag the PR for Hacktoberfest 
<https://hacktoberfest.digitalocean.com/>.
There are two ways 
<https://hacktoberfest.digitalocean.com/hacktoberfest-update> to do this: add a 
tag to the PR or a topic to the project.

Is either something the project is interested in doing?

Rather than polluting our already busy tags menu, the topic seems the easier 
path to me.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: LTS+

2020-10-19 Thread Dr Paul Dale
Unless the change can be argued to be security hardening — an improved entropy 
source would be IMO.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 20 Oct 2020, at 9:10 am, Dr Paul Dale  wrote:
> 
> Not with the wording used.  The feature exists even if it’s rubbish.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 
>> On 20 Oct 2020, at 5:07 am, Tomas Mraz > <mailto:t...@t8m.info>> wrote:
>> 
>> I wonder if something like adding a new entropy source on an existing 
>> platform especially on some that currently don't have adequate ones would be 
>> covered by this.
>> 
>> ⁣Tomáš​
>> 
>> 19. 10. 2020 18:29, 18:29, Matt Caswell > <mailto:m...@openssl.org>> napsal/a:
>>> 
>>> 
>>> On 19/10/2020 15:14, Matt Caswell wrote:
>>>> LTS+ releases may contain the following new features:
>>>> - Support for additional platforms
>>>> - Performance improvements
>>> 
>>> Based on the discussion in PR #13176, I'd like to add to this list:
>>> 
>>> - Extending existing features to existing platforms where that feature
>>> does not yet exist
>>> 
>>> Thoughts?
>>> 
>>> Matt
>> 
> 



Re: LTS+

2020-10-19 Thread Dr Paul Dale
Not with the wording used.  The feature exists even if it’s rubbish.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 20 Oct 2020, at 5:07 am, Tomas Mraz  wrote:
> 
> I wonder if something like adding a new entropy source on an existing 
> platform especially on some that currently don't have adequate ones would be 
> covered by this.
> 
> ⁣Tomáš​
> 
> 19. 10. 2020 18:29, 18:29, Matt Caswell  napsal/a:
>> 
>> 
>> On 19/10/2020 15:14, Matt Caswell wrote:
>>> LTS+ releases may contain the following new features:
>>> - Support for additional platforms
>>> - Performance improvements
>> 
>> Based on the discussion in PR #13176, I'd like to add to this list:
>> 
>> - Extending existing features to existing platforms where that feature
>> does not yet exist
>> 
>> Thoughts?
>> 
>> Matt
> 



Re: OTC VOTE: The PR #11359 (Allow to continue with further checks on UNABLE_TO_VERIFY_LEAF_SIGNATURE) is acceptable for 1.1.1 branch

2020-10-09 Thread Dr Paul Dale
+1

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 9 Oct 2020, at 10:02 pm, Tomas Mraz  wrote:
> 
> topic: The PR #11359 (Allow to continue with further checks on
> UNABLE_TO_VERIFY_LEAF_SIGNATURE) is acceptable for 1.1.1 branch
> As the change is borderline on bug fix/behaviour change OTC needs
> to decide whether it is acceptable for 1.1.1 branch.
> Proposed by Tomas Mraz
> Public: yes
> opened: 2020-10-09
> closed: 2020-mm-dd
> accepted:  yes/no  (for: X, against: Y, abstained: Z, not voted: T)
> 
>  Matt   [  ]
>  Mark   [  ]
>  Pauli  [  ]
>  Viktor [  ]
>  Tim[  ]
>  Richard[  ]
>  Shane  [  ]
>  Tomas  [+1]
>  Kurt   [  ]
>  Matthias   [  ]
>  Nicola [  ]
> 
> -- 
> Tomáš Mráz
> No matter how far down the wrong road you've gone, turn back.
>  Turkish proverb
> [You'll know whether the road is wrong if you carefully listen to your
> conscience.]
> 
> 



Re: VOTE: Weekly OTC meetings until 3.0 beta1 is released

2020-10-09 Thread Dr Paul Dale
Nowhere has it been said that the weekly meeting will be 3 hours.
The existing 1.5 - 2 hour slot should be enough, although perhaps not for a few 
more weeks.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 10 Oct 2020, at 7:29 am, SHANE LONTIS  wrote:
> 
> [+1]  (reluctantly committing to 3 hour meetings :))
> 
> 
>> On 9 Oct 2020, at 10:00 pm, Nicola Tuveri > <mailto:nic@gmail.com>> wrote:
>> 
>> topic: Hold online weekly OTC meetings starting on Tuesday 2020-10-13
>>   and until 3.0 beta1 is released, in lieu of the weekly "developer
>>   meetings".
>> Proposed by Nicola Tuveri
>> Public: yes
>> opened: 2020-10-09
>> closed: 2020-mm-dd
>> accepted:  yes/no  (for: X, against: Y, abstained: Z, not voted: T)
>> 
>>  Matt   [  ]
>>  Mark   [  ]
>>  Pauli  [  ]
>>  Viktor [  ]
>>  Tim[  ]
>>  Richard[  ]
>>  Shane  [  ]
>>  Tomas  [  ]
>>  Kurt   [  ]
>>  Matthias   [  ]
>>  Nicola [+1]
> 



Re: VOTE: Weekly OTC meetings until 3.0 beta1 is released

2020-10-09 Thread Dr Paul Dale
+1

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 9 Oct 2020, at 10:00 pm, Nicola Tuveri  wrote:
> 
> topic: Hold online weekly OTC meetings starting on Tuesday 2020-10-13
>   and until 3.0 beta1 is released, in lieu of the weekly "developer
>   meetings".
> Proposed by Nicola Tuveri
> Public: yes
> opened: 2020-10-09
> closed: 2020-mm-dd
> accepted:  yes/no  (for: X, against: Y, abstained: Z, not voted: T)
> 
>  Matt   [  ]
>  Mark   [  ]
>  Pauli  [  ]
>  Viktor [  ]
>  Tim[  ]
>  Richard[  ]
>  Shane  [  ]
>  Tomas  [  ]
>  Kurt   [  ]
>  Matthias   [  ]
>  Nicola [+1]



Re: VOTE: Technical Items still to be done

2020-10-08 Thread Dr Paul Dale
[to the project list this time]

+1

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 9 Oct 2020, at 12:47 am, Matt Caswell  wrote:
> 
> topic: The following items are required prerequisites for the first beta
> release:
> 1) EVP is the recommended API, it must be feature-complete compared with
>the functionality available using lower-level APIs.
>   - Anything that isn’t available must be put to an OTC vote to exclude.
>   - The apps are the minimum bar for this, subject to exceptions noted
> below.
> 2) Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_,
>RAND_METHOD_.
>   - Does not include macros defining useful constants (e.g.
> SHA512_DIGEST_LENGTH).
>   - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`.
>   - There might be some others.
>   - Review for exceptions.
>   - The apps are the minimum bar to measure feature completeness for
> the EVP
> interface: rewrite them so they do not use internal nor deprecated
> functions (except speed, engine, list, passwd -crypt and the code
> to handle
> the -engine CLI option).  That is, remove the suppression of deprecated
> define.
> - Proposal: drop passwd -crypt (OMC vote required)
>   - Compile and link 1.1.1 command line app against the master headers and
> library.  Run 1.1.1 app test cases against the chimera.  Treat this
> as an
> external test using a special 1.1.1 branch. Deprecated functions
> used by
> libssl should be moved to independent file(s), to limit the
> suppression of
> deprecated defines to the absolute minimum scope.
> 3) Draft documentation (contents but not pretty)
>   - Need a list of things we know are not present - including things we
> have
> removed.
>   - We need to have mapping tables for various d2i/i2d functions.
>   - We need to have a mapping table from “old names” for things into the
> OSSL_PARAMS names.
> - Documentation addition to old APIs to refer to new ones (man7).
> - Documentation needs to reference name mapping.
> - All the legacy interfaces need to have their documentation
> pointing to
>   the replacement interfaces.
> 4) Review (and maybe clean up) legacy bridge code.
> 5) Review TODO(3.0) items #12224.
> 6) Source checksum script.
> 7) Review of functions previously named _with_libctx.
> 8) Encoder fixes (PKCS#8, PKCS#1, etc).
> 9) Encoder DER to PEM refactor.
> 10) Builds and passes tests on all primary, secondary and FIPS platforms.
> 11) Query provider parameters (name, version, ...) from the command line.
> 12) Setup buildbot infrastructure and associated instructions.
> 13) Complete make fipsinstall.
> 14) More specific decoding selection (e.g. params or keys).
> 15) Example code covering replacements for deprecated APIs.
> 16) Drop C code output options from the apps (OMC approval required).
> 17) Address issues and PRs in the 3.0beta1 milestone.
> Proposed by .
> Public: yes
> opened: 2020-10-08
> closed: 2020-mm-dd
> accepted:  yes/no  (for: X, against: Y, abstained: Z, not voted: T)
> 
>  Matt   [+1]
>  Mark   [  ]
>  Pauli  [  ]
>  Viktor [  ]
>  Tim[  ]
>  Richard[  ]
>  Shane  [  ]
>  Tomas  [  ]
>  Kurt   [  ]
>  Matthias   [  ]
>  Nicola [  ]



Re: VOTE: Accept the Fully Pluggable TLSv1.3 KEM functionality

2020-10-08 Thread Dr Paul Dale
+1

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 9 Oct 2020, at 12:27 am, Matt Caswell  wrote:
> 
> topic: We should accept the Fully Pluggable TLSv1.3 KEM functionality as
> shown in PR #13018 into the 3.0 release
> Proposed by Matt Caswell
> Public: yes
> opened: 2020-10-08
> closed: 2020-mm-dd
> accepted:  yes/no  (for: X, against: Y, abstained: Z, not voted: T)
> 
>  Matt   [+1]
>  Mark   [  ]
>  Pauli  [  ]
>  Viktor [  ]
>  Tim[  ]
>  Richard[  ]
>  Shane  [  ]
>  Tomas  [  ]
>  Kurt   [  ]
>  Matthias   [  ]
>  Nicola [  ]



Re: Vote proposal: Private keys can exist independently of public keys

2020-10-07 Thread Dr Paul Dale
Would it be feasible to change code that does ->pub_key to call a function that 
null checks the field and generates the public key if it is absent?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 7 Oct 2020, at 9:29 pm, Matt Caswell  wrote:
> 
> Issue #12612 exposes a problem with how we handle keys that contain
> private components but not public components.
> 
> There is a widespread assumption in the code that keys with private
> components must have public components. There is text in our public
> documentation that states this (and that text dates back to 2006).
> 
> OTOH, the code has not always enforced this. Issue #12612 describes a
> scenario where this has not historically been enforced, and it now is in
> the current 3.0 code causing a regression.
> 
> There are differences of opinion on how this should be handled. Some
> have the opinion that we should change the model so that we explicitly
> allow private keys to exists without the public components. Others feel
> that we should continue with the old model.
> 
> It seems we need a vote to decide this. Here is my proposed vote text:
> 
> We should change the 3.0 code to explicitly allow private components to
> exist in keys without the public components also being present.
> 
> Feedback please on the proposed vote text.
> 
> Matt



Would this be interesting to the project?

2020-10-01 Thread Dr Paul Dale
https://github.blog/2020-09-30-code-scanning-is-now-available/ 
<https://github.blog/2020-09-30-code-scanning-is-now-available/>

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Memory leak in openssl 1.1.1d

2020-09-30 Thread Dr Paul Dale
This isn’t enough information to diagnose the issue.  Which of the leak summary 
records is the problem?
Are you sure that your application is cleaning up properly (hint: it isn’t, 
e.g. OpenSSL never calls operator new() from the second record).

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 10 Sep 2020, at 1:39 am, Abhi Arora  wrote:
> 
> I am using openssl 1.1.1d. I found out around 228 bytes are being directly 
> lost (as per valgrind) report. I have one application which uses curl (7.64) 
> and I wrote the same application using POCO HTTPS and I got the same result.
> 
> I thought it could be related to the cipher suit. I can see the leak is 
> dependent on the URL being used (all were https). For one of the urls, it was 
> leaking 228 (directly) per connection and disconnection. For one of the urls, 
> the same code doesn't leak atall (no matter how many times you run that code 
> in a for loop).
> 
> For the urls which were leaking, the following cipher suites were being used:
> 1. SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> 2. SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> 
> Here is the valgrind summary:
> ==8620== HEAP SUMMARY:
> ==8620== in use at exit: 121,777 bytes in 1,097 blocks
> ==8620==   total heap usage: 129,382 allocs, 128,285 frees, 10,635,842 bytes 
> allocated
> ==8620==
> ==8620== 8 bytes in 1 blocks are possibly lost in loss record 16 of 372
> ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307)
> ==8620==by 0x4E75523: ??? (in /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 8 bytes in 1 blocks are definitely lost in loss record 17 of 372
> ==8620==at 0x483F310: operator new(unsigned int) (vg_replace_malloc.c:336)
> ==8620==by 0x36D15: main (sample_pki.cpp:27)
> ==8620==
> ==8620== 8 bytes in 1 blocks are definitely lost in loss record 18 of 372
> ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307)
> ==8620==by 0x4CBF14B: CRYPTO_zalloc (mem.c:230)
> ==8620==by 0x4C76E07: ECDSA_SIG_new (ec_asn1.c:1214)
> ==8620==by 0x537064B: pkcs11_try_pkey_ec_sign (p11_pkey.c:546)
> ==8620==by 0x537064B: pkcs11_pkey_ec_sign (p11_pkey.c:634)
> ==8620==by 0x4CB70CD: EVP_DigestSignFinal (m_sigver.c:148)
> ==8620==by 0x4BA9FF3: tls_construct_cert_verify (statem_lib.c:307)
> ==8620==by 0x4BA42C7: write_state_machine (statem.c:843)
> ==8620==by 0x4BA42C7: state_machine (statem.c:443)
> ==8620==by 0x4B8BABB: ssl3_read_bytes (rec_layer_s3.c:1278)
> ==8620==by 0x4B900E1: ssl3_read_internal (s3_lib.c:4473)
> ==8620==by 0x4B9013F: ssl3_read (s3_lib.c:4497)
> ==8620==by 0x4B96A6D: ssl_read_internal (ssl_lib.c:1761)
> ==8620==by 0x4B96B09: SSL_read (ssl_lib.c:1775)
> ==8620==
> ==8620== 16 bytes in 2 blocks are possibly lost in loss record 55 of 372
> ==8620==at 0x483EA30: malloc (vg_replace_malloc.c:306)
> ==8620==by 0x48419FF: realloc (vg_replace_malloc.c:834)
> ==8620==by 0x4E73B5D: amqpvalue_set_map_value (in 
> /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 40 bytes in 10 blocks are possibly lost in loss record 182 of 372
> ==8620==at 0x483EA30: malloc (vg_replace_malloc.c:306)
> ==8620==by 0x48419FF: realloc (vg_replace_malloc.c:834)
> ==8620==by 0x4E738BB: amqpvalue_set_list_item (in 
> /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 48 bytes in 1 blocks are possibly lost in loss record 210 of 372
> ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307)
> ==8620==by 0x4E7582D: ??? (in /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 48 bytes in 2 blocks are possibly lost in loss record 211 of 372
> ==8620==at 0x48419B0: realloc (vg_replace_malloc.c:834)
> ==8620==by 0x4E73B5D: amqpvalue_set_map_value (in 
> /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 56 bytes in 1 blocks are possibly lost in loss record 215 of 372
> ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307)
> ==8620==by 0x4E74FF9: ??? (in /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 64 bytes in 2 blocks are possibly lost in loss record 228 of 372
> ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307)
> ==8620==by 0x4E74477: ??? (in /usr/lib/libuamqp.so.1.2.12)
> ==8620==
> ==8620== 64 bytes in 1 blocks are definitely lost in loss record 229 of 372
> ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307)
> ==8620==by 0x4A8D411: __libc_alloc_buffer_allocate 
> (alloc_buffer_allocate.c:26)
> ==8620==by 0x4AE2B51: alloc_buffer_allocate (alloc_buffer.h:143)
> ==8620==by 0x4AE2B51: __resolv_conf_allocate (resolv_conf.c:411)
> ==8620==by 0x4AE147D: __resolv_conf_load (res_init.c:592)
> ==8620==by 0x4AE

Re: Integration of new algorithms

2020-09-30 Thread Dr Paul Dale
Instead of using an engine, you should write a provider (assuming you’re using 
the soon to be released OpenSSL 3.0).  It doesn’t need a NID.

If you are using OpenSSL 1.1.1, try the OBJ_new_nid() function.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 26 Aug 2020, at 6:48 pm, Kris Kwiatkowski  wrote:
> 
> 
> Hey,
> 
> I'm working on development of OpenSSL ENGINE that integrates
> post-quantum algorithms (new NIDs). During integration I
> need to modify OpenSSL code to add custom function, but would
> prefer not to need add anything to OpenSSL code (so engine
> can be dynmicaly loaded by any modern OpenSSL).
> 
> So, In three cases, namely when the code is in callbacks for keygen,
> encryption and ctrl (called by EVP_PKEY_CTX_ctrl, EVP_PKEY_encrypt 
> and EVP_PKEY_keygen) I need to get NID of the scheme. The problem
> is that, those functions are called with EVP_PKEY_CTX object
> provided as an argument. The NID is stored in the 
> EVP_PKEY_CTX->pmeth->pkey_id. I think (AFAIK) there is no API
> which would return that value.
> 
> I've added a simple function that returns pkey_id from the ctx, but
> that means that I need to change OpenSSL code. Is there any way
> to get NID without changing OpenSSL?
> 
> Kind regards,
> Kris
> 
> 
> 
> 



Re: VOTE: Accept the OTC voting policy as defined:

2020-09-28 Thread Dr Paul Dale
+1

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 28 Sep 2020, at 10:02 pm, Dr. Matthias St. Pierre 
>  wrote:
> 
> topic: Accept the OTC voting policy as defined:
> 
>   The proposer of a vote is ultimately responsible for updating the 
> votes.txt
>   file in the repository.  Outside of a face to face meeting, voters MUST 
> reply
>   to the vote email indicating their preference and optionally their 
> reasoning.
>   Voters MAY update the votes.txt file in addition.
> 
>   The proposed vote text SHOULD be raised for discussion before calling 
> the vote.
> 
>   Public votes MUST be called on the project list, not the OTC list and 
> the
>   subject MUST begin with "VOTE:".  Private votes MUST be called on the
>   OTC list with "PRIVATE VOTE:" beginning subject.
> 
> Proposed by Matthias St. Pierre (on behalf of the OTC)
> Public: yes
> opened: 2020-09-28
> closed: 2020-mm-dd
> accepted:  yes/no  (for: X, against: Y, abstained: Z, not voted: T)
> 
>  Matt   [  ]
>  Mark   [  ]
>  Pauli  [  ]
>  Viktor [  ]
>  Tim[  ]
>  Richard[  ]
>  Shane  [  ]
>  Tomas  [  ]
>  Kurt   [  ]
>  Matthias   [+1]
>  Nicola [  ]
> 



Freeze?

2020-09-23 Thread Dr Paul Dale
I’m seeing quite a bit of activity going on which isn’t related to the 3.0beta1 
milestone.
We’re well past the cutoff date announced for new features in the code.

Should we be limiting the “new” stuff going in?

I’m fine with bug fixes, they make sense.  I’m fine with the list of beta1 pull 
requests continuing.
It’s the rest that is more concerning.

Does anyone else have a similar view?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: New GitHub label for release blockers

2020-09-13 Thread Dr Paul Dale
> BTW: It took me all my force of will to resist the temptation of making a pun 
> by naming the label [urgent: beta blocker].

Failed you have.  Your training is not yet compete :)

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




Re: Beta1 PR deadline

2020-09-10 Thread Dr Paul Dale
Shane has done the IG D.9 work: it is in #12835 and is awaiting review.  I 
think it should be in beta if possible.  It’s mostly adding a second test case 
due to NIST changing guidance recently, so low risk and helpful for the lab.

As Shane notes in 12801, there will be some conflicts generated over the FIPS 
error handling and self tests, hopefully they won’t slow progress.

12754 could be considered optional and something better should likely be done 
for 3.1 that what I’m planning there.  With RAND_DRBG removed, there is nothing 
remotely comparable.  Even RAND_DRBG is a bit of a stretch as a comparison.


The API renaming discussion *must* reach a conclusion before beta.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 10 Sep 2020, at 8:40 pm, Matt Caswell  wrote:
> 
> 
> 
> On 09/09/2020 13:03, Kurt Roeckx wrote:
>> On Wed, Aug 26, 2020 at 04:58:26PM +0100, Matt Caswell wrote:
>>> Please can anyone with PRs that they wish to have included in OpenSSL
>>> 3.0 beta1 ensure that they are merged to master by 8th September.
>> 
>> So that date has passed now. Can someone give an overview of what
>> we think is still needed to be done before the beta 1 release? Are
>> there open PRs for everything? Do we a milestone on github to
>> indicate which PRs need to go in before beta1?
> 
> 
> I believe the "blockers" to be:
> 
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12536__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-JQc5z_l4$
>  
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12754__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-JseFWMvQ$
>  
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12750__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-J0OAUQ5s$
>  
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12781__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-JYPbPTZA$
>  
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12801__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-J_mL-S18$
>  
> 
> Aside from these there are potential PRs not yet opened but slated for
> beta1 regarding:
> - Refactor serialization? (owner: Richard)
> - lhash refactor? (owner: Me)
> - IG D.9 update? (owner: Shane)
> 
> There's also this issue - but I'm not sure that it needs to be resolved
> for beta1:
> https://urldefense.com/v3/__https://github.com/openssl/openssl/issues/12195__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-J1-VRwBk$
>  
> 
> Matt



Re: More GitHub labels

2020-09-10 Thread Dr Paul Dale
Matthias,

Your suggestion seems workable too.  PRs are merged with outstanding change 
requests indicated — a reviewer comments, the comments are addressed then a 
different reviewer approves without the original review being removed.  The 
labels are a bit more in your face.  A hybrid “hold: required changes see 
review/comments” might be workable.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 10 Sep 2020, at 4:03 pm, Dr. Matthias St. Pierre 
>  wrote:
> 
>> Just wondering if we should have two new labels: “hold: tests needed” and 
>> “hold: documentation needed” labels?
>> There are a number of PRs that come through where one or both of these are 
>> missing missing.
> 
> The two use cases you mention are actually better handled by a change request 
> (via GitHub review) instead of a label:
> A team member can formulate a change request to block the merging. Here he 
> has more freedom to formulate what
> needs to be done. This includes your two use cases, as well as the use case 
> for the label [hold: need rebase].
> 
> The problem with it is that currently we count only approvals and don’t 
> consider unresolved change requests
> as a blocker. I think we should change that. This does not mean that a 
> reviewer who made a change request
> two months ago and lost interest is forced to re-review, only that such stale 
> reviews must be dismissed
> explicitly, if the reviewer does not respond to a re-review request within a 
> certain time period.
> 
> Matthias
> 



More GitHub labels

2020-09-09 Thread Dr Paul Dale
Just wondering if we should have two new labels: “hold: tests needed” and 
“hold: documentation needed” labels?
There are a number of PRs that come through where one or both of these are 
missing (this post posed by @slontis’s comment in 12826 
<https://github.com/openssl/openssl/pull/12826#pullrequestreview-485480847>).

These would give a clear indication of what additional material is being 
expected as per CLA required and need rebase.

A “hold: code needed" seems less useful :)


Thoughts or should we just add them?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Reordering new API's that have a libctx, propq

2020-09-09 Thread Dr Paul Dale
Still no need for the added complexity:

Push:
OPENSSL_CTX *prevctx = OPENSSL_CTX_set0_default(libctx);

Pop is:
OPENSSL_CTX_set0_default(prevctx);

Push before callback:
OPENSSL_CTX_set0_default(prevctx);

Pop after callback:
prevctx = OPENSSL_CTX_set0_default(libctx);
or
OPENSSL_CTX_set0_default(libctx);
Depending if we want to support call backs changing the Libctx directly — we 
should choose one and always recommend that.


Also the auto allocation of storage for the second on stack cannot fail, so no 
error checking spaghetti.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 10 Sep 2020, at 12:08 am, Tomas Mraz  wrote:
> 
> On Wed, 2020-09-09 at 22:29 +1000, Dr Paul Dale wrote:
>>> On 9 Sep 2020, at 9:38 pm, Tomas Mraz  wrote:
>>> 
>>> We could even provide a convenience thread local stack of lib
>>> contexts
>>> so the caller would not have to keep the old value but would just
>>> push
>>> the new libctx when entering and pop the old one when leaving. With
>>> that, I think the changes needed in the application code would be
>>> fairly simple and minimal.
>> 
>> Let’s not overcomplicate things.
>> We went through this discussion back when this was introduced.
>> 
>> 
>> Push is:
>>OPENSSL_CTX *prevctx = OPENSSL_CTX_set0_default(libctx);
>> 
>> Pop is:
>>OPENSSL_CTX_set0_default(prevctx)
>> 
>> 
>> I don’t see having an explicit stack of these is of any benefit to
>> anything but unwarranted complexity.
> 
> There is one thing where it would be IMO helpful - let's say libcurl
> has a callback into a calling application. With the current API in
> libcurl API calls you would put the
> calls OPENSSL_CTX_set0_default(libctx)/OPENSSL_CTX_set0_default(prevctx
> ) at the beginning and end. But you would have to save the prevctx
> somewhere in the libcurl context structure because on callbacks you
> would have to temoprarily reset the context to the prevctx value. If we
> implemented real stack it would not be needed. But yeah, I am not sure
> this convenience is that much worth it.
> 
> -- 
> Tomáš Mráz
> No matter how far down the wrong road you've gone, turn back.
>  Turkish proverb
> [You'll know whether the road is wrong if you carefully listen to your
> conscience.]



Re: Reordering new API's that have a libctx, propq

2020-09-09 Thread Dr Paul Dale



> On 9 Sep 2020, at 9:38 pm, Tomas Mraz  wrote:
> 
> We could even provide a convenience thread local stack of lib contexts
> so the caller would not have to keep the old value but would just push
> the new libctx when entering and pop the old one when leaving. With
> that, I think the changes needed in the application code would be
> fairly simple and minimal.

Let’s not overcomplicate things.
We went through this discussion back when this was introduced.


Push is:
OPENSSL_CTX *prevctx = OPENSSL_CTX_set0_default(libctx);

Pop is:
OPENSSL_CTX_set0_default(prevctx)


I don’t see having an explicit stack of these is of any benefit to anything but 
unwarranted complexity.



Pauli



Re: Beta1 PR deadline

2020-08-26 Thread Dr Paul Dale
It is also worth noting that new features will not be accepted during the beta 
period.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 27 Aug 2020, at 1:58 am, Matt Caswell  wrote:
> 
> Hi all
> 
> The OMC had a meeting today.
> 
> Please can anyone with PRs that they wish to have included in OpenSSL
> 3.0 beta1 ensure that they are merged to master by 8th September.
> 
> Note in particular that there is no PR at the moment to incorporate SM2
> asymmetric encryption into OpenSSL 3.0. This feature currently exists in
> 1.1.1, so if no PR emerges by the above date then this feature is liable
> to be dropped in 3.0. (Note: PRs for SM2 signatures *do* exist and are
> expected to be present).
> 
> 
> Matt
> 
> 



TLS 1.3 illustrated

2020-08-16 Thread Dr Paul Dale
This might be interesting to some:

https://tls13.ulfheim.net <https://tls13.ulfheim.net/>

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






RAND_DRBG futures

2020-08-03 Thread Dr Paul Dale
I’ve closed the vote.  Five for, none against, the vote passes.  RAND_DRBG will 
be absent in 3.0.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: RAND_DRBG

2020-07-27 Thread Dr Paul Dale
So far a universal voice for removal of the DRBG_RAND APIs.
I’ll write up an OMC vote.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 27 Jul 2020, at 6:51 pm, Matt Caswell  wrote:
> 
> I'm ok with option 1 (but it will require a vote). I think the
> percentage of our user base that are using the existing API is
> sufficiently close to zero that we're not breaking our compatibility
> promises.
> 
> Matt
> 
> 
> On 27/07/2020 02:08, Dr Paul Dale wrote:
>> The RAND_DRBG (crypto/rand/drbg_lib) APIs are quite some mess and sit
>> badly with the move to provider based infrastructure.
>> They are definitely being deprecated in master but without more, the
>> extra layer of indirection and additional complexity generating random
>> numbers will remain.
>> 
>> The option I see are:
>> 
>> 1. Remove, rather than deprecate, RAND_DRBG in 3.0.  This is a breaking
>> change.
>> 2. Bypass RAND_DRBG and live with a breaking change (refer:
>> https://github.com/openssl/openssl/pull/12509#pullrequestreview-455396828)
>> 3. Leave things as they currently are in master.
>> 
>> The first two are breaking changes and will require an OMC vote.
>> 
>> 
>> Some pertinent points:
>> 
>> 1. RAND_bytes will continue to work as is — this API is perfect for
>> almost everyone.
>> 2. The RAND_METHOD functionality remains — this allows wholesale
>> replacement of OpenSSL’s RNGs if desired.
>> 3. The name EVP_RAND is the working name and might change — this is not
>> relevant to this discussion.
>> 4. The RAND_DRBG APIs are unlikely to be widely used — they were
>> introduced in 1.1.1.  The two users I know of (Akamai and NCP) are both
>> fine with them being removed.
>> 
>> 
>> Thoughts anyone?
>> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 



RAND_DRBG

2020-07-26 Thread Dr Paul Dale
The RAND_DRBG (crypto/rand/drbg_lib) APIs are quite some mess and sit badly 
with the move to provider based infrastructure.
They are definitely being deprecated in master but without more, the extra 
layer of indirection and additional complexity generating random numbers will 
remain.

The option I see are:

1. Remove, rather than deprecate, RAND_DRBG in 3.0.  This is a breaking change.
2. Bypass RAND_DRBG and live with a breaking change (refer: 
https://github.com/openssl/openssl/pull/12509#pullrequestreview-455396828)
3. Leave things as they currently are in master.

The first two are breaking changes and will require an OMC vote.


Some pertinent points:

1. RAND_bytes will continue to work as is — this API is perfect for almost 
everyone.
2. The RAND_METHOD functionality remains — this allows wholesale replacement of 
OpenSSL’s RNGs if desired.
3. The name EVP_RAND is the working name and might change — this is not 
relevant to this discussion.
4. The RAND_DRBG APIs are unlikely to be widely used — they were introduced in 
1.1.1.  The two users I know of (Akamai and NCP) are both fine with them being 
removed.


Thoughts anyone?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia



Re: API renaming

2020-07-24 Thread Dr Paul Dale
I think the types should change to match any function name changes.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 24 Jul 2020, at 2:45 am, Short, Todd  wrote:
> 
> They also correspond directly to EVP_MAC and EVP_KDF types. Would the types 
> change as well?
> --
> -Todd Short
> // tsh...@akamai.com <mailto:tsh...@akamai.com>
> // “One if by land, two if by sea, three if by the Internet."
> 
>> On Jul 23, 2020, at 11:56 AM, Matt Caswell > <mailto:m...@openssl.org>> wrote:
>> 
>> 
>> 
>> On 23/07/2020 16:52, Richard Levitte wrote:
>>> On Thu, 23 Jul 2020 12:18:10 +0200,
>>> Dr Paul Dale wrote:
>>>> There has been a suggestion to rename EVP_RAND to OSSL_RAND.  This seems 
>>>> reasonable.  Would it
>>>> also make sense to rename the other new APIs similarly.
>>>> More specifically, EVP_MAC and EVP_KDF to OSSL_MAC and OSSL_KDF 
>>>> respectively?
>>> 
>>> This is a good question...
>>> 
>>> Historically speaking, even though EVP_MAC and EVP_KDF are indeed new
>>> APIs, they have a previous history of EVP APIs, through EVP_PKEY.  The
>>> impact of relocating them outside of the EVP "family" may be small,
>>> but still, history gives me pause.
>>> 
>>> RAND doesn't carry the same sort of history, which makes it much
>>> easier for me to think "just do it and get it over with"...
>> 
>> I have the same pause - so  I'm thinking just RAND for now.
>> 
>> Matt
>> 
> 



Re: API renaming

2020-07-24 Thread Dr Paul Dale
Adherence to the code style will also be required (indentation will change).  
This will be harder to automate.

Changing EVP_RAND -> OSSL_RAND is worse because it will change line breaks as 
well as indentation.  OSSL_RNG avoids this, if we accept not using RAND in the 
name.
KDF and MAC also get longer and would have the same issue.


Then there are the inevitable merge conflicts….


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 24 Jul 2020, at 6:15 pm, Dr. Matthias St. Pierre 
>  wrote:
> 
>> I was thinking OSSL_LIBCTX?
> 
> That's a good choice and consistent with how we name the variable in most 
> (but not all) places:
> 
>   OPENSSL_CTX *libctx;
> 
> I volunteer to raise a pull request which does a scripted bulk rename, as 
> soon as we have made
> a decision. Ideally, the bulk renaming should go in shortly before the next 
> alpha. Having it
> automated by a script would ease rebasing of other still unmerged pull 
> requests over the change.
> 
> Matthias
> 
>> -Original Message-
>> From: SHANE LONTIS 
>> Sent: Friday, July 24, 2020 9:43 AM
>> To: Dr. Matthias St. Pierre 
>> Cc: Richard Levitte ; openssl-project@openssl.org
>> Subject: Re: API renaming
>> 
>> I was thinking OSSL_LIBCTX?
>> 
> 



Re: API renaming

2020-07-23 Thread Dr Paul Dale
The exact same points apply to EVP_MAC & EVP_KDF.

We have also been telling people “use EVP” for ages.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 24 Jul 2020, at 3:20 pm, Richard Levitte  wrote:
> 
> A couple of points:
> 
> 1.  Quite a while ago, we (the team at the time) made a decision to
>have all new APIs prefixed with 'OPENSSL_' or 'OSSL_'.  It seems
>that we never voted on it, though, but still.
> 
> 2.  The new RAND API hasn't been merged yet, so it's not like we're
>renaming something that already exists.
> 
> So in terms of "it's just a prefix", OSSL_ would be just as suitable.
> It's a bit more blatantly "OpenSSL", though.
> 
> Cheers,
> Richard
> 
> On Thu, 23 Jul 2020 23:30:25 +0200,
> Tim Hudson wrote:
>> Placing everything under EVP is reasonable in my view. It is just a prefix 
>> and it really has no
>> meaning these days as it became nothing more than a common prefix to use.
>> 
>> I don't see any significant benefit in renaming at this point - even for 
>> RAND.
>> 
>> Tim.
>> 
>> On Fri, 24 Jul 2020, 1:56 am Matt Caswell,  wrote:
>> 
>>On 23/07/2020 16:52, Richard Levitte wrote:
>>> On Thu, 23 Jul 2020 12:18:10 +0200,
>>> Dr Paul Dale wrote:
>>>> There has been a suggestion to rename EVP_RAND to OSSL_RAND.  This seems 
>>>> reasonable.  Would
>>it
>>>> also make sense to rename the other new APIs similarly.
>>>> More specifically, EVP_MAC and EVP_KDF to OSSL_MAC and OSSL_KDF 
>>>> respectively?
>>> 
>>> This is a good question...
>>> 
>>> Historically speaking, even though EVP_MAC and EVP_KDF are indeed new
>>> APIs, they have a previous history of EVP APIs, through EVP_PKEY.  The
>>> impact of relocating them outside of the EVP "family" may be small,
>>> but still, history gives me pause.
>>> 
>>> RAND doesn't carry the same sort of history, which makes it much
>>> easier for me to think "just do it and get it over with"...
>> 
>>I have the same pause - so  I'm thinking just RAND for now.
>> 
>>Matt
>> 
>> 
> -- 
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/



API renaming

2020-07-23 Thread Dr Paul Dale
There has been a suggestion to rename EVP_RAND to OSSL_RAND.  This seems 
reasonable.  Would it also make sense to rename the other new APIs similarly.
More specifically, EVP_MAC and EVP_KDF to OSSL_MAC and OSSL_KDF respectively?

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia



Re: Reducing the security bits for MD5 and SHA1 in TLS

2020-06-17 Thread Dr Paul Dale
I’d agree it’s major for for SHA1 but not for MD5.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 18 Jun 2020, at 12:20 pm, Tim Hudson  wrote:
> 
> Given that this change impacts interoperability in a major way it should be a 
> policy vote of the OMC IMHO.
> 
> Tim.
> 
> 
> On Thu, 18 Jun 2020, 5:57 am Kurt Roeckx,  <mailto:k...@roeckx.be>> wrote:
> On Wed, May 27, 2020 at 12:14:13PM +0100, Matt Caswell wrote:
> > PR 10787 proposed to reduce the number of security bits for MD5 and SHA1
> > in TLS (master branch only, i.e. OpenSSL 3.0):
> > 
> > https://github.com/openssl/openssl/pull/10787 
> > <https://github.com/openssl/openssl/pull/10787>
> > 
> > This would have the impact of meaning that TLS < 1.2 would not be
> > available in the default security level of 1. You would have to set the
> > security level to 0.
> > 
> > In my mind this feels like the right thing to do. The security bit
> > calculations should reflect reality, and if that means that TLS < 1.2 no
> > longer meets the policy for security level 1, then that is just the
> > security level doing its job. However this *is* a significant breaking
> > change and worthy of discussion. Since OpenSSL 3.0 is a major release it
> > seems that now is the right time to make such changes.
> > 
> > IMO it seems appropriate to have an OMC vote on this topic (or should it
> > be OTC?). Possible wording:
> 
> So should that be an OMC or OTC vote, or does it not need a vote?
> 
> 
> Kurt
> 



Re: Backports to 1.1.1 and what is allowed

2020-06-16 Thread Dr Paul Dale
I’d also support 11968 being merged.  I find the recent discussion and the 
likelihood of all major distros (supporting the S390x) picking up the patches 
fairly compelling, especially since the same people will be maintaining it.

I would need to go and double check the non-S390x specific parts for possible 
breakage but the bulk of the changes are S390x specific.


I support formalising the rules better than we have at the moment.  Even if 
this is in conflict with the above.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 17 Jun 2020, at 1:43 pm, Benjamin Kaduk  wrote:
> 
> On Tue, Jun 16, 2020 at 02:03:58PM +0100, Matt Caswell wrote:
>> PR 11188 proposes to backport a series of s390x patches to the 1.1.1
>> branch. IIUC it includes performance improvements as well as support for
>> new hardware instructions.
>> 
>> I think we need to have a much clearer and more explicit policy about
>> exactly what is allowed to be backported to a stable branch. For example
>> PR #11968 was a significant recent PR that backported AES CTR-DRBG
>> performance improvements to the 1.1.1 branch and was allowed (should it
>> have been?).
> 
> I am happy to see 11968 landed; some informal testing that I hope to make
> more formal soon seems to show a ~20% slowdown/regression for large RNG
> requests going from 1.1.1d to 1.1.g, and 11968 provided substantially more
> significant of a speedup (again, very informal testing, though).
> 
> In this case you could say that the "bug" is that we're only calling AES on
> a block at a time despite having much more effective backends available for
> multi-block calls.
> 
>> We have always said that the stable releases should only receive bug and
>> security fixes. Performance improvements have been a bit of a grey area,
>> e.g. a few lines of code that significantly improves the performance of
>> a particular algorithm or codepath could reasonably be justified as
>> "fixing a performance bug". OTOH bringing in whole new files of
>> assembler seems to go way beyond that definition.
> 
> There's probably some semi-subtle distinction to make along the lines of
> "making the algorithm more efficient" vs "using a new algorithm, that
> happens to run faster", with the latter counting as a feature.
> 
>> However, when I tried to find a reference to the policy which says
>> exactly what we are allowed to backport I could find one. Do we have
>> such a thing?
>> 
>> In any case I think we should develop a much more explicit policy that
>> will enable us to decide whether PRs such as 11188 are reasonable or
>> not. See for example Ubuntu's Stable Release Updates policy:
> 
> I agree that having something written down will be very useful, even if
> just as a fixed benchmark against which to think about how exceptional any
> given "exception case" is where we want to deviate from it.
> 
> -Ben
> 
>> https://wiki.ubuntu.com/StableReleaseUpdates
>> 
>> 
>> Matt



Seeking assistance

2020-06-03 Thread Dr Paul Dale
The OMC has approved working with the FIPS lab to include a number of 
additional algorithms in the upcoming FIPS validation.
One slated for inclusion turns out to be beyond the time available for the 
fellows and the two others working on 3.0.

This is the X9.42 KDF.  What it needs is the X509 and CMS calls removed from 
the KDF and the various structures constructed piecemeal using calls that 
already exist within the FIPS provider.


If somebody is willing to take this work on, it will likely be included in the 
FIPS validation for 3.0.
If not, it won’t be.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Vote re: #11577

2020-06-03 Thread Dr Paul Dale
topic: Accept and merge #11577.
comment: #11577 reduces the maximum length of TLS labels.
 It also breaks standards compliance.

8 against, none for, no abstentions, 3 not yet voted.

The vote failed, the PR will be closed.


Pauli

-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: OMC Vote on deprecation of command line apps

2020-05-08 Thread Dr Paul Dale
This vote has passed: 3 for, 1 against and 2 abstentions.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 8 May 2020, at 3:08 pm, Dr Paul Dale  wrote:
> 
> PR 11575 <https://github.com/openssl/openssl/pull/11575> has been blocking 
> awaiting decision for a while now.  Time for a vote:
> 
> topic: Merge #11575 for 3.0.
> comment: This PR removes the notes indicating that a number of the command
>  line utilities are deprecated.  Not merging it will leave them 
> flagged
>  as deprecated.
> Proposed by: Paul Dale
> Public: yes
> opened: 2020-05-08
> 
> Ideally we’ll have a decision in time for the next 3.0 alpha release.
> 
> 
> The crux of the matter is that a number of the command line utilities are 
> flagged as deprecated currently:
> dhparam
> dsa
> dsaparam
> ec
> ecparam
> agendas
> rsa
> These commands are not being removed in 3.0, instead they’ve been rewritten 
> to use the PKEY APIs instead of the low level APIs as far as possible.
> 
> 
> The reasons for keeping them are:
> they are easier to use than the pkey replacements
> a web search will likely result in thees commands not the pkey replacements.
> 
> The reason for removing them is one of maintenance: having duplicate commands 
> means having to make changes in two places and this has been missed in the 
> past and will be in the future.
> 
> 
> Other random notes:
> Deprecation of these commands does not mandate that they are removed at the 
> first opportunity.  It only indicates that we want to move away from them.
> Rewriting these commands so that they call the pkey replacements looks to be 
> very difficult.  Reproducing the exact behaviours will be challenging, 
> although the basic functionality would be straightforward.
> The rsautl command is deprecated and isn’t slated for being restored — 
> pkeyutl is every bit as easy to use.
> The -dsaparam option to dhparam is deprecated — it cannot be supported 
> without direct access to low level functionality we want to remove.
> Post quantum crypto will make the discussion obsolete — none of these 
> algorithms are useful in a quantum computer world.
> 
> My personal opinion is that these commands are good being deprecated but that 
> we should not remove them until their usefulness is at an end.  This will 
> likely mean not removing them after five years of deprecation.  It would mean 
> removing them once quantum computers are shown to be effective.  Without 
> deprecation now, we can’t remove them until a lot later.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 



OMC Vote on deprecation of command line apps

2020-05-07 Thread Dr Paul Dale
PR 11575 <https://github.com/openssl/openssl/pull/11575> has been blocking 
awaiting decision for a while now.  Time for a vote:

topic: Merge #11575 for 3.0.
comment: This PR removes the notes indicating that a number of the command
 line utilities are deprecated.  Not merging it will leave them flagged
 as deprecated.
Proposed by: Paul Dale
Public: yes
opened: 2020-05-08

Ideally we’ll have a decision in time for the next 3.0 alpha release.


The crux of the matter is that a number of the command line utilities are 
flagged as deprecated currently:
dhparam
dsa
dsaparam
ec
ecparam
agendas
rsa
These commands are not being removed in 3.0, instead they’ve been rewritten to 
use the PKEY APIs instead of the low level APIs as far as possible.


The reasons for keeping them are:
they are easier to use than the pkey replacements
a web search will likely result in thees commands not the pkey replacements.

The reason for removing them is one of maintenance: having duplicate commands 
means having to make changes in two places and this has been missed in the past 
and will be in the future.


Other random notes:
Deprecation of these commands does not mandate that they are removed at the 
first opportunity.  It only indicates that we want to move away from them.
Rewriting these commands so that they call the pkey replacements looks to be 
very difficult.  Reproducing the exact behaviours will be challenging, although 
the basic functionality would be straightforward.
The rsautl command is deprecated and isn’t slated for being restored — pkeyutl 
is every bit as easy to use.
The -dsaparam option to dhparam is deprecated — it cannot be supported without 
direct access to low level functionality we want to remove.
Post quantum crypto will make the discussion obsolete — none of these 
algorithms are useful in a quantum computer world.

My personal opinion is that these commands are good being deprecated but that 
we should not remove them until their usefulness is at an end.  This will 
likely mean not removing them after five years of deprecation.  It would mean 
removing them once quantum computers are shown to be effective.  Without 
deprecation now, we can’t remove them until a lot later.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






FIPS_mode() vote results

2020-05-04 Thread Dr Paul Dale
The vote:

Remove the calls FIPS_mode() & FIPS_mode_set() in 3.0.


Has closed.

For: 3
Against: 1
Abstain: 2


The vote passes.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Cherry-pick proposal

2020-04-29 Thread Dr Paul Dale
My concern is are 1.1.1 and 3.0 really all that different?
The core is quite different but the cryptographic algorithms aren’t.  CMS, 
x509, …?

I’d rather not introduce a burden where it isn’t necessary.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 29 Apr 2020, at 10:08 pm, Matt Caswell  wrote:
> 
> 
> The OTC have received this proposal and a request that we vote on it:
> 
> I would like to request that we do not allow cherry-picks between master
> and 1.1.1-stable because these two versions are now very different, if a
> cherry-pick succeeds, there is no guarantee that the result will work.
> Because we have no CI for the cherry-pick. If a cherry-pick is needed, a
> new PR for 1.1.1 should be done and approved independently.
> 
> Before starting a vote I'd like to provide opportunity for comments, and
> also what the vote text should be.
> 
> Thanks
> 
> Matt



Re: An OpenSSL cookbook, where and how?

2020-03-07 Thread Dr Paul Dale
Might the demos be useful for something like this?
I know they aren’t in great state and could do with better documentation but 
they seem to fulfil most of the suggested goals.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 7 Mar 2020, at 7:03 pm, Richard Levitte  wrote:
> 
> On Sat, 07 Mar 2020 10:01:59 +0100,
> Richard Levitte wrote:
>> 
>> As part of a documentation effort, more specifically in some
>> discussions in 
>> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/11220__;!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5BmcJuDJ7pw$
>>   (exact
>> references below), I've floated the idea that bigger coding examples
>> should be placed into a cookbook.
> 
> I forgot the references:
> 
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/11220*discussion_r386751635__;Iw!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5Bmc3tLgiYQ$
>  
> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/11220*issuecomment-596060267__;Iw!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5BmcAjPjGdw$
>  
> 
> -- 
> Richard Levitte levi...@openssl.org
> OpenSSL Project 
> https://urldefense.com/v3/__http://www.openssl.org/*levitte/__;fg!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5BmcbK3Tc-Y$
>  



Re: Deprecations

2020-03-04 Thread Dr Paul Dale
Matthew,

Good idea.  I’ll add it.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 5 Mar 2020, at 8:55 am, Matthew Lindner  wrote:
> 
> Shouldn't the deprecation notice that's printed also print the version it was 
> deprecated in?
> 
> - Matthew Lindner
> 
>> On Mar 4, 2020, at 2:48 PM, Dr Paul Dale > <mailto:paul.d...@oracle.com>> wrote:
>> 
>> EXTERNAL MAIL: openssl-project-boun...@openssl.org 
>> <mailto:openssl-project-boun...@openssl.org>
>> 
>> Unless I’ve missed something, the documentation should specify the 
>> alternatives.  Mostly these are one to one, but in one case it is one to two 
>> (and there both are listed).
>> With the caveat that I might not have got every place or detail correct.
>> 
>> Moreover, the deprecated commands print something to the effect of: "The 
>> command dsa is deprecated. Use ‘pkey’ instead." when executed.
>> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 
>> 
>> 
>> 
>>> On 5 Mar 2020, at 5:15 am, Kurt Roeckx >> <mailto:k...@roeckx.be>> wrote:
>>> 
>>> On Mon, Mar 02, 2020 at 11:41:57AM +, Matt Caswell wrote:
>>>> 
>>>> 
>>>> On 28/02/2020 23:43, Dr Paul Dale wrote:
>>>>> Any suggestions for a consensus on this thread?
>>>> 
>>>> I think we can probably agree that:
>>>> 
>>>> - Command option deprecations should be handled better
>>>> - We should look at whether we can resurrect some of the "old" commands
>>>> (possibly by writing them as wrappers for genpkey, pkey and pkeyutl)
>>> 
>>> What about at least pointing to the alternative function in the
>>> documentation?
>>> 
>>> 
>>> Kurt
>>> 
>> 
> 



Re: Deprecations

2020-03-04 Thread Dr Paul Dale
Unless I’ve missed something, the documentation should specify the 
alternatives.  Mostly these are one to one, but in one case it is one to two 
(and there both are listed).
With the caveat that I might not have got every place or detail correct.

Moreover, the deprecated commands print something to the effect of: "The 
command dsa is deprecated. Use ‘pkey’ instead." when executed.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 5 Mar 2020, at 5:15 am, Kurt Roeckx  wrote:
> 
> On Mon, Mar 02, 2020 at 11:41:57AM +, Matt Caswell wrote:
>> 
>> 
>> On 28/02/2020 23:43, Dr Paul Dale wrote:
>>> Any suggestions for a consensus on this thread?
>> 
>> I think we can probably agree that:
>> 
>> - Command option deprecations should be handled better
>> - We should look at whether we can resurrect some of the "old" commands
>> (possibly by writing them as wrappers for genpkey, pkey and pkeyutl)
> 
> What about at least pointing to the alternative function in the
> documentation?
> 
> 
> Kurt
> 



Face to face

2020-03-03 Thread Dr Paul Dale
I propose that we don’t have either an OMC or OTC face to face meeting at ICMC.
Travel restrictions and the availability list make it look unlikely.

Instead, I’d suggest a video conference for both.

This will probably require some kind of vote (for both).


Any suggestions as to date and time.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Deprecations

2020-03-02 Thread Dr Paul Dale
I've started working on moving some of the old commands forward using PKEY 
calls.  My intention is for them to still print out a deprecated message when 
run but for them to not actually be removed by the no-deprecated configure 
option.

Having them print equivalent pkey command looks to be somewhat problematic.  
There isn’t a 1:1 conversion and some of the legacy options simply aren’t 
supported.


I’m hoping to have a preliminary PR up later this week.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 2 Mar 2020, at 9:41 pm, Matt Caswell  wrote:
> 
> 
> 
> On 28/02/2020 23:43, Dr Paul Dale wrote:
>> Any suggestions for a consensus on this thread?
> 
> I think we can probably agree that:
> 
> - Command option deprecations should be handled better
> - We should look at whether we can resurrect some of the "old" commands
> (possibly by writing them as wrappers for genpkey, pkey and pkeyutl)
> 
> I am slightly concerned that the latter option (rewriting as wrappers)
> may turn into a big black hole of effort. It *might* be easier to just
> rewrite them as-is to use EVP. Whichever approach we take, I don't think
> this should be a goal for alpha1.
> 
> Matt
> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 
>> 
>> 
>> 
>>> On 24 Feb 2020, at 5:08 pm, Dr Paul Dale >> <mailto:paul.d...@oracle.com>> wrote:
>>> 
>>> Most of the conversions to using PKEY were straightforward.  One
>>> didn’t require any changes (dsa but my memory is suspect).  One seemed
>>> quite difficult.  Some I didn’t check.
>>> 
>>> Modifying the commands so that they continue to work and print (to
>>> stderr) an alternative pkey based command might be workable too.
>>> 
>>> 
>>> Pauli
>>> -- 
>>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>>> Phone +61 7 3031 7217
>>> Oracle Australia
>>> 
>>> 
>>> 
>>> 
>>>> On 24 Feb 2020, at 5:53 am, Viktor Dukhovni
>>>> mailto:openssl-us...@dukhovni.org>> wrote:
>>>> 
>>>>> On Feb 22, 2020, at 4:53 AM, Richard Levitte >>>> <mailto:levi...@openssl.org>> wrote:
>>>>> 
>>>>> Something that could be done is to take all those aged commands and
>>>>> rewrite them as wrappers for genpkey, pkey and pkeyutl.  Simply create
>>>>> and populate a new argv and call genpkey_main(), pkey_main() or
>>>>> pkeyutl_main().
>>>> 
>>>> Agreed, that sounds quite reasonable at first blush, and could be
>>>> fantastic
>>>> if it can be made to work (no immediate obstacles come to mind).
>>>> 
>>>> -- 
>>>> Viktor.
>>>> 
>>> 
>> 



Re: Deprecations

2020-02-28 Thread Dr Paul Dale
Any suggestions for a consensus on this thread?

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 24 Feb 2020, at 5:08 pm, Dr Paul Dale  wrote:
> 
> Most of the conversions to using PKEY were straightforward.  One didn’t 
> require any changes (dsa but my memory is suspect).  One seemed quite 
> difficult.  Some I didn’t check.
> 
> Modifying the commands so that they continue to work and print (to stderr) an 
> alternative pkey based command might be workable too.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 
>> On 24 Feb 2020, at 5:53 am, Viktor Dukhovni > <mailto:openssl-us...@dukhovni.org>> wrote:
>> 
>>> On Feb 22, 2020, at 4:53 AM, Richard Levitte >> <mailto:levi...@openssl.org>> wrote:
>>> 
>>> Something that could be done is to take all those aged commands and
>>> rewrite them as wrappers for genpkey, pkey and pkeyutl.  Simply create
>>> and populate a new argv and call genpkey_main(), pkey_main() or
>>> pkeyutl_main().
>> 
>> Agreed, that sounds quite reasonable at first blush, and could be fantastic
>> if it can be made to work (no immediate obstacles come to mind).
>> 
>> -- 
>>  Viktor.
>> 
> 



Re: Deprecations

2020-02-23 Thread Dr Paul Dale
Most of the conversions to using PKEY were straightforward.  One didn’t require 
any changes (dsa but my memory is suspect).  One seemed quite difficult.  Some 
I didn’t check.

Modifying the commands so that they continue to work and print (to stderr) an 
alternative pkey based command might be workable too.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 24 Feb 2020, at 5:53 am, Viktor Dukhovni  
> wrote:
> 
>> On Feb 22, 2020, at 4:53 AM, Richard Levitte  wrote:
>> 
>> Something that could be done is to take all those aged commands and
>> rewrite them as wrappers for genpkey, pkey and pkeyutl.  Simply create
>> and populate a new argv and call genpkey_main(), pkey_main() or
>> pkeyutl_main().
> 
> Agreed, that sounds quite reasonable at first blush, and could be fantastic
> if it can be made to work (no immediate obstacles come to mind).
> 
> -- 
>   Viktor.
> 



Re: Deprecations

2020-02-21 Thread Dr Paul Dale
The added complexity was of some concern to me when doing the deprecations.

I suspect we’ll also encounter difficulties getting 100% equivalent behaviour 
via PKEY.  There are some pretty arcane options in some of these.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 22 Feb 2020, at 9:51 am, Kurt Roeckx  wrote:
> 
> On Fri, Feb 21, 2020 at 11:27:55PM +, Matt Caswell wrote:
>> 
>> 
>> On 21/02/2020 23:18, Kurt Roeckx wrote:
>>> On Fri, Feb 21, 2020 at 11:00:10PM +, Matt Caswell wrote:
>>>> 
>>>> dhparam itself has been deprecated. For that reason we are not
>>>> attempting to rewrite it to use non-deprecated APIs. The informed
>>>> decision we have made about DH_check use in dhparam is to not build the
>>>> whole application in a no-deprecated build:
>>>> 
>>>>  *) The command line utilities dhparam, dsa, gendsa and dsaparam have been
>>>> deprecated.  Instead use the pkeyparam, pkey, genpkey and pkeyparam
>>>> programs respectively.
>>>> [Paul Dale]
>>> 
>>> For some reason I seem to have missed various things.
>>> 
>>> But I think deprecating tools like dhparam, dsaparam in favour of
>>> genpkey is something that we should reconsider.
>> 
>> What is your reasoning?
>> 
>> (I just realised that what the CHANGES entry says is that
>> dhparam/dsaparam are deprecated in favour of pkeyparam - but actually I
>> think the equivalent functionality is more split between genpkey and
>> pkeyparam)
> 
> Some equivalants:
> openssl dhparam 2048
> openssl genpkey -genparam --algorithm DH -pkeyopt dh_paramgen_prime_len:2048
> 
> openssl dsaparam 2048
> openssl genpkey -genparam -algorithm DSA -pkeyopt dsa_paramgen_bits:2048
> 
> 
> If you search internet, you will more than likely find the first
> ones. They are very easy. I have to look up at the manual page
> examples to know how to use genpkey.
> 
> 
> Kurt



Re: Errored: openssl/openssl#31939 (master - 34b1676)

2020-02-14 Thread Dr Paul Dale
An alternative would be to only run a cut down selection of tests with msan.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 14 Feb 2020, at 11:00 pm, Matt Caswell  wrote:
> 
> 
> 
> On 14/02/2020 12:23, Nicola Tuveri wrote:
>> If ASAN is too slow to run in the CI should we restore the previous
>> homemade checks for memory leaks as an alternative to be run in regular
>> CI runs and leave ASAN builds to run-checker on the master branch only? 
> 
> To be clear the build that is timing out uses *msan* not *asan*.
> 
> As I understand it msan detects unitialised reads. whereas asan detects
> memory corruption, buffer overflows, use-after-free bugs, and memory leaks.
> 
> The previous "home-made" checks only detected memory leaks, so it is not
> comparable with the functionality offered by msan.
> 
> The msan documentation
> (https://urldefense.com/v3/__https://clang.llvm.org/docs/MemorySanitizer.html__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6xiEvnC0$
>  ) suggests that a slow
> down of 3x is typical.
> 
> It seems reasonable to me to disable msan checks in Travis entirely, and
> have them only in run-checker.
> 
>> 
>> Here is another idea that would be interesting if we restore the
>> previous checks:
>> I don't know what kind of options github offers on this, but would it be
>> possible to run triggered CI on something that is not Travis and does
>> not timeout and still have the results in the PR?
> 
> I am sure there are hooks to do this. Richard has been talking for quite
> a while about setting up a buildbot infrastructure. If that could be
> integrated into github that would be really neat.
> 
> Matt
> 
> 
>> If something like that would be possible we could move the ASAN builds
>> to extended_tests, rely on the previous memleak detection for the
>> regular CI runs, and then trigger with a script or Github Action the
>> extended_tests when the approval:done label is added. 
>> 
>> That way, by the time something is ready to be merged we should have a
>> full picture! 
>> 
>> 
>> Nicola
>> 
>> On Wed, Feb 5, 2020, 10:25 Matt Caswell > <mailto:m...@openssl.org>> wrote:
>> 
>>Since we fixed the Travis builds 4 out of the 8 builds on master that
>>have taken place have errored due to a timeout.
>> 
>>The msan build is consistently taking a *very* long time to run. If it
>>gets to 50 minutes then Travis cuts it off and the build fails.
>> 
>>Should we disable the msan build?
>> 
>>Matt
>> 
>> 
>> Forwarded Message 
>>Subject:Errored: openssl/openssl#31939 (master - 34b1676)
>>Date:   Wed, 05 Feb 2020 00:02:01 +
>>From:   Travis CI mailto:bui...@travis-ci.org>>
>>To: openssl-comm...@openssl.org <mailto:openssl-comm...@openssl.org>
>> 
>> 
>> 
>>openssl
>> 
>>/
>> 
>>openssl
>> 
>>
>> <https://urldefense.com/v3/__https://travis-ci.org/openssl/openssl?utm_medium=notification_source=email__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6bPlKFFo$
>>  >
>> 
>> 
>>branch iconmaster 
>> <https://urldefense.com/v3/__https://github.com/openssl/openssl/tree/master__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6tYedpt0$
>>  >
>> 
>>build has errored
>>Build #31939 has errored
>>
>> <https://urldefense.com/v3/__https://travis-ci.org/openssl/openssl/builds/646181069?utm_medium=notification_source=email__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6dntSzqk$
>>  >
>>arrow to build time
>>clock icon50 mins and 3 secs
>> 
>>Pauli avatarPauli
>> 
>>34b1676 CHANGESET →
>>
>> <https://urldefense.com/v3/__https://github.com/openssl/openssl/compare/e3b1ccad694a...34b167625af5__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6KOlK4nk$
>>  >
>> 
>>Make minimum size for secure memory a size_t.
>> 
>>The minimum size argument to CRYPTO_secure_malloc_init() was an int but
>>ought
>>to be a size_t since it is a size.
>> 
>>From an API perspective, this is a change. However, the minimum size is
>>verified as being a positive power of two and it will typically be a
>>small
>>constant.
>> 
>>Reviewed-by: David von Oheimb > 

Re: Deprecation

2020-02-14 Thread Dr Paul Dale
Roumen in https://github.com/openssl/openssl/pull/10977#issuecomment-584818517 
<https://github.com/openssl/openssl/pull/10977#issuecomment-584818517>
Dmitry in https://github.com/openssl/openssl/pull/11082#issuecomment-585603911 
<https://github.com/openssl/openssl/pull/11082#issuecomment-585603911>
And a further one via private email.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 14 Feb 2020, at 7:37 pm, Matt Caswell  wrote:
> 
> 
> 
> On 14/02/2020 02:30, Dr Paul Dale wrote:
>> There is some pushback against the deprecations going on in various PRs.
> 
> I've not followed all of the PRs. Can you point at some specific
> comments you've received pushing back on this?
> 
> Matt
> 
> 
>> 
>> The plan has always been to deprecate engines in 3.0 and removing
>> support for them 5+ years later.  Originally, the path was to have
>> included an engine provider that could load engines and make them appear
>> to be a provider.  After a fair amount of investigation, this was deemed
>> to be too difficult in the 3.0 time frame.
>> 
>> Do we still want to deprecate engines in 3.0?
>> Should we defer until 4.0 instead?
>> 
>> 
>> The main benefits seem to boil down to continuing to support existing
>> engines vs removing the legacy code paths and switching to the provider
>> model.
>> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 



Deprecation

2020-02-13 Thread Dr Paul Dale
There is some pushback against the deprecations going on in various PRs.

The plan has always been to deprecate engines in 3.0 and removing support for 
them 5+ years later.  Originally, the path was to have included an engine 
provider that could load engines and make them appear to be a provider.  After 
a fair amount of investigation, this was deemed to be too difficult in the 3.0 
time frame.

Do we still want to deprecate engines in 3.0?
Should we defer until 4.0 instead?


The main benefits seem to boil down to continuing to support existing engines 
vs removing the legacy code paths and switching to the provider model.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia



Re: Github PR label automation

2020-02-08 Thread Dr Paul Dale
Please don’t automatically drop the "appoval: done" label after a comment.  I 
feel that is not uncommon for comments to be added that in no way invalidate 
the approval.

I agree with not switching to “ready to merge” if there are comments — manual 
intervention in this case is required to judge the relevancy.

Agreed also over the “urgent” label.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 9 Feb 2020, at 1:56 am, Mark J Cox  wrote:
> 
> I've currently got a cron job running every hour that looks at open PR
> requests against github openssl repo and does various actions.  So if
> you were wondering why I was altering labels and making comments at
> 4am, now you know.  No doubt we'll use some tool user for this in the
> future.
> 
> So right now here's what it does:
> 
> Every hour it looks at open PRs that are labelled "approval: done".
> If 24 hours has elapsed since that label was assigned and if there
> have been no comments made to the PR since the label was assigned then
> it is automatically moved to "approval: ready to merge" with a comment
> added to trigger notifications.  So if you want to stop something
> going to "ready to merge" just add any comment to the PR.
> 
> I'm thinking of using this script also to 1) collect interesting stats
> and 2) do some other actions.  So if there's some automation you'd
> like to see just add an enhancement issue against the openssl/tools
> repo.
> 
> 1 Matt already asked for committer notification trigger for anything
> labelled Urgent.
> 
> 2 If there were comments made after "approval: done" then I think we
> really ought to drop the "approval: done" label as the comments likely
> invalidated the approval.  So I'll likely add that next week (if
> "approval: done" label and has comments since that label then remove
> the label and add a comment 'please review if this is really approval:
> done'.  If the approval: done label gets set again then after 24 hours
> the existing automation will trigger.  #10786 is a good example of
> this.
> 
> Mark



Re: Travis in solid red mode again

2020-02-01 Thread Dr Paul Dale
I thought I was subscribed but don’t seem to see the failures.
I do get the (very many) PR activity emails….


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 1 Feb 2020, at 8:35 pm, Dr. Matthias St. Pierre 
>  wrote:
> 
>> -Original Message-
>> On 01/02/2020 00:34, Salz, Rich wrote:
>>> 
>>>> Could we add a CI check for PRs that confirms that the target branch is
>>>green in travis?
>>> 
>>> Looking at 
>>> https://docs.travis-ci.com/user/notifications/#conditional-notifications 
>>> and https://config.travis-
>> ci.com/ref/notifications/email it seems like it should be fairly easy 
>> configure builds to do "send email to openssl-commits when builds on
>> master fail"
>> 
>> We already have that.
> 
> Maybe we should make it a requirement that at least all OTC members subscribe 
> to openssl-commits?
> 
> Matthias
> 



Re: crypt(3)

2020-01-20 Thread Dr Paul Dale
Thanks for the feedback everyone.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia



Re: crypt(3)

2020-01-18 Thread Dr Paul Dale
I meant “what default makes the most sense for the passwd command line 
application?”
It was crypt which is deprecated.  Should it be BSD’s MD5?  One of the SHA2 
based algorithms?  Or should it produce an error if no algorithm is selected?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 19 Jan 2020, at 12:28 am, Kurt Roeckx  wrote:
> 
> On Sat, Jan 18, 2020 at 10:47:04AM +1000, Dr Paul Dale wrote:
>> Could the people who work with distros confirm this default choice or 
>> suggest what they use please?
> 
> I'm not sure what you're asking, but crypt() has moved on from
> using DES like 20 years ago, see crypt(5).
> 
> 
> Kurt
> 



Re: crypt(3)

2020-01-17 Thread Dr Paul Dale
Could the people who work with distros confirm this default choice or suggest 
what they use please?


Thanks,

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 18 Jan 2020, at 10:05 am, Dr Paul Dale  wrote:
> 
> I’ve made the deprecation changes to the password application.
> 
> The default has been changed from crypt to the BSD MD5 algorithm.
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 
>> On 18 Jan 2020, at 9:27 am, Dr Paul Dale > <mailto:paul.d...@oracle.com>> wrote:
>> 
>> Okay, it looks like the consensus is option 3 — deprecate and forget.
>> 
>> As far as I can tell, they are only used (by us) in one place outside of 
>> libcrypto, so that will deprecate as well.
>> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 
>> 
>> 
>> 
>>> On 18 Jan 2020, at 6:53 am, Richard Levitte >> <mailto:levi...@openssl.org>> wrote:
>>> 
>>> Right. Such a KDF could be implemented elsewhere, as a separate project.
>>> 
>>> Cheers
>>> Richard
>>> 
>>> 
>>> Kurt Roeckx mailto:k...@roeckx.be>> skrev: (17 januari 
>>> 2020 21:35:00 CET)
>>>> On Fri, Jan 17, 2020 at 04:31:06PM +1000, Dr Paul Dale wrote:
>>>>> I’ve got several choices:
>>>>> Leave them public and unchanged — that is, don’t deprecate these two
>>>> functions yet.
>>>>> Deprecate them and add KDFs to replace them.
>>>>> Deprecate them, leave them alone and hope they go away painlessly at
>>>> some point.
>>>> 
>>>> I really see no point in adding something that we at the same time
>>>> would like to remove. Just deprecate it.
>>>> 
>>>> 
>>>> Kurt
>>> 
>>> -- 
>>> Richard by mobile
>> 
> 



Re: crypt(3)

2020-01-17 Thread Dr Paul Dale
Okay, it looks like the consensus is option 3 — deprecate and forget.

As far as I can tell, they are only used (by us) in one place outside of 
libcrypto, so that will deprecate as well.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 18 Jan 2020, at 6:53 am, Richard Levitte  wrote:
> 
> Right. Such a KDF could be implemented elsewhere, as a separate project.
> 
> Cheers
> Richard
> 
> 
> Kurt Roeckx  skrev: (17 januari 2020 21:35:00 CET)
>> On Fri, Jan 17, 2020 at 04:31:06PM +1000, Dr Paul Dale wrote:
>>> I’ve got several choices:
>>> Leave them public and unchanged — that is, don’t deprecate these two
>> functions yet.
>>> Deprecate them and add KDFs to replace them.
>>> Deprecate them, leave them alone and hope they go away painlessly at
>> some point.
>> 
>> I really see no point in adding something that we at the same time
>> would like to remove. Just deprecate it.
>> 
>> 
>> Kurt
> 
> -- 
> Richard by mobile



Re: crypt(3)

2020-01-17 Thread Dr Paul Dale
My experience with embedded systems is that crypt(3) is in the standard library 
and not accessed via OpenSSL.  Apps/password uses DES_crypt() and password 
crackers used to use OpenSSL for performance reasons.  Neither seems like a 
huge deal.  I.e. I can’t think of a good reason to keep them.

Removing these calls will require an OMC vote as a breaking API change.  I’m 
fine to call one if it seems justified.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 17 Jan 2020, at 5:41 pm, Viktor Dukhovni  
> wrote:
> 
> On Fri, Jan 17, 2020 at 04:31:06PM +1000, Dr Paul Dale wrote:
> 
>> There are two functions (DES_crypt and DES_fcrypt) which implement the
>> old crypt(3) password algorithm.  Once these are deprecated, they will
>> no longer be reachable via EVP.  The confounding point is that they
>> aren’t quite DES — close but not identical.  I would be surprised if
>> they aren’t still in use for /etc/passwd files on old and/or embedded
>> systems.
> 
> Generally speaking, on Unix-like systems that use crypt(3) for
> /etc/passwd I'd expect to find a standaline crypt() implementation in
> libc, that is independent of OpenSSL.  That is, if your system still
> uses crypt() for passwords, you don't need OpenSSL to compute crypt
> hashes.
> 
> That said, this is experience from general-purpose computers running
> Unix-like OSes, not embedded systems, where I have no idea whether
> crypt() is popular, and whether it is provided by a port of libcrypto
> to that platform.
> 
>> I’ve got several choices: Leave them public and unchanged — that is,
>> don’t deprecate these two functions yet.  Deprecate them and add KDFs
>> to replace them.  Deprecate them, leave them alone and hope they go
>> away painlessly at some point.
> 
> I would not expect to find many users of OpenSSL's crypt(), except
> internally within OpenSSL itself.
> 
>> The apps/password.c applet calls these which is how I stumbled over
>> the complication.  I’m fine refactoring this based on the solution
>> chosen.  I’d also be okay with factoring out all the password
>> derivation functions into KDFs if necessary.
>> 
>> Thoughts?  Other alternatives?
> 
> I don't know enough about embedded systems to speak about what if
> anything we need to do for those with respect to crypt().
> 
> -- 
>Viktor.



crypt(3)

2020-01-16 Thread Dr Paul Dale
In the deprecation efforts for 3.0, I’ve hit something in the DES code that I’d 
appreciate input on.

There are two functions (DES_crypt and DES_fcrypt) which implement the old 
crypt(3) password algorithm.  Once these are deprecated, they will no longer be 
reachable via EVP.  The confounding point is that they aren’t quite DES — close 
but not identical.  I would be surprised if they aren’t still in use for 
/etc/passwd files on old and/or embedded systems.

I’ve got several choices:
Leave them public and unchanged — that is, don’t deprecate these two functions 
yet.
Deprecate them and add KDFs to replace them.
Deprecate them, leave them alone and hope they go away painlessly at some point.

The apps/password.c applet calls these which is how I stumbled over the 
complication.  I’m fine refactoring this based on the solution chosen.  I’d 
also be okay with factoring out all the password derivation functions into KDFs 
if necessary.


Thoughts?  Other alternatives?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia



Re: Legacy provider

2020-01-15 Thread Dr Paul Dale
I’m not sure what more I can write.

I proposed the vote text around the time I sent the notification here: no 
comments.
I created the vote, early in the voting period, the clarification was sought 
and made.
All OMC members registered their vote and the vote closed early.

The criteria for being valid as per the bylaws 
<https://www.openssl.org/policies/omc-bylaws.html> was met.  As votes go, this 
one was quick taking two days of the two weeks.

Abstentions are frequent in votes for a number of reasons.
The reasons each person uses are not revealed and not asked for.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 16 Jan 2020, at 6:07 am, Benjamin Kaduk  wrote:
> 
> Hi Pauli,
> 
> On Tue, Jan 14, 2020 at 09:34:40PM +1000, Dr Paul Dale wrote:
>> The OMC vote is closed.
>> 
>> The vote text being:
>> 
>> The legacy provider should be disabled by default in 3.0
>> 
>> With the clarification that "disabled" in this context means "not loaded”.
>> 
>> The vote passed (two for, one against, four abstain)
> 
> It's good to have a decision here, but I'm kind of worried about the four
> abstains -- it's easy for me to leap to a conclusion that the individuals
> in question just didn't want to to spend the time to come to a considered
> position, even though this issue has substantial potential impact for our
> userbase.  I'm trying to not make faulty assumptions, so some greater
> clarity on the circumstances would be helpful, if possible.
> 
> Thanks,
> 
> Ben



Legacy provider

2020-01-14 Thread Dr Paul Dale
The OMC vote is closed.

The vote text being:

The legacy provider should be disabled by default in 3.0

With the clarification that "disabled" in this context means "not loaded”.

The vote passed (two for, one against, four abstain)


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Legacy Provider

2020-01-08 Thread Dr Paul Dale
Kurt,

It’s a policy decision: should we cause pain for users (& Matt) or effectively 
delay the end for these old/broken algorithms.

Technically it is easy.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 9 Jan 2020, at 6:30 am, Kurt Roeckx  wrote:
> 
> On Tue, Jan 07, 2020 at 09:57:55AM +1000, Dr Paul Dale wrote:
>> The refactoring/FIPS work needs the question resolved about loading the 
>> legacy provider or not by default.  We’ve been through this before on the 
>> project list [1] and in at least one PR [2].
>> 
>> I expect that its resolution will require an OMC vote.
> 
> Why OMC and not OTC?
> 
> 
> 
> Kurt
> 



Legacy Provider

2020-01-06 Thread Dr Paul Dale
The refactoring/FIPS work needs the question resolved about loading the legacy 
provider or not by default.  We’ve been through this before on the project list 
[1] and in at least one PR [2].

I expect that its resolution will require an OMC vote.

Regardless of the outcome, the current intention is that the low level direct 
access functions (e.g. IDEA_encrypt) will continue to work (albeit deprecated), 
only the EVP access will go (again, by default).


Before the vote is called, are there any additional thoughts from the past six 
months?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia


[1] https://mta.openssl.org/pipermail/openssl-project/2019-July/001492.html
[2] https://github.com/openssl/openssl/pull/9373




Re: Flaw in our process for dealing with trivial changes

2019-12-12 Thread Dr Paul Dale
A better example of this problem: #10607.  Both Paul and I approved it 
yesterday and I merged it today without noticing until too late that it was 
tagged “CLA: trivial” :(
I’ve not reverted it at this point but will if necessary.

Let’s get the label in.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 13 Dec 2019, at 11:02 am, Richard Levitte  wrote:
> 
> On Thu, 12 Dec 2019 22:31:10 +0100,
> Dr Paul Dale wrote:
>> 
>> A red blocker along the lines of: “Triviality Unconfirmed”. One of
>> the reviewers needs to remove this before the PR can be merged.
>> 
>> It’s in our face, it prevent accidental merges and its low overhead.
> 
> I still think simply adding the label should be sufficient.  I dunno
> about you, but I look at labels all the time, for all sorts of
> reasons, and one saying [cla: trivial] would certainly attract my
> attention.
> 
> Let's make it bright red-orange, that'll catch anyone's eye (even mine)
> 
> Also, removing that label will rapidly be annoying as soon as someone
> closes and re-opens a PR...  or whatever other action that triggers
> the "pull_request" event (and there's a lot that does that...  our
> script is being kept busy!).
> 
> Cheers,
> Richard
> 
> -- 
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/



Re: Flaw in our process for dealing with trivial changes

2019-12-12 Thread Dr Paul Dale
A red blocker along the lines of: “Triviality Unconfirmed”. One of the 
reviewers needs to remove this before the PR can be merged.

It’s in our face, it prevent accidental merges and its low overhead.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 13 Dec 2019, at 7:06 am, Dr Paul Dale  wrote:
> 
> Before we start over engineering a solution, how about we try just having an 
> automatic visual indicator for trivial PRs.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 
>> On 13 Dec 2019, at 3:24 am, Kurt Roeckx > <mailto:k...@roeckx.be>> wrote:
>> 
>> On Thu, Dec 12, 2019 at 12:10:35PM +, Matt Caswell wrote:
>>> 
>>> But in principle I agree that addrev could be used to do this. It's not
>>> quite as robust as doing it in the commit hook - because you don't
>>> *have* to use addrev. But, AFAIK, everyone does - so that's probably
>>> good enough.
>> 
>> I have never used addrev.
>> 
>> 
>> Kurt
>> 
> 



Re: Flaw in our process for dealing with trivial changes

2019-12-12 Thread Dr Paul Dale
Before we start over engineering a solution, how about we try just having an 
automatic visual indicator for trivial PRs.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 13 Dec 2019, at 3:24 am, Kurt Roeckx  wrote:
> 
> On Thu, Dec 12, 2019 at 12:10:35PM +, Matt Caswell wrote:
>> 
>> But in principle I agree that addrev could be used to do this. It's not
>> quite as robust as doing it in the commit hook - because you don't
>> *have* to use addrev. But, AFAIK, everyone does - so that's probably
>> good enough.
> 
> I have never used addrev.
> 
> 
> Kurt
> 



Re: Flaw in our process for dealing with trivial changes

2019-12-12 Thread Dr Paul Dale
I agree that there is a possible flaw in the workflow.  What’s saved us so far 
is that new contributors don’t generally include the "CLA: trivial" line or put 
it in the GitHub text.

Could we have a “trivial” tag that is added whenever the "CLA: trivial" line is 
present?  Better would be to add it only if the submitter doesn’t have a CLA on 
file but either works.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 12 Dec 2019, at 7:20 pm, Matt Caswell  wrote:
> 
> I notice that PR 10594 (Add support for otherName:NAIRealm in output)
> got merged yesterday:
> https://github.com/openssl/openssl/pull/10594
> 
> The commit description contained "CLA: trivial" and so the "hold: cla
> required" label was not automatically applied to the PR. But the
> discussion in the PR suggested a CLA should be submitted. But it got
> merged anyway! Fortunately the CLA had already been processed - just not
> noted in the PR. So, in this case, it makes no difference.
> 
> I think this points to a possible flaw in our workflow for dealing with
> trivial changes. Because the "CLA: trivial" header suppresses the "hold:
> cla required" label and the git hooks don't complain when commits get
> pushed with the "CLA: trivial" header and no CLA on file, it seems
> possible to me that we could push commit all the way through the process
> without the reviewers even realising that the author is claiming
> triviality on the commit.
> 
> Not sure what the solution to that is.
> 
> Matt



Re: Check NULL pointers or not...

2019-11-29 Thread Dr Paul Dale
Oops, you are correct.  I was under the mistaken impression that ossl_assert 
compiled to nothing outside of debug mode.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 29 Nov 2019, at 7:22 pm, Matt Caswell  wrote:
> 
> 
> 
> On 29/11/2019 08:38, Dr Paul Dale wrote:
>> I’d prefer option 1 or the middle ground.  I’ve lost count of the
>> number of times I’ve seen programs crashing in the crypto library
>> which required mammoth debugging efforts to irrefutably demonstrate
>> that the caller is at fault rather than the crypto library :(
>> 
>> Option 1 would be preferable from this point of view but it can cause
>> a performance hit — most of the time it wouldn’t matter but when it
>> does it would be a big deal.  The middle ground doesn’t entail any
>> performance loss in production code (it does in debug but that
>> shouldn’t be relevant).
> 
> 
> I think you misunderstand the middle ground option:
> 
>if (!ossl_assert(ptr != NULL)) {
>ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER);
>return 0;
>}
> 
> In debug code this will crash if ptr is NULL. In production code this
> acts exactly like option 1 - so has exactly the same performance hit.
> 
> For the record my preference is the middle ground option as being the
> norm for new code and where we make a significant refactor of old code.
> If something truly is performance critical then we can choose not to do
> it in those cases.
> 
> Matt
> 



Re: Check NULL pointers or not...

2019-11-29 Thread Dr Paul Dale
I’d prefer option 1 or the middle ground.  I’ve lost count of the number of 
times I’ve seen programs crashing in the crypto library which required mammoth 
debugging efforts to irrefutably demonstrate that the caller is at fault rather 
than the crypto library :(

Option 1 would be preferable from this point of view but it can cause a 
performance hit — most of the time it wouldn’t matter but when it does it would 
be a big deal.  The middle ground doesn’t entail any performance loss in 
production code (it does in debug but that shouldn’t be relevant).


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On Wed, Nov 27, 2019 at 10:38:41AM +0100, Richard Levitte wrote:
>> Depending on who you're asking, you get different responses.
>> 
>> The topic is: should we check pointers coming from applications before
>> using them or not?
>> 
>> There are two positions on this:
>> 
>> 1. Yes we should so we don't crash processes unnecessarily, and we
>>   should return an error if pointers are NULL (unless that's valid
>>   input, such as for the freeing functions).  There is a generic
>>   error reason code for it, ERR_R_PASSED_NULL_PARAMETER.
>> 
>> 2. No we should not, as we don't cater for user programming errors.
>>   Also, it allows us to catch our own errors early.
>> 
>> For a while, our pendulum was going with option 2, but lately, we seem
>> to go back to option 1.
>> 
>> Both options have their pros and cons:
>> 
>> 1. cons: there's the very real possibility that users don't check for
>> errors as they should, and things go wrong further on,
>> sometimes in ways where the actual cause is very hard to
>> figure out.
>>   pros: programs will not spuriously crash because we didn't catch an
>> internal corner case in our tests.
>> 
>> 2. cons: programs may crash, sometimes due to our own mistakes,
>> sometimes due to user errors.
>>   pros: some very subtle bugs are found, either quicker or at all.
>> An actual case is PR 7630 [1], which was uncovered because
>> there was a crash due to a NULL being used (this was
>> ultimately due to users not checking errors).
>> 
>> As a middle ground, and perhaps a way to satify both camps, we could
>> use ossl_assert() in our checks for input NULL, as follows:
>> 
>>if (!ossl_assert(ptr != NULL)) {
>>ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER);
>>return 0;
>>}
>> 
>> This means that in a debug build, there may be crashes due to pointers
>> being NULL, while in a release build, the caller gets an error.
>> 
>> Thoughts?




Re: Malloc failures check

2019-11-20 Thread Dr Paul Dale

Adding a compile time check would be good.  I’m not sure how.
It would be possible to implement a malloc failure feature in the test suite 
that systematically runs a test many times, failing successive malloc calls.

I’m kind of surprised that the various static analysers hadn't found some of 
these.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 21 Nov 2019, at 1:26 pm, Dmitry Belyavsky  wrote:
> 
> Hello,
> 
> Observing a series of similar bugs related to a lack of checks of the malloc 
> return values, I wonder if we could automate the search of these errors on 
> the compile level (e.g. similar to the __owur macro)?
> 
> -- 
> SY, Dmitry Belyavsky



Re: #10388

2019-11-15 Thread Dr Paul Dale
The consensus seems to be to add the deprecated API to 3.0.

I’ve removed the hold.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 15 Nov 2019, at 10:40 pm, Matthias St. Pierre 
>  wrote:
> 
> 
> 
> On 14.11.19 20:40, Viktor Dukhovni wrote:
>>> On Nov 14, 2019, at 9:15 AM, Matt Caswell  wrote:
>>> 
>>> "No existing public interface can be removed until its replacement has
>>> been in place in an LTS stable release. The original interface must also
>>> have been documented as deprecated for at least 5 years. A public
>>> interface is any function, structure or macro declared in a public
>>> header file."
>>> 
>>> So the functions we're talking about don't yet have a replacement -
>>> there will be one in 3.0. So presumably they will be documented as
>>> deprecated in 3.0. So we have to support them for at least 5 years from
>>> the release of 3.0.
>> I think that when we're deprecating an entire family of *related* accessors
>> (for the same structure) that have been in place for some time, the addition
>> of a missing member of that family is reasonably interpreted as not resetting
>> the support clock on the entire family.  We can still remove them all as 
>> though
>> the missing members of the family had been in place all along.
>> 
>> That is, we should (and if that requires a policy update and vote so be it) 
>> be
>> able to interpret the rules based on the "spirit" (intent) without getting
>> unduly burdened by the "letter", where common sense would suggest that we're
>> getting the intended outcome.
> 
> I don't think we need a reinterpretation, or even a change, of the existing 
> policy for
> this specific case,  since already now the *entire* engine api can only be 
> deprecated
> in version 3.0 and removed afterwards according to the policy cited by Matt.
> 
> We can simply add the missing accessor as bugfix to 1.1.1 and 3.0, and 
> immediately
> deprecate it on the latter. The only difference will be that we end up with 
> n+1 deprecated
> functions instead of n  (for some natural number n) for version 3.0.
> 
> Even after 3.0 is released and as long as 1.1.1 LTS is still supported, we 
> can proceed
> in the same way without violating existing policies.
> 
> 
> Matthias



#10388

2019-11-13 Thread Dr Paul Dale
With reference to #10388: https://github.com/openssl/openssl/pull/10388 
<https://github.com/openssl/openssl/pull/10388>

This PR intends to add missing APIs to 1.1.1.  I’m fine with this being 
considered a bug and adding them.


I’m more concerned about adding these to 3.0.  It means we’ll have to support 
the new API for a long time and it is one which we are currently trying to move 
away from.


Thoughts or comments anyone?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Commit access to openssl/tools and openssl/web

2019-10-04 Thread Dr Paul Dale
I believed that it required two OMC approvals but was pointed to an earlier 
instance where only one was present and I flew with it without checking further.
My apologies for merging prematurely and I’ll back out the changes if any OMC 
member wants.

As for discussing this at the upcoming face to face, I agree wholeheartedly.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 4 Oct 2019, at 5:39 pm, Matt Caswell  wrote:
> 
> 
> 
> On 04/10/2019 08:15, Dr. Matthias St. Pierre wrote:
>> Dear OMC,
>> 
>> while the process of merging and committing to openssl/openssl has been 
>> formalized,
>> no similar (official) rules for pull requests by non-OMC-member seem to 
>> apply to the
>> other two repositories openssl/tools and openssl/web. Probably it's because 
>> hardly
>> anybody outside the OMC else ever raises them? Or is it the other way around?
> 
> There are clear official rules. This vote was passed by the OMC over a year 
> ago:
> 
> topic: Openssl-web and tools repositories shall be under the same review
>   policy as per the openssl repository where the reviewers are OMC members
> 
> So it needs two approvals from an OMC member. It looks like recent commits
> haven't obeyed those rules.
> 
> 
>> I would like to raise the question whether it wouldn't be beneficial for all 
>> of us,
>> if we would apply the same rules (commit access for all committers, plus the 
>> well
>> known approval rules) to all of our repos. After all, the openssl/openssl 
>> repository
>> is the most valuable of the three and I see no reason why the others would 
>> need
>> more protection. In the case of the openssl/web repository which targets the
>> official website, you might want to consider a 2OMC approval rule, but even 
>> there
>> I don't see why the usual OMC veto rule wouldn't be sufficient.
> 
> There is a lot of merit in that. Certainly for tools. I've added it to the OMC
> agenda for Nuremburg.
> 
> Matt
> 



Re: Reorganization of the header files (GitHub #9333)

2019-09-28 Thread Dr Paul Dale
Go for it, the antipodean contingent aren’t busy this weekend.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 28 Sep 2019, at 5:05 pm, Dr. Matthias St. Pierre 
>  wrote:
> 
>> Merge early is pretty much my default position ... and that applies to this 
>> context in my view.
> 
> Thank you Tim for your quick and  clear consent.
> 
> Meanwhile, pull request #9333 and its counterpart #9681 for the 1.1.1 stable
> branch have been approved by Richard. Unless I hear any protest until Sunday
> afternoon (CEST), I would like to merge in the evening, just in time before 
> the
> Australian plumbers start their weekly shift again.
> 
> Matthias



Re: RAND, FIPS and providers

2019-09-24 Thread Dr Paul Dale
Matt, thanks for the clarification.  I’ve looked at the DRBG setup code dozens 
of times and it never clicked.

It seems we’re down to making the DRBGs and, perhaps, the seed source available 
using fetch.
That doesn’t seem anything like as difficult.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 24 Sep 2019, at 6:32 pm, Matt Caswell  wrote:
> 
> 
> 
> On 24/09/2019 01:17, Dr Paul Dale wrote:
>> This started in #9954, the topic of RAND being used by the legacy provider 
>> came up (in the context of DES).  The abridged version is:
>> 
>> * @levitte suggested the possibility of making RAND detachable.
>> * I noted that this was desirable and in fact necessary for FIPS.
>> * @mattcaswell added that the DRBGs and seeding is available inside the FIPS 
>> provider.
>> 
>> 
>> That the FIPS provider includes a copy of the relevant RAND files, means it 
>> can satisfy internal requests for random numbers.
>> However, external entities (TLS stack, user applications) won’t git FIPS 
>> approved random numbers.
>> 
>> I can’t see currently an alternative to making the RAND functionality 
>> fetchable.
> 
> I think making RAND fetchable is highly desirable and should be done (I had
> always assumed we would do this).
> 
>> I also suspect it will need to be per library context which might interfere 
>> with the per thread DRBGs we’re using.
> 
> I see no problems here. The RAND code is already library context aware. You 
> get
> per thread DRBGs for each OPENSSL_CTX. For example calling
> OPENSSL_CTX_get0_private_drbg() will get you the private DRBG for the current
> thread in the specified OPENSSL_CTX. RAND_DRBG_get0_private() does the same
> thing for the default OPENSSL_CTX.
> 
> 
>> As for what to fetch: the DRBG instances and the seed material source would 
>> be ideal, although we don’t need the seed source for FIPS (so long as the 
>> DRBGs seed from inside their own provider).
> 
> I had always assumed we would fetch DRBG instances.
> 
> Matt
> 
>> 
>> 
>> Thoughts or input anyone?
>> 
>> 
>> Pauli
>> 



RAND, FIPS and providers

2019-09-23 Thread Dr Paul Dale
This started in #9954, the topic of RAND being used by the legacy provider came 
up (in the context of DES).  The abridged version is:

* @levitte suggested the possibility of making RAND detachable.
* I noted that this was desirable and in fact necessary for FIPS.
* @mattcaswell added that the DRBGs and seeding is available inside the FIPS 
provider.


That the FIPS provider includes a copy of the relevant RAND files, means it can 
satisfy internal requests for random numbers.
However, external entities (TLS stack, user applications) won’t git FIPS 
approved random numbers.

I can’t see currently an alternative to making the RAND functionality 
fetchable.  I also suspect it will need to be per library context which might 
interfere with the per thread DRBGs we’re using.

As for what to fetch: the DRBG instances and the seed material source would be 
ideal, although we don’t need the seed source for FIPS (so long as the DRBGs 
seed from inside their own provider).


Thoughts or input anyone?


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia






Re: Being socially aware

2019-09-17 Thread Dr Paul Dale
I’m not disputing the great effort put into this.
My dispute is that it should be under the openssl list command…..

I agree, this shouldn’t have been a “good first issue”.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 17 Sep 2019, at 3:49 pm, Richard Levitte  wrote:
> 
> ...  or how our usual style of dispute can sometimes deter help from
> the community.
> 
> https://github.com/openssl/openssl/pull/9912
> 
> In this PR, there's a lot of discussions going on, a bit back an
> forth, about the right way to do things, and what does what like what
> other thing and so on.
> 
> This is often our modus operandi, and it has often given us pretty
> good quality code (or at least, so we hope), even though it can be a
> bit exhausting at times.
> 
> I would like to emphasise that I think this is absolutely fine...
> when it's among us who have been along for a bit, perhaps have come to
> know each other a bit more and have some kind of sense of our
> respective strengths and weaknesses, or even with someone that has
> shown they can handle this type of discussion.
> 
> However, in pull requests like the one cited, where the associated
> issue is marked "good first issue", and the author has done quite well
> what was asked from the issue, it can be quite unexpected, not to say
> overwhelming to be met with this discussion style that could be seen
> as getting out of proportion.  A "good first issue" is supposed to be
> a bite size thing after all, and I fear that if this is how we welcome
> new contributors, they might more often than not choose to step away
> from it all.
> 
> So maybe let's be a little more careful with contributions for "good
> first issue" and potential newcomers, yeah?
> 
> Cheers,
> Richard
> 
> -- 
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/



Re: Thread sanitiser problems

2019-07-31 Thread Dr Paul Dale
The locks aren’t being held for long.

I have noticed that the lock in the OPENSSL_CTX only seems to be locked for 
reading and only in one place.  crypto/context.c in openssl_ctx_get_data().  
Writes seem to go via CRYPTO_alloc_ex_data() which uses a global write lock (on 
a different lock).  Would it be feasible to drop the OPENSSL_CTX lock 
completely?  This would address the issue.  Matt?  I think you know this code 
the best.

CRYPTO_alloc_ex_data() is calling the new_func pointer under the read lock and 
the new function can in turn cause a search of the providers for 
implementations which grabs the provider lock.  Conversely, searching the 
providers for implementations grabs the provider lock and can lead to 
openssl_ctx_get_data() grabbing the context lock.

We don’t appear to have recursion at the moment, but the setup seems a little 
fragile.  We do have two different ordering for grabbing locks which is also 
bad.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 31 Jul 2019, at 2:10 pm, Viktor Dukhovni  
> wrote:
> 
>> On Jul 30, 2019, at 10:02 PM, Dr Paul Dale  wrote:
>> 
>> The #9454 description includes thread sanitisizer logs showing different 
>> lock orderings — this has the potential to dead lock.  Agreed with Rich that 
>> giving up the lock would make sense, but I don’t see a way for this to be 
>> easily done.
> 
> My take is that we should never hold any lock long enough to even consider
> acquiring another lock.  No more than one lock should be held at any one
> time, and only long enough to bump a reference count to ensure that the
> object of interest is no deallocated before we (or our caller in a "get1"
> type interface) is done with it.
> 
> I don't know what "long-term" locks we're holding, but it would be great
> if it were possible to never (or never recursively) hold any such locks.
> 
> -- 
>   Viktor.
> 



Re: Thread sanitiser problems

2019-07-30 Thread Dr Paul Dale
Yes, I’m mostly talking about #9454 here.  #9455 is a bug (clearing the flush 
flag after flushing not before).  The fix in #9477 addresses this and also 
removes the dependence on RAND_bytes.

The #9454 description includes thread sanitisizer logs showing different lock 
orderings — this has the potential to dead lock.  Agreed with Rich that giving 
up the lock would make sense, but I don’t see a way for this to be easily done.

That this also involves the RAND subsystem could be coincidental.  The other 
stack trace is getting an MD via a digest operation (i.e. both traces are for 
algorithms that use other algorithms).  The locks in question are the provider 
store lock and the context lock.

crypto/context.c:
OPENSSL_CTX *ctx; /* function argument */
CRYPTO_THREAD_read_lock(ctx->oncelock);

crypto/provider_core.c:
OPENSSL_CTX *ctx; /* function argument */
struct provider_store_st *store = get_provider_store(ctx);
CRYPTO_THREAD_read_lock(store->lock);


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 30 Jul 2019, at 8:52 pm, Matthias St. Pierre 
>  wrote:
> 
> Sorry, my reply was misleading, since Pauli is talking mainly about #9454.
> Please take a look at the issue description
> 
> https://github.com/openssl/openssl/issues/9454
> 
> instead.
> 
> Matthias
> 
> 
> 
> On 30.07.19 12:47, Matthias St. Pierre wrote:
>> 
>> 
>> On 30.07.19 12:43, Kurt Roeckx wrote:
>>> 
>>> I currently fail to see how that's a problem, unless that
>>> EVP_CIPHER_CTX tries to use a DRBG.
>> 
>> This is what I mean when I say that things have gotten more complicated 
>> under the hood
>> due to the replumbing. To understand the problem, please take at a look of 
>> the sanitizer
>> callstack in
>> 
>> https://github.com/openssl/openssl/issues/9455
>> 
>> 
>> Matthias
>> 
>> 
>> 
> 



Thread sanitiser problems

2019-07-29 Thread Dr Paul Dale
Bringing the discussions over to the project list.

The problem is initially mentioned in #9454.  Followup issues with infinite 
recursion and use after free also get mentioned in #9455.  These are addressed 
by #9477 which clears the flush flag before flushing and removes the dependence 
on the RAND call.  Reproduction seems to require gcc-8, gcc-7 doesn’t have the 
appropriate thread sanitisation.

Overly simplified, the problem boils down to the CTR DRBG needing an AES CTR 
cipher context to work.  When creating the former, a recursive call is made to 
get the latter.  A deadlock results due to a cycle in the locking.  The 
RAND/DRBG code will not be the only place where this occurs.  KDF, MAC and some 
public key operations will suffer a similar issue.

This is almost certainly going to hurt moving forward.


On to possible mitigations:

1. Make our locks recursive.  Add a reference counter and a thread ID to the 
lock structure.  There would be no need to make either use atomic operations.  
Getting a read lock after a write lock would be considered an extra write lock, 
handing a write lock while holding a read lock could be problematic.  This 
seems very messy.

2. Return dependent algorithms as part of the registration process.  The 
particular algorithm could be preloaded somehow.  I’m not sure how ugly this 
will become but it will need names (nids) for each possible DRBG type.



Thoughts anyone?
Any better solutions?
Any other solutions?


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



Vote on PR

2019-07-07 Thread Dr Paul Dale
The following vote passed 6 to 0.

topic: Accept the changes to the OpenSSL policies as per PR#133 (openssl/web).
comment: The definition of trivial being clarified and moved to the web page
 that the missing CLA note references.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia





Re: Removing function names from errors (PR 9058)

2019-06-13 Thread Dr Paul Dale
I’m behind ditching the function identifier #defines but not their text names.


The varargs based error function seems like a good idea too.  It would be nicer 
if the ERR_raise_error function included the file and line information without 
them needing to be specified.  As an exercise for the reader, I’ll skip any 
implementation details but will note that it is possible without using variadic 
macros.


Pauli


Hint:
#define ERR_raise_error ERR_raise_error_internal(__FILE__, __LINE__, __FUNC__)
int (*)(int, const char *, ...) ERR_raise_error_internal(const char *, int, 
const char *);
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 14 Jun 2019, at 12:04 pm, Viktor Dukhovni  
> wrote:
> 
> On Wed, Jun 12, 2019 at 05:51:44AM +0200, Richard Levitte wrote:
> 
>> A discussion point in that PR is whether it's still interesting to
>> keep information on the system function/callback that was called when
>> we're reporting a system error, i.e. this type of error report:
>> 
>>SYSerr(SYS_F_FSTAT, errno);
>> 
>> (incidently, there's another PR, 9072, which changes those to
>> 'SYSerr("fstat", errno)')
>> 
>> So, the main points of discussion seem to be:
>> 
>> - should we remove function indicators in our error reporting?
>> - should we make an exception for system errors (SYSerr())?
> 
> Bottom line, my take is that function indicators should stay, but
> string names would be more convenient all around than numeric codes.
> 
> -- 
>   Viktor.



Start up entropy gathering

2019-06-13 Thread Dr Paul Dale
Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on 
boot and first boot problems.
Comment: PR#9084 removed such mitigation due to the negative side effects.

This vote has passed (two for, four abstain, none against):


The discussion now needs to turn to how this can be done.

The mitigation is for older Linux kernels only (and perhaps old Unix machines 
of other flavours).  So no Windows, getentropy(3), ….


I have a suggestion.  Two kind of.

Use a volatile mechanism for communicating between processes — a file in /tmp, 
a block of shared memory or whatever.
On system boot, the first instance of libcrypto that attempts to read 
/dev/urandom, first locks the shared resource and reads one byte from 
/dev/random.  It uses the shared resource to indicate that the read was 
successful and continues seeding from /dev/urandom.
Other instances of libcrypto start up, see that the shared resource is properly 
initialised and go straight to using /dev/urandom.
On system shutdown/reboot, the communication mechanism disappears and the 
entire process will repeat on the next boot.
There are some fine details but this is the gist of it.

This is more difficult to describe than code: see #9152 
<https://github.com/openssl/openssl/pull/9152>.


Notes:

1. Reading the byte from /dev/random avoid problems on systems that cannot 
select(2) on /dev/random.

2. Early start up of /libcrypto will block until entropy is available (and 
consume eight bits of same).  Later starts have the overhead of the locking and 
resource access.  Importantly, no entropy will be drained from the system pool 
after start up.



The second suggestion is broadly similar but requires a file containing entropy 
that persists across reboots.  This alternative requires a more management: the 
entropy file once read needs to be rewritten immediately (and ideally on 
shutdown as well).  It also introduces a new attack vector against the entropy 
storage.  It also isn’t possible to skip the entropy file read/rewrite sequence 
because it is impossible to determine if /dev/urandom has actually been seeded. 
 I’ve not attempted to code this, persistent files containing seed material 
potentially introduce other problems.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia





Re: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT

2019-06-07 Thread Dr Paul Dale
The devices where I’ve experienced long start up delays didn’t all have a HRNG. 
 The rng-tools wouldn’t have achieved anything.

ColdFire based routers running a 2.0 Linux kernel were the first time I 
personally saw this one.  uClinux still supported 2.0 kernels last I checked — 
they are very small and relatively fast.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 8 Jun 2019, at 5:25 am, Kurt Roeckx  wrote:
> 
> On Fri, Jun 07, 2019 at 03:08:24PM -0400, Viktor Dukhovni wrote:
>>> On Jun 7, 2019, at 2:41 PM, Kurt Roeckx  wrote:
>>> 
>>>> This is not the sort of thing to bolt into the kernel, but is not
>>>> unreasonable for systemd and the like.
>>> 
>>> The kernel actually already does this in recent versions, if
>>> configured to do it.
>> 
>> We're talking about what to do with for older kernels.
> 
> For older kernels you install rng-tools that feeds the hwrng in
> the kernel.
> 
> 
> Kurt
> 



VOTE Apply PR#9084 reverting DEVRANDOM_WAIT

2019-06-07 Thread Dr Paul Dale
This vote has been closed, it passed 5 votes to 2 with no abstentions.


Up for discussion is the text of the next vote.  I’m proposing this:

Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on 
boot and first boot problems.
Comment: PR#9084 removed such mitigation due to the negative side effects.


I’ll make this formal in a day or so, so if anyone wants to suggest alternative 
wording, that’s the time line.  The vote text is the “topic” line, the comment 
is explanatory only.

Note: I’m not mentioning the mechanism used, that still needs to be decided on. 
 This is just saying that 3.0.0 *will* have some mechanism.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia





Re: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT

2019-06-07 Thread Dr Paul Dale

Storing entropy has risks — cloning attacks, reading the file attacks, etc.  It 
is better trying than doing nothing.  Counting such stored entropy as 
additional data is good but many systems have no alternative entropy available. 
 I have had to introduce such storing schemes (with generation during 
fulfilment) when devices would take well over ten minutes to find enough 
entropy to generate their initial (long term) key material.  There are ways to 
mitigate the reuse risk — lock the entropy file, read it, add the contents to 
the kernel, regenerate the file and finally unlock it.  Plus regenerating the 
contents and storing them on shutdown (if not periodically).  A cloned VM would 
still be vulnerable (at least for a while).

Also agreed for an option to disable this.  I’d prefer that it is on by default 
(i.e. secure by default and requiring a decision to not be secure).


I’m expecting a somewhat lively discussion about a sensitive topic :)


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 7 Jun 2019, at 6:18 pm, Tomas Mraz  wrote:
> 
> On Fri, 2019-06-07 at 18:03 +1000, Dr Paul Dale wrote:
>> Mark, I agree that the project area would be better for this
>> discussion.
>> 
>> For those not involved, this revolves around the vote to merge
>> PR#9084 and possible alternate mitigations for the underlying
>> problem.
>> 
>> 
>> Tim’s explanatory message is:
>> 
>>> For the record, I think we need to do a better job of tackling the
>>> underlying issue.
>>> Dealing with systems on first boot with no entropy is an issue for
>>> a class of systems.
>>> However we should be distinguishing between contexts where we have
>>> decent seeding material (from having had reasonable entropy) from
>>> those where we haven't.
>>> We also should be able to persist seeding across boots and have
>>> recommendations for how vendors should do that.
>>> We should be able to fall back to handling it at the user level
>>> (i.e. non-root user, persisted in the file system).
>>> We need knowledge of install state (for FIPS) and that can also act
>>> as a first-boot state effectively.
>>> 
>>> And this applies equally to getentropy usage - it is only a problem
>>> in very limited contexts where blocking actually makes sense to
>>> perform.
>>> That the kernel has no additional entropy available is not
>>> necessarily an issue - except in a very limited context (first time
>>> boot with no other entropy available from previous contexts). 
> 
> The previous context can easily provide false entropy - for example due
> to the booting system being a cloned virtual machine. Or boot image for
> a small router device which contains such saved entropy which is
> distributed to millions of devices worldwide.
> 
> This was the reason why systemd guys rejected the idea to actually seed
> the kernel rng with such entropy with the call that counts this seed
> entropy. They just use it as additional seed with no accounting for the
> entropy it would provide if it was not reused.
> 
>> 
>> Viktor replied:
>> 
>>> I just want to make sure we don't lock ourselves in to a single
>>> potentially clumsy solution in the library.  Various strategies
>>> may be appropriate depending on the platform, and ultimately the
>>> cooperation of the system administrator to enable the required
>>> mechanisms.
>>> 
>>> Potential additional sources for initial entropy on systems
>>> without getrandom(2) might include:
>>> 
>>>  RDSEED/RDRAND
>>>  TPM (or Virtualized TPM which is sometimes better!)
>>>  RNG state saved across reboots.
>>>  ...
>>> 
>>> This requires knowledge of various platforms, and potential
>>> help from the platform vendors.  It might, for example, make
>>> sense to delegate initial seeding to systemd on platforms
>>> that use systemd, and work with the systemd maintainers to
>>> provide a set of sensible entropy sources for initial boot.
>>> 
>>> Exposing a decent RNG to virtual guests and containers is
>>> would be another area of focus.
> 
> From the point of view of distribution maintainer of OpenSSL I would
> say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT had
> no real problems for us.
> 
> However I can understand that when OpenSSL is built and installed
> separately by application vendors on some legacy system that they do
> not have control of they might appreciate that the library itself tries
> to solve the

Re: OSSL_PARAMs

2019-06-04 Thread Dr Paul Dale
The OSSL_PARAM structure needs to be visible and not subject to change.
Providers shouldn’t necessarily have a dependency on functions from libcrypto.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 5 Jun 2019, at 12:47 pm, SHANE LONTIS  wrote:
> 
> 
> 
>> On 5 Jun 2019, at 12:34 pm, Richard Levitte > <mailto:levi...@openssl.org>> wrote:
>> 
>> Aside from the discussion below, if there's one thing I would like to
>> change, it the double indirection for the _PTR data types.  The data
>> types could still be used to indicate that the value isn't short
>> lived, but could possibly change names to something like
>> OSSL_PARAM_UTF8_CSTRING and OSSL_PARAM_OCTET_CSTRING (C for Constant).
>> 
>> On Wed, 05 Jun 2019 01:18:56 +0200,
>> Dr Paul Dale wrote:
>>> Shane’s major complaints are about the indirection the OSSL_PARAM structure 
>>> forces — for integers
>>> and return lengths and the necessity of allocating additional memory in 
>>> parallel with the
>>> OSSL_PARAM.
>>> 
>>> The extra indirection was intended to support const arrays of OSSL_PARAM, 
>>> which turn out to be a
>>> rarity because they aren’t thread safe.
>> 
>> The reason why we have this issue is our base C language version
>> choice.  C90 doesn't allow this construct:
>> 
>>int foo(whatever)
>>{
>>int haha = 0;
>>const OSSL_PARAM params[] = {
>>{ 'foo', OSSL_PARAM_INTEGER, , sizeof(haha), NULL },
>>{ NULL, 0, NULL, 0, NULL }
>>};
>> 
>>...
>>}
>> 
> The above code is great in theory, but it looks like in practice we end up 
> dynamically allocating in most cases anyway (via the construct_ methods).
> And if this is the normal use case then OSSL_PARAMS could be made opaque and 
> only accessed by API’s, then the argument about adding
> extra types later on should also disappear?
> 
> 
>> Because the compiler for that language version isn't allowed to emit
>> code to use '' in an inititializer.  Newer C language versions
>> allow this.
>> 
>> So while this is an issue for *us*, it isn't necessarily an issue for
>> our users, all depending on what C language version they use.
>> 
>>> With most OSSL_PARAM structure being dynamically created,
>>> the need for the indirection seems redundant.  E.g. could the return length 
>>> be moved into
>>> OSSL_PARAM?  I think so.
>> 
>> The design was not only to be able to have nice compile time
>> initialization, but also to be able to pass the array as 'const
>> OSSL_PARAM *', i.e. an indication to the recipient that the array
>> itself should never be modified (less chance of compromise).  Maybe
>> that's overly paranoid, but that was a line of thinking.
>> 
>>> Moving integral values into the structure is more difficult because BIGNUMs 
>>> will always need to be
>>> references.  Allocating additional memory will still be required.  I’ve got 
>>> three obvious
>>> solutions:
>>> 
>>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when 
>>> the structure is
>>> destroyed or
>> 
>> It's actually perfectly possible to do this today.  We already have
>> this pointer, it's called 'data'.
>> 
>>> 2. have a block of data in the OSSL_PARAM structure that can be used for 
>>> native types
>>> (OSSL_UNION_ALIGN works perfectly for this) or
>> 
>> My major concern with that, apart from having to modify the OSSL_PARAM
>> items themselves¸ is that some time in the future, we will want to add
>> another native type that's larger, which means we modify the size of a
>> OSSL_PARAM.  It's a public structure, so that can't be treated
>> lightly.
>> 
>> Also, with a union of native types, we're losing uniformity on MSB
>> first platforms.  Having an exact 1:1 integer size match will be
>> crucial, and that complicates the code quite a bit...  not to mention
>> that we have a compatibility problem as soon as one end has a new
>> native type in the union and the other doesn't.
>> (one would imagine that simply using uintmax_t would cover all integer
>> sizes apart from BIGNUM, but the potential size change of that type
>> with newer compilers make such a choice precarious)
>> 
>>> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value 
>>> needs to be freed.
>> 
>> By whom?  The owner of the ar

Re: OSSL_PARAMs

2019-06-04 Thread Dr Paul Dale



-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 5 Jun 2019, at 12:47 pm, Richard Levitte  wrote:

> But you're talking about allocating the whole OSSL_PARAM array on the
> heap, aren't you?  While not structly opposed (even though I think
> that's wasteful), I think this should be done with a separate set of
> functions (intuitively, having names with '_alloc_' rather than
> '_construct_' could be an idea).

Not the whole OSSL_PARAM array, just the data pointed to by (some) of the 
elements of one.
_alloc_ instead of _construct_ makes sense and is better than NULL.

I’m fine with `OSSL_PARAM params[20]` in code and filling as many slots as 
desired.

To do entirely heap allocated arrays, we’d want to go via a stack (which is 
something we shouldn’t require providers to depend on) or use a linked list of 
some kind.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia





Re: OSSL_PARAMs

2019-06-04 Thread Dr Paul Dale
Richard wrote:
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> So while this is an issue for *us*, it isn't necessarily an issue for
> our users, all depending on what C language version they use.

Supporting things *we* can’t use seems a little odd.  The alternative 
construction is about as onerous (the construct calls).  On stack arrays won’t 
provide much performance benefit.  Static arrays would but without thread 
safety.


>> With most OSSL_PARAM structure being dynamically created,
>> the need for the indirection seems redundant.  E.g. could the return length 
>> be moved into
>> OSSL_PARAM?  I think so.
> 
> The design was not only to be able to have nice compile time
> initialization, but also to be able to pass the array as 'const
> OSSL_PARAM *', i.e. an indication to the recipient that the array
> itself should never be modified (less chance of compromise).  Maybe
> that's overly paranoid, but that was a line of thinking.

This is a better reason, not that I think “const” is all that useful here.

An aside: having a const struct that has a pointer to non-const memory isn’t 
entirely obvious to many.  This is a public API, make it as simple as necessary.


>> Moving integral values into the structure is more difficult because BIGNUMs 
>> will always need to be
>> references.  Allocating additional memory will still be required.  I’ve got 
>> three obvious
>> solutions:
>> 
>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when 
>> the structure is
>> destroyed or
> 
> It's actually perfectly possible to do this today.  We already have
> this pointer, it's called 'data’.

And how is a pointer known to be malloced or not?  I’m trying to make this easy 
for users without losing the efficiency that is possible if it is required 
somewhere.

I’m looking at making this kind of code work:

OSSL_PARAMS params[10];
int n = 0, max_len;
char my_size[20];

scanf(“%d”, _len);
params[n++] = OSSL_PARAM_construct_utf8_string(“size”, _size, 
sizeof(my_size), NULL);
params[n++] = OSSL_PARAM_construct_utf8_string(“name”, NULL, max_len, NULL);
params[n++] = OSSL_PARAM_construct_end();

…

OSSL_PARAM_deconstruct_all(params);

It is a contrived case but I think it would make using the params easier.


>> 2. have a block of data in the OSSL_PARAM structure that can be used for 
>> native types
>> (OSSL_UNION_ALIGN works perfectly for this) or
> 
> My major concern with that, apart from having to modify the OSSL_PARAM
> items themselves¸ is that some time in the future, we will want to add
> another native type that's larger, which means we modify the size of a
> OSSL_PARAM.  It's a public structure, so that can't be treated
> lightly.

This is a valid concern.  OSSL_UNION_ALIGN isn’t appropriate.
Likewise, uintmax_t isn’t binary compatible.


> If you're thinking that the receiving side should free certain values,
> then you need to pass a pointer to the routine to be used to free the
> value rather than just a flag.

I agree, this is a bad idea.  

The API isn’t easy to use at the moment.  It is also error prone as soon as 
something complex is encountered (which we haven’t yet).
Shane struggled to figure this out and get it working when trying the KDFs and 
he is far more experienced than most.


I’ll see if I can get some kind of diff or PR together.


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia




Re: OSSL_PARAMs

2019-06-04 Thread Dr Paul Dale
For a minimally invasive change, the various OSSL_PARAM_construct_ calls could 
be modified to allocate space for the parameter and its return size if NULL is 
passed for either.
There would need to be some indication of this and a cleanup array call.

Done properly, this would permit the same efficiency on the owner’s side but 
would also simplify OSSL_PARAM array creation for those who aren’t as concerned.
The theoretical const array of OSSL_PARAM would still be possible.


Is this workable or should something more significantly different be used 
before things freeze with the 3.0 release?


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 5 Jun 2019, at 10:50 am, Dr Paul Dale  wrote:
> 
> I thought the references were to allow const arrays of OSSL_PARAM to be 
> viable.
> 
> A quick check through the code reveals these in test and doc only.  There are 
> two instances of OSSL_PARAM arrays being declared in evp, both add the 
> pointed to variable after creation, both only have two elements (the integer 
> and the terminator) and both are stack allocated.  I.e. there is currently is 
> no example of the use case for which the indirection is present :(
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Cryptographer | Network Security & Encryption 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
>> On 5 Jun 2019, at 10:31 am, SHANE LONTIS > <mailto:shane.lon...@oracle.com>> wrote:
>> 
>> I presume the reference approach was used to solve the issue of who actually 
>> owns/free's the data.
>>  
>> 
>>> On 5 Jun 2019, at 9:18 am, Dr Paul Dale >> <mailto:paul.d...@oracle.com>> wrote:
>>> 
>>> Shane’s major complaints are about the indirection the OSSL_PARAM structure 
>>> forces — for integers and return lengths and the necessity of allocating 
>>> additional memory in parallel with the OSSL_PARAM.
>>> 
>>> The extra indirection was intended to support const arrays of OSSL_PARAM, 
>>> which turn out to be a rarity because they aren’t thread safe.  With most 
>>> OSSL_PARAM structure being dynamically created, the need for the 
>>> indirection seems redundant.  E.g. could the return length be moved into 
>>> OSSL_PARAM?  I think so.
>>> 
>>> Moving integral values into the structure is more difficult because BIGNUMs 
>>> will always need to be references.  Allocating additional memory will still 
>>> be required.  I’ve got three obvious solutions:
>>> 
>>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when 
>>> the structure is destroyed or
>>> 2. have a block of data in the OSSL_PARAM structure that can be used for 
>>> native types (OSSL_UNION_ALIGN works perfectly for this) or
>>> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value 
>>> needs to be freed.
>>> 
>>> The memory allocation comes to the for when reading e.g. a file and 
>>> extracting data — either the reader needs a lot of local variables to hold 
>>> everything or it has to allocated for each.  The file’s data is transient 
>>> in memory.
>>> 
>>> 
>>> For the most part, the receiver side APIs seem reasonable.  It is the 
>>> owning side that has the complications.
>>> 
>>> I think I might be able come up with some owner side routines that assist 
>>> here but allowing changes to the params structure would be far easier.
>>> 
>>> 
>>> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl 
>>> functions if not ctrl as well (subject to backward compatibility concerns).
>>> 
>>> 
>>> Pauli
>>> -- 
>>> Dr Paul Dale | Cryptographer | Network Security & Encryption 
>>> Phone +61 7 3031 7217
>>> Oracle Australia
>>> 
>>> 
>>> 
>>>> On 4 Jun 2019, at 11:26 pm, Richard Levitte >>> <mailto:levi...@openssl.org>> wrote:
>>>> 
>>>> On Tue, 04 Jun 2019 14:57:00 +0200,
>>>> Salz, Rich wrote:
>>>>> 
>>>>> 
>>>>>>   Part of the idea was that this would be a means of communication
>>>>>between application and provider, just like controls are with
>>>>>libcrypto sub-systems.
>>>>> 
>>>>> 
>>>>> I can probably find the email thread (or maybe it was a GitHub
>>>>> comment on my proposal for params), where you said, quite
>>>>> definitively, that th

Re: OSSL_PARAMs

2019-06-04 Thread Dr Paul Dale
I thought the references were to allow const arrays of OSSL_PARAM to be viable.

A quick check through the code reveals these in test and doc only.  There are 
two instances of OSSL_PARAM arrays being declared in evp, both add the pointed 
to variable after creation, both only have two elements (the integer and the 
terminator) and both are stack allocated.  I.e. there is currently is no 
example of the use case for which the indirection is present :(


Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 5 Jun 2019, at 10:31 am, SHANE LONTIS  wrote:
> 
> I presume the reference approach was used to solve the issue of who actually 
> owns/free's the data.
>  
> 
>> On 5 Jun 2019, at 9:18 am, Dr Paul Dale > <mailto:paul.d...@oracle.com>> wrote:
>> 
>> Shane’s major complaints are about the indirection the OSSL_PARAM structure 
>> forces — for integers and return lengths and the necessity of allocating 
>> additional memory in parallel with the OSSL_PARAM.
>> 
>> The extra indirection was intended to support const arrays of OSSL_PARAM, 
>> which turn out to be a rarity because they aren’t thread safe.  With most 
>> OSSL_PARAM structure being dynamically created, the need for the indirection 
>> seems redundant.  E.g. could the return length be moved into OSSL_PARAM?  I 
>> think so.
>> 
>> Moving integral values into the structure is more difficult because BIGNUMs 
>> will always need to be references.  Allocating additional memory will still 
>> be required.  I’ve got three obvious solutions:
>> 
>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when 
>> the structure is destroyed or
>> 2. have a block of data in the OSSL_PARAM structure that can be used for 
>> native types (OSSL_UNION_ALIGN works perfectly for this) or
>> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value 
>> needs to be freed.
>> 
>> The memory allocation comes to the for when reading e.g. a file and 
>> extracting data — either the reader needs a lot of local variables to hold 
>> everything or it has to allocated for each.  The file’s data is transient in 
>> memory.
>> 
>> 
>> For the most part, the receiver side APIs seem reasonable.  It is the owning 
>> side that has the complications.
>> 
>> I think I might be able come up with some owner side routines that assist 
>> here but allowing changes to the params structure would be far easier.
>> 
>> 
>> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl 
>> functions if not ctrl as well (subject to backward compatibility concerns).
>> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Cryptographer | Network Security & Encryption 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 
>> 
>> 
>>> On 4 Jun 2019, at 11:26 pm, Richard Levitte >> <mailto:levi...@openssl.org>> wrote:
>>> 
>>> On Tue, 04 Jun 2019 14:57:00 +0200,
>>> Salz, Rich wrote:
>>>> 
>>>> 
>>>>>   Part of the idea was that this would be a means of communication
>>>>between application and provider, just like controls are with
>>>>libcrypto sub-systems.
>>>> 
>>>> 
>>>> I can probably find the email thread (or maybe it was a GitHub
>>>> comment on my proposal for params), where you said, quite
>>>> definitively, that this was *not* a general-purpose mechanism but
>>>> rather a way to expose the necessary internals for opaque objects
>>>> like RSA keys.
>>> 
>>> Either I misunderstood what you said at the time, or you misunderstood
>>> what I said...  there's definitely a disconnect here somewhere.
>>> 
>>> What I wonder is why it should be exclusively only one of those
>>> options?
>>> 
>>> Either way, the OSSL_PARAM is defined publically and openly (i.e.
>>> non-opaque), and we currently have the following functions in the
>>> public API:
>>> 
>>>EVP_MD_CTX_set_params
>>>EVP_MD_CTX_get_params
>>>OSSL_PROVIDER_get_params
>>> 
>>> I fully expect that more will come.  I have a branch where I've
>>> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if
>>> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before
>>> long (I'm actually rather surprised they haven't already), and I'm
>>> absolutely sure we will see similar functions for asymmet

OSSL_PARAMs

2019-06-04 Thread Dr Paul Dale
A question for the committers and beyond.

Should the OSSL_PARAM concept be used for more than just the libcrypto ⇿ 
provider interface?

The one that springs to mind is a more general form of the various ctrl calls, 
there are some string ctrl calls that attempt to do the same thing in a less 
structured manner.


Thoughts?

Pauli
-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia





RE: No two reviewers from same company

2019-05-23 Thread Paul Dale
There hasn't been a vote about this, however both Shane and I have committed to 
not approve each other's PRs.

I also asked Richard if this could be mechanically enforced, which I expect 
will happen eventually.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia


-Original Message-
From: Salz, Rich [mailto:rs...@akamai.com] 
Sent: Friday, 24 May 2019 1:01 AM
To: openssl-project@openssl.org
Subject: Re: No two reviewers from same company

> I understand that OpenSSL is changing things so that, by mechanism (and 
maybe by
> policy although it’s not published yet), two members of the same company 
cannot
> approve the same PR.  That’s great.  (I never approved Akamai requests 
unless it
> was trivial back when I was on the OMC.)

No such decision has been made as far as I know although it has been 
discussed
at various times.

In private email, and 
https://github.com/openssl/openssl/pull/8886#issuecomment-494624313 the 
implication is that this was a policy.

> Should this policy be extended to OpenSSL’s fellows?

IMO, no.

Why not?  I understand build process is always handled by Matt and Richard 
(despite many attempts in the past to expand this), but I think if Oracle or 
Akamai can't "force a change" then it seems to me that the OMC shouldn't either.




Update

2019-05-19 Thread Paul Dale
Rich made a good point:  HYPERLINK 
"https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openssl_openssl_pull_8910-23discussion-5Fr283111312=DwMFaQ=96ZbZZcaMF4w0F4jpN6LZg=4LM0GbR0h9Fvx86FtsKI-w=CD4glOsr2177VCbCmwwcW44E7W9Fgpq1omQwWqU8RRI=5qMw3USMNfHa7EYgiJq_2Fw34OVBznS9v2zuRy9NWS8="https://github.com/openssl/openssl/pull/8910#discussion_r283111312

(China's modified TLS defined by GM/T 0024-2014).  The PR has been closed 
unmerged but it raises a question.

 

The current requirement for inclusion is "HYPERLINK 
"https://www.openssl.org/blog/blog/2018/01/18/f2f-london/"national standard" or 
better.  Thus, this change should be accepted.

 

For TLS, would it be better if the inclusion requirement were amended to also 
include "IETF codepoints allocated"?

Presumably DTLS and QUIC too.

 

 

Pauli

-- 

Oracle

Dr Paul Dale | Cryptographer | Network Security & Encryption 

Phone +61 7 3031 7217

Oracle Australia

 


Re: Any timeframe for the 1.1.1c release?

2019-05-06 Thread Dr Paul Dale
This seems reasonable to me.


Pauli

-- 
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia



> On 6 May 2019, at 5:40 pm, Richard Levitte  wrote:
> 
> Our last update release was by the end of February.  With our usual
> 3-ish month interval, end of May would be quite suitable, methinks.
> 
> I propose Tuesday May 28th as a release date for 1.1.1c.
> 
> Cheers,
> Richard
> 
> 
> 
> There should perhaps be a 1.1.1c soonish...  There are indeed many useful
> improvements committed, but not yet released.
> 
> -- 
>   Viktor.
> 
> -- 
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/
> 
> 



SP 800-90C 10.1.2

2019-04-09 Thread Paul Dale
Do any of the FIPS sponsors or OpenSSL project people think that SP 800-90C 
section 10.1.2 "Accessing a Source DRBG with Prediction Resistance to Obtain 
any Security Strength" is worthwhile including in the code base?

 

The main use is to allow a stronger DRBG to be seeded from a weaker one.  For 
example: seeding AES-CTR-256-DRBG from AES-CTR-128-DRBG.  The reasons in favour 
don't seem very compelling:

. There are some obscure use cases for which there is a fairly easy 
work around (use stronger DRBGs everywhere). 

. A low quality hardware source could be used for higher strength 
applications.

. It would also provide some benefit for poorly set up DRBG chains.

. It can be used to construct randomness of any strength but I'm not 
aware of a current method to compress this down to high quality entropy that is 
directly usable (i.e. preserves the strength). 

 

The PR is done (#8660 https://github.com/openssl/openssl/pull/8660) but I've 
closed it since it seems unloved.  If anyone here does think that that would 
beneficial, say something as justification or it is gone.

 

 

Pauli

-- 

Oracle

Dr Paul Dale | Cryptographer | Network Security & Encryption 

Phone +61 7 3031 7217

Oracle Australia

 


  1   2   >