Re: Backports to 1.1.1 and what is allowed

2020-06-16 Thread Dr Paul Dale
I’d also support 11968 being merged.  I find the recent discussion and the 
likelihood of all major distros (supporting the S390x) picking up the patches 
fairly compelling, especially since the same people will be maintaining it.

I would need to go and double check the non-S390x specific parts for possible 
breakage but the bulk of the changes are S390x specific.


I support formalising the rules better than we have at the moment.  Even if 
this is in conflict with the above.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 17 Jun 2020, at 1:43 pm, Benjamin Kaduk  wrote:
> 
> On Tue, Jun 16, 2020 at 02:03:58PM +0100, Matt Caswell wrote:
>> PR 11188 proposes to backport a series of s390x patches to the 1.1.1
>> branch. IIUC it includes performance improvements as well as support for
>> new hardware instructions.
>> 
>> I think we need to have a much clearer and more explicit policy about
>> exactly what is allowed to be backported to a stable branch. For example
>> PR #11968 was a significant recent PR that backported AES CTR-DRBG
>> performance improvements to the 1.1.1 branch and was allowed (should it
>> have been?).
> 
> I am happy to see 11968 landed; some informal testing that I hope to make
> more formal soon seems to show a ~20% slowdown/regression for large RNG
> requests going from 1.1.1d to 1.1.g, and 11968 provided substantially more
> significant of a speedup (again, very informal testing, though).
> 
> In this case you could say that the "bug" is that we're only calling AES on
> a block at a time despite having much more effective backends available for
> multi-block calls.
> 
>> We have always said that the stable releases should only receive bug and
>> security fixes. Performance improvements have been a bit of a grey area,
>> e.g. a few lines of code that significantly improves the performance of
>> a particular algorithm or codepath could reasonably be justified as
>> "fixing a performance bug". OTOH bringing in whole new files of
>> assembler seems to go way beyond that definition.
> 
> There's probably some semi-subtle distinction to make along the lines of
> "making the algorithm more efficient" vs "using a new algorithm, that
> happens to run faster", with the latter counting as a feature.
> 
>> However, when I tried to find a reference to the policy which says
>> exactly what we are allowed to backport I could find one. Do we have
>> such a thing?
>> 
>> In any case I think we should develop a much more explicit policy that
>> will enable us to decide whether PRs such as 11188 are reasonable or
>> not. See for example Ubuntu's Stable Release Updates policy:
> 
> I agree that having something written down will be very useful, even if
> just as a fixed benchmark against which to think about how exceptional any
> given "exception case" is where we want to deviate from it.
> 
> -Ben
> 
>> https://wiki.ubuntu.com/StableReleaseUpdates
>> 
>> 
>> Matt



Re: Backports to 1.1.1 and what is allowed

2020-06-16 Thread Benjamin Kaduk
On Tue, Jun 16, 2020 at 02:03:58PM +0100, Matt Caswell wrote:
> PR 11188 proposes to backport a series of s390x patches to the 1.1.1
> branch. IIUC it includes performance improvements as well as support for
> new hardware instructions.
> 
> I think we need to have a much clearer and more explicit policy about
> exactly what is allowed to be backported to a stable branch. For example
> PR #11968 was a significant recent PR that backported AES CTR-DRBG
> performance improvements to the 1.1.1 branch and was allowed (should it
> have been?).

I am happy to see 11968 landed; some informal testing that I hope to make
more formal soon seems to show a ~20% slowdown/regression for large RNG
requests going from 1.1.1d to 1.1.g, and 11968 provided substantially more
significant of a speedup (again, very informal testing, though).

In this case you could say that the "bug" is that we're only calling AES on
a block at a time despite having much more effective backends available for
multi-block calls.

> We have always said that the stable releases should only receive bug and
> security fixes. Performance improvements have been a bit of a grey area,
> e.g. a few lines of code that significantly improves the performance of
> a particular algorithm or codepath could reasonably be justified as
> "fixing a performance bug". OTOH bringing in whole new files of
> assembler seems to go way beyond that definition.

There's probably some semi-subtle distinction to make along the lines of
"making the algorithm more efficient" vs "using a new algorithm, that
happens to run faster", with the latter counting as a feature.

> However, when I tried to find a reference to the policy which says
> exactly what we are allowed to backport I could find one. Do we have
> such a thing?
> 
> In any case I think we should develop a much more explicit policy that
> will enable us to decide whether PRs such as 11188 are reasonable or
> not. See for example Ubuntu's Stable Release Updates policy:

I agree that having something written down will be very useful, even if
just as a fixed benchmark against which to think about how exceptional any
given "exception case" is where we want to deviate from it.

-Ben

> https://wiki.ubuntu.com/StableReleaseUpdates
> 
> 
> Matt


Monthly Status Report (May 2020)

2020-06-16 Thread Richard Levitte
Apart from normal business, such as normal reviews, OMC business,
normal system administration tasks, small fixes, etc., key activities
this month:

* Development

  - [not_yet_merged] [WIP] Refactor CPUID code
(PR openssl/openssl#11311)
  - EVP: Only use the engine when one is defined, in pkey_mac_ctrl()
(PR openssl/openssl#11674)
  - WPACKET: don't write DER length when we don't want to
(PR openssl/openssl#11703)
  - util/perl/OpenSSL/OID.pm: remove the included unit test
(PR openssl/openssl#11704)
  - Fix reason code clash
(PR openssl/openssl#11708)
  - PSS pack: Add provider support for PSS parameters
(PR openssl/openssl#11710)
  - Configure: avoid perl regexp bugs
(PR openssl/openssl#11737)
  - EVP: when setting the operation to EVP_PKEY_OP_UNDEFINED, clean up!
(PR openssl/openssl#11750)
  - Warn that objects with opaque types must never be passed on.
(PR openssl/openssl#11754)
  - OSSL_STORE additions
(PR openssl/openssl#11756)
  - Fix some misunderstandings in our providers' main modules
(PR openssl/openssl#11777)
  - Fix d2i_PrivateKey_ex() to work as documented
(PR openssl/openssl#11787)
  - Fix CHANGES.md issues reported by markdownlint
(PR openssl/openssl#11788)
  - Remove explicit dependency on configdata.pm when processing .in files
(PR openssl/openssl#11790)
  - PROV: Add a proper provider context structure for OpenSSL providers
(PR openssl/openssl#11803)
  - SSL: refactor ssl_cert_lookup_by_pkey() to work with provider side keys
(PR openssl/openssl#11828)
  - test/evp_extra_test.c: Add OPENSSL_NO_CMAC around CMAC test
(PR openssl/openssl#11833)
  - CORE: Fix a couple of bugs in algorithm_do_this()
(PR openssl/openssl#11837)
  - CORE: query for operations only once per provider (unless no_store is true)
(PR openssl/openssl#11842)
  - Add OSSL_PROVIDER_do_all()
(PR openssl/openssl#11858)
  - Refactor the provider side DER constants and writers
(PR openssl/openssl#11868)
  - rsa_padding_add_PKCS1_OAEP_mgf1_with_libctx(): fix check of |md|
(PR openssl/openssl#11869)
  - STORE: Make try_decode_PrivateKey() ENGINE aware
(PR openssl/openssl#11872)
  - Fix d2i_PrivateKey() to work as documented [1.1.1]
(PR openssl/openssl#11888)
  - PROV: Fix RSA-OAEP memory leak
(PR openssl/openssl#11927)
  - Add header file docs for openssl/core_numbers.h and openssl/core_names.h
(PR openssl/openssl#11963)
  - util/mkpod2html.pl: Fix unbalanced quotes
(PR openssl/openssl#11969)
  - Fix EVP_CIPHER_fetch race condition
(PR openssl/openssl#11977)
  - [not_yet_merged] [WIP] EVP: retrieve EVP_CIPHER constants in the 
evp_cipher_from_dispatch()
(PR openssl/openssl#11980)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/


Late Monthly Status Report (March 2020)

2020-06-16 Thread Richard Levitte
Apart from normal business, such as normal reviews, OMC business,
normal system administration tasks, small fixes, etc., key activities
this month:

* Development

  - [not_yet_merged] WIP: apps: Switch to using OSSL_STORE for loading keys, 
certs, ...
(PR openssl/openssl#7390)
  - Implement domparam and key generation
(PR openssl/openssl#10289)
  - DOC: Add documentation related to X509_LOOKUPs [1.1.1]
(PR openssl/openssl#11120)
  - Refactor CRMF_poposigningkey_init() to work with provider keys
(PR openssl/openssl#11126)
  - Configuration: Add post module building check
(PR openssl/openssl#11170)
  - build.info extensions: variable value substitutions and multi-item 
statements
(PR openssl/openssl#11185)
  - .travis.yml: where it matters, have build and source nesting levels differ
(PR openssl/openssl#11186)
  - crypto/perlasm/x86_64-xlate.pl: detect GNU as to deal with quirks
(PR openssl/openssl#11191)
  - EVP: Check that key methods aren't foreign when exporting
(PR openssl/openssl#11193)
  - Andoid cross compile: change ANDROID_NDK_HOME to ANDROID_NDK_ROOT
(PR openssl/openssl#11206)
  - config, Configure: move the check of removed crypto/ sub-systems
(PR openssl/openssl#11217)
  - Configurations: Fix "android" configuration target
(PR openssl/openssl#11238)
  - util/wrap.pl: do not look at EXE_SHELL
(PR openssl/openssl#11258)
  - Restructure documentation of implementations in providers
(PR openssl/openssl#11270)
  - DOCS: Fix documentation on asymmetric keydata types
(PR openssl/openssl#11275)
  - DOCS: Fix the description of OSSL_PARAM_allocate_from_text()
(PR openssl/openssl#11279)
  - Refactor sm2_id
(PR openssl/openssl#11302)
  - Fix RSA structure
(PR openssl/openssl#11315)
  - Fix legacy_ctrl_to_param() to pay better attention to keytype
(PR openssl/openssl#11329)
  - Refactor sm2_id - addendum
(PR openssl/openssl#11335)
  - EVP: fetch the EVP_KEYMGMT earlier
(PR openssl/openssl#11343)
  - [WIP] EVP: Fix EVP_PKEY_copy_parameters() for a newly allocated |to|
(PR openssl/openssl#11368)
  - DH, DSA, EC_KEY: Fix exporters to allow domain parameter keys
(PR openssl/openssl#11374)
  - EVP: Downgrade keys rather than upgrade
(PR openssl/openssl#11375)
  - util/wrap.pl: Correct exit code when signalled
(PR openssl/openssl#11379)
  - EC: Refactor ec_curve_name2nid() to accept NIST curve names
(PR openssl/openssl#11391)
  - PROV: Fix EC_KEY exporters to allow domain parameter keys
(PR openssl/openssl#11394)

* Web

  - Fix 'make relupd'

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/


Late Monthly Status Report (February 2020)

2020-06-16 Thread Richard Levitte
Apart from normal business, such as normal reviews, OMC business,
normal system administration tasks, small fixes, etc., key activities
this month:

* Development

  - PROV: add RSA signature implementation
(PR openssl/openssl#10557)
  - Disable devcryptoeng on newer OpenBSD versions [1.1.1]
(PR openssl/openssl#10565)
  - EVP: Adapt EVP_PKEY checking, comparing and copying for provider keys
(PR openssl/openssl#10807)
  - DOC: document in more detail what a BIO_read_ex() via BIO_f_buffer() does
(PR openssl/openssl#10890)
  - Refactor SM2
(PR openssl/openssl#10942)
  - Decentralize legacy_ctrl_str_to_param()
(PR openssl/openssl#10947)
  - config: ensure the perl Configure run is the last statement
(PR openssl/openssl#10953)
  - EVP: Small refactor of keymgmt library code
(PR openssl/openssl#10963)
  - DOC: Add documentation related to X509_LOOKUPs
(PR openssl/openssl#10986)
  - Redesign the KEYMGMT libcrypto <-> provider interface
(PR openssl/openssl#11006)
  - EVP: Adapt EVP_PKEY checking, comparing and copying for provider keys, take 
2
(PR openssl/openssl#11025)
  - Configure: Add easy to use disabled deprecated functionality indicators
(PR openssl/openssl#11027)
  - PROV: Ensure the AlgorithmIdentifier registers in DSA signature impl
(PR openssl/openssl#11037)
  - X509_PUBKEY_set(): Fix memory leak
(PR openssl/openssl#11038)
  - test/recipes/80-test_ssl_old.t: use 'openssl genpkey'
(PR openssl/openssl#11041)
  - Make util/find-doc-nits runnable from the build tree
(PR openssl/openssl#11045)
  - Refactor OSSL_PARAM_allocate_from_text() for better flexibility
(PR openssl/openssl#11046)
  - Add OSSL_SERIALIZER_PUBKEY_TO_DER_PQ and friends
(PR openssl/openssl#11055)
  - Adapt i2d_PrivateKey for provider only keys
(PR openssl/openssl#11056)
  - Document OSSL_SERIALIZER_PUBKEY_TO_DER_PQ and friends
(PR openssl/openssl#11071)
  - Refactor evp_pkey_make_provided() to do legacy to provider export
(PR openssl/openssl#11074)
  - Adapt i2d_PUBKEY for provider only keys
(PR openssl/openssl#11078)
  - TEST: Create test specific output directories
(PR openssl/openssl#11080)
  - include/openssl/whrlpool.h: correct unbalanced deprecation guards
(PR openssl/openssl#11087)
  - Fix VMS build [1.1.1 only]
(PR openssl/openssl#11088)
  - PROV: Build the main FIPS module code with FIPS_MODE defined
(PR openssl/openssl#11090)
  - TEST: add util/wrap.pl and use it #0
(PR openssl/openssl#0)
  - Rethink the EVP_PKEY cache of provider side keys
(PR openssl/openssl#11148)
  - VMS: mitigate for the C++ compiler that doesn't understand certain pragmas
(PR openssl/openssl#11159)
  - Deprecate ASN1_sign(), ASN1_verify() and ASN1_digest()
(PR openssl/openssl#11161)
  - Fix util/mktar.sh to use the new VERSION information
(PR openssl/openssl#11190)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/


Late Monthly Status Report (January 2020)

2020-06-16 Thread Richard Levitte
Apart from normal business, such as normal reviews, OMC business,
normal system administration tasks, small fixes, etc., key activities
this month:

* Development

  - [not_yet_merged] WIP: OSSL_STORE for providers
(PR openssl/openssl#9389)
  - CORE & EVP: Adapt KEYEXCH, SIGNATURE and ASYM_CIPHER to handle key types 
better
(PR openssl/openssl#10647)
  - Configuration: synchronise the variables on the build file templates
(PR openssl/openssl#10753)
  - EVP: Fix method to determine if a PKEY is legacy or not
(PR openssl/openssl#10758)
  - DOCS: The interpretation of OPENSSL_API_COMPAT has changed, update docs
(PR openssl/openssl#10765)
  - Add missing inclusion of "internal/deprecated.h"
(PR openssl/openssl#10766)
  - EVP: If a key can't be exported to provider, fallback to legacy
(PR openssl/openssl#10771)
  - Add the DSA serializers to the default provider tools
(PR openssl/openssl#10772)
  - EVP: make EVP_PKEY_{bits,security_bits,size} work with provider only keys
(PR openssl/openssl#10778)
  - PROV: Fix mixup between general and specialized GCM implementations
(PR openssl/openssl#10783)
  - Configure: use $list_separator_re only for defines and includes
(PR openssl/openssl#10793)
  - Eliminate some EVP_PKEY_size() uses
(PR openssl/openssl#10798)
  - EVP: clear error when falling back from failed EVP_KEYMGMT_fetch()
(PR openssl/openssl#10803)
  - CORE: renumber OSSL_FUNC_KEYMGMT macros
(PR openssl/openssl#10804)
  - Fix documentation for EVP_DigestSign* and EVP_DigestVerify*
(PR openssl/openssl#10805)
  - Fix EVP_Digest{Sign,Verify}Final() and EVP_Digest{Sign,Verify}() for 
provider only keys
(PR openssl/openssl#10806)
  - EVP: Adapt EVP_PKEY Seal and Open for provider keys
(PR openssl/openssl#10808)
  - Move the definition of OPENSSL_BUILDING_OPENSSL
(PR openssl/openssl#10813)
  - Change returned -2 to 0 in EVP_Digest{Sign,Verify}Init()
(PR openssl/openssl#10815)
  - Add EVP_PKEY_get_default_digest_name()
(PR openssl/openssl#10824)
  - CRYPTO: Remove support for ex_data fields when building the FIPS module
(PR openssl/openssl#10837)
  - Modify EVP_CIPHER_is_a() and EVP_MD_is_a() to handle legacy methods too
(PR openssl/openssl#10845)
  - Move the stored namemap pre-population to namemap construction
(PR openssl/openssl#10846)
  - [1.1.1] Fix documentation of return value for EVP_Digest{Sign,Verify}Init()
(PR openssl/openssl#10847)
  - Build file templates: Use explicit files instead of $< or $? for pods
(PR openssl/openssl#10849)
  - EVP: Add evp_pkey_make_provided() and refactor around it
(PR openssl/openssl#10850)
  - Adapt X509_PUBKEY_set() for use with provided implementations
(PR openssl/openssl#10851)
  - For all assembler scripts where it matters, recognise clang > 9.x
(PR openssl/openssl#10855)
  - Add GNU properties note for Intel CET in x86_64-xlate.pl
(PR openssl/openssl#10875)
  - Configure: Better detection of '-static' in @{$config{LDFLAGS}}
(PR openssl/openssl#10878)
  - PROV: Fix bignum printout in text serializers
(PR openssl/openssl#10891)
  - OpenSSL::Test: bring back the relative paths
(PR openssl/openssl#10913)
  - Adapt ASN1_item_sign_ctx() for use with provided keypairs
(PR openssl/openssl#10920)
  - Add internal maxsize macros
(PR openssl/openssl#10928)
  - test/recipes/30-test_evp.t: Fix multiple definition of @bffiles
(PR openssl/openssl#10944)

* Administration

  - Stop making snaps for 1.1.0 and 1.0.2, and make 3.0-dev snaps
  - Switch final review to be for OTC rather than OMC

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/


Backports to 1.1.1 and what is allowed

2020-06-16 Thread Matt Caswell
PR 11188 proposes to backport a series of s390x patches to the 1.1.1
branch. IIUC it includes performance improvements as well as support for
new hardware instructions.

I think we need to have a much clearer and more explicit policy about
exactly what is allowed to be backported to a stable branch. For example
PR #11968 was a significant recent PR that backported AES CTR-DRBG
performance improvements to the 1.1.1 branch and was allowed (should it
have been?).

We have always said that the stable releases should only receive bug and
security fixes. Performance improvements have been a bit of a grey area,
e.g. a few lines of code that significantly improves the performance of
a particular algorithm or codepath could reasonably be justified as
"fixing a performance bug". OTOH bringing in whole new files of
assembler seems to go way beyond that definition.

However, when I tried to find a reference to the policy which says
exactly what we are allowed to backport I could find one. Do we have
such a thing?

In any case I think we should develop a much more explicit policy that
will enable us to decide whether PRs such as 11188 are reasonable or
not. See for example Ubuntu's Stable Release Updates policy:

https://wiki.ubuntu.com/StableReleaseUpdates


Matt