lhash statistics functions deprecation

[Dr Paul Dale]

Topic: lhash statistics functions to always report 0 in both master and 3.0.
   In addition we should deprecate the functions in master.
details: PR https://github.com/openssl/openssl/pull/17931
Proposed by: Pauli
Issue link: https://github.com/openssl/technical-policies/issues/35
Public: yes
Opened: 2022-03-22
Closed: 2022-03-22
Accepted:  yes  (for: 7, against: 0, abstained: 0, not voted: 1)

  Dmitry [+1]
  Matt   [+1]
  Pauli  [+1]
  Tim    [ 0]
  Richard    [+1]
  Shane  [+1]
  Tomas  [+1]
  Kurt   [  ]
  Matthias   [+1]
  Nicola [+0]

Re: Additional things for deprecation

[Richard Levitte]

Hmmm, are we going to start marking types for deprecation?  I would
advise against it on a general level, 'cause that's likely to cause an
implosion.  See for example marking ENGINE for deprecation, i.e. this:

diff --git a/include/openssl/types.h b/include/openssl/types.h
index ee024cef29..4e73c0dc0d 100644
--- a/include/openssl/types.h
+++ b/include/openssl/types.h
@@ -172,7 +172,9 @@ typedef struct ossl_init_settings_st OPENSSL_INIT_SETTINGS;
 typedef struct ui_st UI;
 typedef struct ui_method_st UI_METHOD;
 
-typedef struct engine_st ENGINE;
+# ifndef OPENSSL_NO_DEPRECATED_3_0
+OSSL_DEPRECATEDIN_3_0 typedef struct engine_st ENGINE;
+# endif
 typedef struct ssl_st SSL;
 typedef struct ssl_ctx_st SSL_CTX;
 

Try that and see your build become...  interesting.

Cheers,
Richard

On Tue, 13 Oct 2020 11:40:37 +0200,
Tim Hudson wrote:
> 
> In a 3.0 context, EVP_PKEY_ASN1_METHOD and all the associated functions 
> should be marked
> deprecated in my view.
> 
> Tim.
-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/

Additional things for deprecation

[Tim Hudson]

In a 3.0 context, EVP_PKEY_ASN1_METHOD and all the associated functions
should be marked deprecated in my view.

Tim.

Re: OMC Vote on deprecation of command line apps

[Dr Paul Dale]

This vote has passed: 3 for, 1 against and 2 abstentions.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 8 May 2020, at 3:08 pm, Dr Paul Dale  wrote:
> 
> PR 11575 <https://github.com/openssl/openssl/pull/11575> has been blocking 
> awaiting decision for a while now.  Time for a vote:
> 
> topic: Merge #11575 for 3.0.
> comment: This PR removes the notes indicating that a number of the command
>  line utilities are deprecated.  Not merging it will leave them 
> flagged
>  as deprecated.
> Proposed by: Paul Dale
> Public: yes
> opened: 2020-05-08
> 
> Ideally we’ll have a decision in time for the next 3.0 alpha release.
> 
> 
> The crux of the matter is that a number of the command line utilities are 
> flagged as deprecated currently:
> dhparam
> dsa
> dsaparam
> ec
> ecparam
> agendas
> rsa
> These commands are not being removed in 3.0, instead they’ve been rewritten 
> to use the PKEY APIs instead of the low level APIs as far as possible.
> 
> 
> The reasons for keeping them are:
> they are easier to use than the pkey replacements
> a web search will likely result in thees commands not the pkey replacements.
> 
> The reason for removing them is one of maintenance: having duplicate commands 
> means having to make changes in two places and this has been missed in the 
> past and will be in the future.
> 
> 
> Other random notes:
> Deprecation of these commands does not mandate that they are removed at the 
> first opportunity.  It only indicates that we want to move away from them.
> Rewriting these commands so that they call the pkey replacements looks to be 
> very difficult.  Reproducing the exact behaviours will be challenging, 
> although the basic functionality would be straightforward.
> The rsautl command is deprecated and isn’t slated for being restored — 
> pkeyutl is every bit as easy to use.
> The -dsaparam option to dhparam is deprecated — it cannot be supported 
> without direct access to low level functionality we want to remove.
> Post quantum crypto will make the discussion obsolete — none of these 
> algorithms are useful in a quantum computer world.
> 
> My personal opinion is that these commands are good being deprecated but that 
> we should not remove them until their usefulness is at an end.  This will 
> likely mean not removing them after five years of deprecation.  It would mean 
> removing them once quantum computers are shown to be effective.  Without 
> deprecation now, we can’t remove them until a lot later.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 

OMC Vote on deprecation of command line apps

[Dr Paul Dale]

PR 11575 <https://github.com/openssl/openssl/pull/11575> has been blocking 
awaiting decision for a while now.  Time for a vote:

topic: Merge #11575 for 3.0.
comment: This PR removes the notes indicating that a number of the command
 line utilities are deprecated.  Not merging it will leave them flagged
 as deprecated.
Proposed by: Paul Dale
Public: yes
opened: 2020-05-08

Ideally we’ll have a decision in time for the next 3.0 alpha release.


The crux of the matter is that a number of the command line utilities are 
flagged as deprecated currently:
dhparam
dsa
dsaparam
ec
ecparam
agendas
rsa
These commands are not being removed in 3.0, instead they’ve been rewritten to 
use the PKEY APIs instead of the low level APIs as far as possible.


The reasons for keeping them are:
they are easier to use than the pkey replacements
a web search will likely result in thees commands not the pkey replacements.

The reason for removing them is one of maintenance: having duplicate commands 
means having to make changes in two places and this has been missed in the past 
and will be in the future.


Other random notes:
Deprecation of these commands does not mandate that they are removed at the 
first opportunity.  It only indicates that we want to move away from them.
Rewriting these commands so that they call the pkey replacements looks to be 
very difficult.  Reproducing the exact behaviours will be challenging, although 
the basic functionality would be straightforward.
The rsautl command is deprecated and isn’t slated for being restored — pkeyutl 
is every bit as easy to use.
The -dsaparam option to dhparam is deprecated — it cannot be supported without 
direct access to low level functionality we want to remove.
Post quantum crypto will make the discussion obsolete — none of these 
algorithms are useful in a quantum computer world.

My personal opinion is that these commands are good being deprecated but that 
we should not remove them until their usefulness is at an end.  This will 
likely mean not removing them after five years of deprecation.  It would mean 
removing them once quantum computers are shown to be effective.  Without 
deprecation now, we can’t remove them until a lot later.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia

Re: Deprecation

[Benjamin Kaduk]

On Fri, Feb 14, 2020 at 05:59:32PM +0100, Tomas Mraz wrote:
> On Fri, 2020-02-14 at 16:17 +, Salz, Rich wrote:
> > I think we should delay the deprecation of engine stuff to 4.0.
> > Personally I don't have sense of stability of provider API.
> >  
> > I share this concern.  This is the first release of the provider
> > infrastructure, and while it will be known to work for FIPS modules,
> > it’s hubris to think OpenSSL will get it completely right for other
> > uses.
> >  
> > All other deprecations point to existing API’s or, if they are brand
> > new, are not a lot of code/work to implement them.  The provider
> > interface is both brand new and significant work.  Deprecating and
> > saying “use providers” without at least one release cycle of 
> > providers, seems premature.
> 
> This is an argument for not removing engines in 4.0 (or earlier than
> one major release after the provider interface fully stabilizes and
> proves viable). However I do not think that it is argument for not
> deprecating it. Deprecation is declaration of intent and not a way to
> force people stop using the API immediately.
> 
> How is the provider API going to improve/stabilize, if the 3rd party
> engine suppliers will not get the the message that the engine API is
> eventually going away in future and start writing providers as
> replacement.

I have similar feelings to Tomas.

Note that the current policy's test for "is removal allowed" is a
two-pronged test, and if the next release after 3.0 is 4.0, we would not be
allowed to remove engines in 4.0, since providers (the "engine
replacement") will not have been available in a (previous) LTS release.

If we know that something will need to go away eventually, it seems like we
can use deprecation annotations to publicize that fact, even if the
eventual removal will be delayed for quite some time due to other
considerations.

-Ben

Re: Deprecation

[Tomas Mraz]

On Fri, 2020-02-14 at 16:17 +, Salz, Rich wrote:
> I think we should delay the deprecation of engine stuff to 4.0.
> Personally I don't have sense of stability of provider API.
>  
> I share this concern.  This is the first release of the provider
> infrastructure, and while it will be known to work for FIPS modules,
> it’s hubris to think OpenSSL will get it completely right for other
> uses.
>  
> All other deprecations point to existing API’s or, if they are brand
> new, are not a lot of code/work to implement them.  The provider
> interface is both brand new and significant work.  Deprecating and
> saying “use providers” without at least one release cycle of 
> providers, seems premature.

This is an argument for not removing engines in 4.0 (or earlier than
one major release after the provider interface fully stabilizes and
proves viable). However I do not think that it is argument for not
deprecating it. Deprecation is declaration of intent and not a way to
force people stop using the API immediately.

How is the provider API going to improve/stabilize, if the 3rd party
engine suppliers will not get the the message that the engine API is
eventually going away in future and start writing providers as
replacement.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]

Re: Deprecation

[Salz, Rich]

  *   I think we should delay the deprecation of engine stuff to 4.0. 
Personally I don't have sense of stability of provider API.

I share this concern.  This is the first release of the provider 
infrastructure, and while it will be known to work for FIPS modules, it’s 
hubris to think OpenSSL will get it completely right for other uses.

All other deprecations point to existing API’s or, if they are brand new, are 
not a lot of code/work to implement them.  The provider interface is both brand 
new and significant work.  Deprecating and saying “use providers” without at 
least one release cycle of providers, seems premature.

Re: Deprecation

[Richard Levitte]

On Fri, 14 Feb 2020 12:17:30 +0100,
Dmitry Belyavsky wrote:
> 
> 
> Dear Richard,
> 
> On Fri, Feb 14, 2020 at 1:37 PM Richard Levitte  wrote:
> 
> It should be pointed out that the engine stuff isn't as stable as you
> might think, because it shares internal structures with libcrypto.
> Even if we handle those structures via function calls, all it takes is
> loading an engine linked against - say - 1.1.0 in 1.1.1 to run a real
> risk of a spectacular KABOOM if any of the structure that are touched
> changed between those OpenSSL versions.
> 
> Does ABI compatibility cover a significant share of these cases?

Yes.  Any structure change between those OpenSSL version potentially
means an ABI incompatibility between an engine and the libcrypto that
loads it.

This is precisely the reason why we have forbidden shared opaque
structures over the libcrypto <-> provider boundary.

...

> > But some features I'm interested in imply engine model (and it will
> > be great if somebody else could look at PR 10904 to avoid it when
> > possible).
>
> Apart from a few details, that PR looks rather innocuous.  I frankly
> haven't had time to look at it too deeply (I just discovered that I
> was prompted), as I haven't dived in the CMS code yet...
> 
> Well, the problem is introducing some new control values. I don't
> feel policy here very well. I also plan to submit at least one
> TLS-related PR significantly more innocent...

We haven't formed a clear policy on those, it's a bit of play as we go
and do what makes most sense.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/

Re: Deprecation

[Dmitry Belyavsky]

Dear Richard,

On Fri, Feb 14, 2020 at 1:37 PM Richard Levitte  wrote:

> On Fri, 14 Feb 2020 10:41:05 +0100,
> Dmitry Belyavsky wrote:
> >
> >
> > Hello,
> >
> > On Fri, Feb 14, 2020 at 5:30 AM Dr Paul Dale 
> wrote:
> >
> > There is some pushback against the deprecations going on in various
> PRs.
> >
> > The plan has always been to deprecate engines in 3.0 and removing
> support for them 5+ years later.
> > Originally, the path was to have included an engine provider that
> could load engines and make them appear
> > to be a provider.  After a fair amount of investigation, this was
> deemed to be too difficult in the 3.0
> > time frame.
> >
> > Do we still want to deprecate engines in 3.0?
> > Should we defer until 4.0 instead?
> >
> > I think we should delay the deprecation of engine stuff to 4.0.
> Personally I don't have sense of stability of
> > provider API.
>
> It should be pointed out that the engine stuff isn't as stable as you
> might think, because it shares internal structures with libcrypto.
> Even if we handle those structures via function calls, all it takes is
> loading an engine linked against - say - 1.1.0 in 1.1.1 to run a real
> risk of a spectacular KABOOM if any of the structure that are touched
> changed between those OpenSSL versions.
>

Does ABI compatibility cover a significant share of these cases?


> This is a consequence of making structures opaque and feeling much
> more liberty in changing them, and we didn't quite pay attention.
>
> The whole model around providers is intentionally designed to allow
> providers to run flexibly with any OpenSSL version, even if they are
> linked with some (possibly different) libcrypto version.
>
> So the question of stability depends on what you look at.
>

Agreed.

It's true that some parts of the provider API is fluctuating a bit, as
> early designs need modifications to better meet demands.  We're still
> in development.  Come beta1 (schedule for June), I expect that it will
> have stabilized.  Come the release, it must have.
>

Sure.

> The main benefits seem to boil down to continuing to support existing
> engines vs removing the legacy code
> > paths and switching to the provider model.
> >
> > For me, both as open-source and commercial engine developer seems
> > reasonable to delay conversion from engines to providers at least
> > until 3.0.0 feature freeze happens.
>
> According to our policy, a deprecation starts at the release that has
> those deprecations, and removal of deprecated stuff can only happen 5
> years later at the earliest.  Seeing deprecations in the source now
> doesn't change that, the clock will start ticking when we release
> 3.0, so consider what you see happening on github as a pre-deprecation
> warning.
>

Yes.


> > But some features I'm interested in imply engine model (and it will
> > be great if somebody else could look at PR 10904 to avoid it when
> > possible).
>
> Apart from a few details, that PR looks rather innocuous.  I frankly
> haven't had time to look at it too deeply (I just discovered that I
> was prompted), as I haven't dived in the CMS code yet...
>

Well, the problem is introducing some new control values. I don't feel
policy here very well.
I also plan to submit at least one TLS-related PR significantly more
innocent...

-- 
SY, Dmitry Belyavsky

Re: Deprecation

[Dmitry Belyavsky]

Dear Matt,

On Fri, Feb 14, 2020 at 12:48 PM Matt Caswell  wrote:

>
>
> On 14/02/2020 09:41, Dmitry Belyavsky wrote:
> > Hello,
> >
> > On Fri, Feb 14, 2020 at 5:30 AM Dr Paul Dale  > <mailto:paul.d...@oracle.com>> wrote:
> >
> > There is some pushback against the deprecations going on in various
> PRs.
> >
> > The plan has always been to deprecate engines in 3.0 and removing
> > support for them 5+ years later.  Originally, the path was to have
> > included an engine provider that could load engines and make them
> > appear to be a provider.  After a fair amount of investigation, this
> > was deemed to be too difficult in the 3.0 time frame.
> >
> > Do we still want to deprecate engines in 3.0?
> > Should we defer until 4.0 instead?
> >
> >
> > I think we should delay the deprecation of engine stuff to 4.0.
> > Personally I don't have sense of stability of provider API.
>
> Well - it isn't stable *right now*. Its in development. But moving
> forward the provider model *is* the way we intend to go. By the time of
> the 3.0 release the provider stuff had better be stable, otherwise we've
> gone wrong.
>

Totally agree. Though the conversion between engines and providers is not
as straightforward as I want, so I prefer to wait for some time.

>
> As has been pointed out many times. Deprecation does not mean removal
> (yet). Its just a signal that at some later time we will remove it.
>

Sure.

>
> And the 3.0 is the right time to signal that. We don't want to hold on
> the "legacy" codepaths for any longer than we have to. They were only
> ever originally intended to be temporary, and to be removed before 3.0
> got released. However it now looks like they will live beyond the 3.0
> release. They should not live for any longer than absolutely necessary.
>

Hmmm. Not sure here. I remember the introduction of EVP_PKEY_ameth/pmeth
currently moving to providers.

> The main benefits seem to boil down to continuing to support
> > existing engines vs removing the legacy code paths and switching to
> > the provider model.
>

It's worth explaining the benefits to engine authors :)
I understand the benefits of OpenSSL as a product and still now don't have
a firm understanding of benefits for the surrounding ecosystem. Though I
did not dive into the providers deeply.

I still have a suspicion that we will not have function signatures for
callbacks in providers.
Do we? If don't, we'll have a set of errors implementing it.

> For me, both as open-source and commercial engine developer seems
> > reasonable to delay conversion from engines to providers at least until
> > 3.0.0 feature freeze happens.
>
> That's a perfectly reasonable strategy. But this decision is independent
> of whether we deprecate the ENGINE APIs in 3.0 or not. It would be
> perfectly ok for people to continue to *use* engines in 3.0 even though
> they are deprecated.
>

Sure.


> > But some features I'm interested in imply engine model (and it will be
> > great if somebody else could look at PR 10904 to avoid it when possible).
>
> If there are gaps then we need to plug them. The provider model should
> be able to do everything that you can do now in ENGINEs.
>

Totally agree.

-- 
SY, Dmitry Belyavsky

Re: Deprecation

[Matt Caswell]

On 14/02/2020 10:46, Dr Paul Dale wrote:
> Roumen in
> https://github.com/openssl/openssl/pull/10977#issuecomment-584818517
> Dmitry
> in https://github.com/openssl/openssl/pull/11082#issuecomment-585603911

Thanks.

Having re-read the comments and the thread here, I am still of the
opinion that we should press ahead with the deprecations as planned.

Matt

> And a further one via private email.
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 
> 
> 
> 
>> On 14 Feb 2020, at 7:37 pm, Matt Caswell > > wrote:
>>
>>
>>
>> On 14/02/2020 02:30, Dr Paul Dale wrote:
>>> There is some pushback against the deprecations going on in various PRs.
>>
>> I've not followed all of the PRs. Can you point at some specific
>> comments you've received pushing back on this?
>>
>> Matt
>>
>>
>>>
>>> The plan has always been to deprecate engines in 3.0 and removing
>>> support for them 5+ years later.  Originally, the path was to have
>>> included an engine provider that could load engines and make them appear
>>> to be a provider.  After a fair amount of investigation, this was deemed
>>> to be too difficult in the 3.0 time frame.
>>>
>>> Do we still want to deprecate engines in 3.0?
>>> Should we defer until 4.0 instead?
>>>
>>>
>>> The main benefits seem to boil down to continuing to support existing
>>> engines vs removing the legacy code paths and switching to the provider
>>> model.
>>>
>>>
>>> Pauli
>>> -- 
>>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>>> Phone +61 7 3031 7217
>>> Oracle Australia
>>>
> 

Re: Deprecation

[Dr Paul Dale]

Roumen in https://github.com/openssl/openssl/pull/10977#issuecomment-584818517 

Dmitry in https://github.com/openssl/openssl/pull/11082#issuecomment-585603911 

And a further one via private email.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 14 Feb 2020, at 7:37 pm, Matt Caswell  wrote:
> 
> 
> 
> On 14/02/2020 02:30, Dr Paul Dale wrote:
>> There is some pushback against the deprecations going on in various PRs.
> 
> I've not followed all of the PRs. Can you point at some specific
> comments you've received pushing back on this?
> 
> Matt
> 
> 
>> 
>> The plan has always been to deprecate engines in 3.0 and removing
>> support for them 5+ years later.  Originally, the path was to have
>> included an engine provider that could load engines and make them appear
>> to be a provider.  After a fair amount of investigation, this was deemed
>> to be too difficult in the 3.0 time frame.
>> 
>> Do we still want to deprecate engines in 3.0?
>> Should we defer until 4.0 instead?
>> 
>> 
>> The main benefits seem to boil down to continuing to support existing
>> engines vs removing the legacy code paths and switching to the provider
>> model.
>> 
>> 
>> Pauli
>> -- 
>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
>> Phone +61 7 3031 7217
>> Oracle Australia
>> 

Re: Deprecation

[Richard Levitte]

On Fri, 14 Feb 2020 10:41:05 +0100,
Dmitry Belyavsky wrote:
> 
> 
> Hello,
> 
> On Fri, Feb 14, 2020 at 5:30 AM Dr Paul Dale  wrote:
> 
> There is some pushback against the deprecations going on in various PRs.
>
> The plan has always been to deprecate engines in 3.0 and removing support 
> for them 5+ years later. 
> Originally, the path was to have included an engine provider that could 
> load engines and make them appear
> to be a provider.  After a fair amount of investigation, this was deemed 
> to be too difficult in the 3.0
> time frame.
>
> Do we still want to deprecate engines in 3.0?
> Should we defer until 4.0 instead?
> 
> I think we should delay the deprecation of engine stuff to 4.0. Personally I 
> don't have sense of stability of
> provider API.

It should be pointed out that the engine stuff isn't as stable as you
might think, because it shares internal structures with libcrypto.
Even if we handle those structures via function calls, all it takes is
loading an engine linked against - say - 1.1.0 in 1.1.1 to run a real
risk of a spectacular KABOOM if any of the structure that are touched
changed between those OpenSSL versions.
This is a consequence of making structures opaque and feeling much
more liberty in changing them, and we didn't quite pay attention.

The whole model around providers is intentionally designed to allow
providers to run flexibly with any OpenSSL version, even if they are
linked with some (possibly different) libcrypto version.

So the question of stability depends on what you look at.

It's true that some parts of the provider API is fluctuating a bit, as
early designs need modifications to better meet demands.  We're still
in development.  Come beta1 (schedule for June), I expect that it will
have stabilized.  Come the release, it must have.

> The main benefits seem to boil down to continuing to support existing 
> engines vs removing the legacy code
> paths and switching to the provider model.
> 
> For me, both as open-source and commercial engine developer seems
> reasonable to delay conversion from engines to providers at least
> until 3.0.0 feature freeze happens.

According to our policy, a deprecation starts at the release that has
those deprecations, and removal of deprecated stuff can only happen 5
years later at the earliest.  Seeing deprecations in the source now
doesn't change that, the clock will start ticking when we release
3.0, so consider what you see happening on github as a pre-deprecation
warning.

> But some features I'm interested in imply engine model (and it will
> be great if somebody else could look at PR 10904 to avoid it when
> possible).

Apart from a few details, that PR looks rather innocuous.  I frankly
haven't had time to look at it too deeply (I just discovered that I
was prompted), as I haven't dived in the CMS code yet...

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/

Re: Deprecation

[Matt Caswell]

On 14/02/2020 09:41, Dmitry Belyavsky wrote:
> Hello,
> 
> On Fri, Feb 14, 2020 at 5:30 AM Dr Paul Dale  <mailto:paul.d...@oracle.com>> wrote:
> 
> There is some pushback against the deprecations going on in various PRs.
> 
> The plan has always been to deprecate engines in 3.0 and removing
> support for them 5+ years later.  Originally, the path was to have
> included an engine provider that could load engines and make them
> appear to be a provider.  After a fair amount of investigation, this
> was deemed to be too difficult in the 3.0 time frame.
> 
> Do we still want to deprecate engines in 3.0?
> Should we defer until 4.0 instead?
> 
> 
> I think we should delay the deprecation of engine stuff to 4.0.
> Personally I don't have sense of stability of provider API.

Well - it isn't stable *right now*. Its in development. But moving
forward the provider model *is* the way we intend to go. By the time of
the 3.0 release the provider stuff had better be stable, otherwise we've
gone wrong.

As has been pointed out many times. Deprecation does not mean removal
(yet). Its just a signal that at some later time we will remove it.

And the 3.0 is the right time to signal that. We don't want to hold on
the "legacy" codepaths for any longer than we have to. They were only
ever originally intended to be temporary, and to be removed before 3.0
got released. However it now looks like they will live beyond the 3.0
release. They should not live for any longer than absolutely necessary.

> 
> The main benefits seem to boil down to continuing to support
> existing engines vs removing the legacy code paths and switching to
> the provider model.
> 
> 
> For me, both as open-source and commercial engine developer seems
> reasonable to delay conversion from engines to providers at least until
> 3.0.0 feature freeze happens.

That's a perfectly reasonable strategy. But this decision is independent
of whether we deprecate the ENGINE APIs in 3.0 or not. It would be
perfectly ok for people to continue to *use* engines in 3.0 even though
they are deprecated.


> But some features I'm interested in imply engine model (and it will be
> great if somebody else could look at PR 10904 to avoid it when possible).

If there are gaps then we need to plug them. The provider model should
be able to do everything that you can do now in ENGINEs.

Matt

Re: Deprecation

[Dmitry Belyavsky]

Hello,

On Fri, Feb 14, 2020 at 5:30 AM Dr Paul Dale  wrote:

> There is some pushback against the deprecations going on in various PRs.
>
> The plan has always been to deprecate engines in 3.0 and removing support
> for them 5+ years later.  Originally, the path was to have included an
> engine provider that could load engines and make them appear to be a
> provider.  After a fair amount of investigation, this was deemed to be too
> difficult in the 3.0 time frame.
>
> Do we still want to deprecate engines in 3.0?
> Should we defer until 4.0 instead?
>

I think we should delay the deprecation of engine stuff to 4.0. Personally
I don't have sense of stability of provider API.

The main benefits seem to boil down to continuing to support existing
> engines vs removing the legacy code paths and switching to the provider
> model.
>

For me, both as open-source and commercial engine developer seems
reasonable to delay conversion from engines to providers at least until
3.0.0 feature freeze happens.
But some features I'm interested in imply engine model (and it will be
great if somebody else could look at PR 10904 to avoid it when possible).


-- 
SY, Dmitry Belyavsky

Re: Deprecation

[Matt Caswell]

On 14/02/2020 02:30, Dr Paul Dale wrote:
> There is some pushback against the deprecations going on in various PRs.

I've not followed all of the PRs. Can you point at some specific
comments you've received pushing back on this?

Matt


> 
> The plan has always been to deprecate engines in 3.0 and removing
> support for them 5+ years later.  Originally, the path was to have
> included an engine provider that could load engines and make them appear
> to be a provider.  After a fair amount of investigation, this was deemed
> to be too difficult in the 3.0 time frame.
> 
> Do we still want to deprecate engines in 3.0?
> Should we defer until 4.0 instead?
> 
> 
> The main benefits seem to boil down to continuing to support existing
> engines vs removing the legacy code paths and switching to the provider
> model.
> 
> 
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
> 

Re: Deprecation

[Dmitry Belyavsky]

Hello,

I think OPENSSL_NO_DEPRECATED_1_0_2 should be added to this list too.


On Fri, Feb 14, 2020 at 12:31 PM Matthias St. Pierre <
matthias.st.pie...@ncp-e.com> wrote:

>
>
> On 14.02.20 10:21, Matthias St. Pierre wrote:
> >
> > Because deprecation without removal is futile, it increases complexity
> of the code
> > instead of reducing it.
> >
> > Matthias
> >
>
> To underlinie my last statement, I added the initial release dates:
>
>  OPENSSL_NO_DEPRECATED_0_9_8  # 05 Jul 2005
>  OPENSSL_NO_DEPRECATED_1_0_0  # 29 Mar 2010
>  OPENSSL_NO_DEPRECATED_1_0_1  # 14 Mar 2012
>


-- 
SY, Dmitry Belyavsky

Re: Deprecation

[Matthias St. Pierre]

On 14.02.20 10:21, Matthias St. Pierre wrote:


Because deprecation without removal is futile, it increases complexity of the 
code
instead of reducing it.

Matthias



To underlinie my last statement, I added the initial release dates:

    OPENSSL_NO_DEPRECATED_0_9_8  # 05 Jul 2005
    OPENSSL_NO_DEPRECATED_1_0_0  # 29 Mar 2010
    OPENSSL_NO_DEPRECATED_1_0_1  # 14 Mar 2012

Re: Deprecation

[Matthias St. Pierre]

On 14.02.20 08:24, Tomas Mraz wrote:


I do not understand the pushback too much - Perhaps it could be
resolved by proper explanation that deprecation does not mean a
removal?

Also even if some stuff deprecated in 3.0 is removed in 4.0, it does
not necessarily mean that engines must be removed in the same release.
They can continue to be supported (just deprecated) until 5.0 for
example.

I think that doing the deprecation as early as possible is better - it
properly gives the signal that the engine interface is legacy and it
will disappear at some point. It provides more time for 3rd party
engines to transform into providers.



I agree with Tomas. In addition, I think it is high time to publish a definite 
timescale
for actually *removing* the older deprecated APIs, at least for the first three 
entries
in the list below:

    OPENSSL_NO_DEPRECATED_0_9_8  #
    OPENSSL_NO_DEPRECATED_1_0_0  #
    OPENSSL_NO_DEPRECATED_1_0_1  #

    OPENSSL_NO_DEPRECATED_1_0_2
    OPENSSL_NO_DEPRECATED_1_1_0
    OPENSSL_NO_DEPRECATED_1_1_1
    OPENSSL_NO_DEPRECATED_3_0

Because deprecation without removal is futile, it increases complexity of the 
code
instead of reducing it.

Matthias

Re: Deprecation

[Tomas Mraz]

On Fri, 2020-02-14 at 12:30 +1000, Dr Paul Dale wrote:
> There is some pushback against the deprecations going on in various
> PRs.
> 
> The plan has always been to deprecate engines in 3.0 and removing
> support for them 5+ years later.  Originally, the path was to have
> included an engine provider that could load engines and make them
> appear to be a provider.  After a fair amount of investigation, this
> was deemed to be too difficult in the 3.0 time frame.
> 
> Do we still want to deprecate engines in 3.0?
> Should we defer until 4.0 instead?
> 
> 
> The main benefits seem to boil down to continuing to support existing
> engines vs removing the legacy code paths and switching to the
> provider model.


I do not understand the pushback too much - Perhaps it could be
resolved by proper explanation that deprecation does not mean a
removal?

Also even if some stuff deprecated in 3.0 is removed in 4.0, it does
not necessarily mean that engines must be removed in the same release.
They can continue to be supported (just deprecated) until 5.0 for
example.

I think that doing the deprecation as early as possible is better - it
properly gives the signal that the engine interface is legacy and it
will disappear at some point. It provides more time for 3rd party
engines to transform into providers.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]

Deprecation

[Dr Paul Dale]

There is some pushback against the deprecations going on in various PRs.

The plan has always been to deprecate engines in 3.0 and removing support for 
them 5+ years later.  Originally, the path was to have included an engine 
provider that could load engines and make them appear to be a provider.  After 
a fair amount of investigation, this was deemed to be too difficult in the 3.0 
time frame.

Do we still want to deprecate engines in 3.0?
Should we defer until 4.0 instead?


The main benefits seem to boil down to continuing to support existing engines 
vs removing the legacy code paths and switching to the provider model.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia

Re: Deprecation of stuff

[Viktor Dukhovni]

+1 (and more) to the below!

> On Sep 4, 2019, at 10:15 AM, David Woodhouse  wrote:
> 
> I'd note that the question of *versioning* mechanisms is a very very
> special case of "when to deprecate stuff". So much so as to almost make
> it a completely separate question altogether.
> 
> My own favourite application is littered with checks on
> OPENSSL_VERSION_NUMBER, and the occasional call to SSLeay() to check
> for things that were fixed at runtime without an ABI change.
> 
> http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/openssl.c
> http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/openssl-dtls.c
> 
> A change to the versioning scheme is very much more than just another
> thing that's been deprecated; it's a change to the very mechanism by
> which we handle those deprecations. Changing that (and by extension,
> rapidly deprecating it) requires a lot more work on the part of
> application authors who want their code to build against whatever
> version of OpenSSL may be present on the platforms they need to
> support.

-- 
Viktor.

Re: Deprecation of stuff

[David Woodhouse]

On Mon, 2019-09-02 at 08:38 +0200, Richard Levitte wrote:
> The dispute in PR https://github.com/openssl/openssl/pull/7853 has
> made it quote obvious that we have some very different ideas on when
> and why we should or shouldn't deprecate stuff.
> 
> What does deprecation mean?  Essentially, it's a warning that at some
> point in the future, the deprecated functionality will be removed.  I
> believe that part of the issue surrounding this is uncertainty about
> when that removal will happen, so let me just remind you what's
> written in our release strategy document:
> 
>   * No existing public interface can be removed until its replacement
> has been in place in an LTS stable release. The original interface
> must also have been documented as deprecated for at least 5 years.
> A public interface is any function, structure or macro declared in
> a public header file.
> 
> Ref: https://www.openssl.org/policies/releasestrat.html
> 
> I'd like to get this thread started with the following four questions,
> for as many of us to answer as possible:
> 
> 1. Why should we deprecate stuff
> 
> 2. Why should we not deprecate stuff
> 
> 3. When should we deprecate stuff
> 
> 4. When should we not deprecate stuff
> 
> Cheers,

(I know -project won't accept my mail; you can choose to include it in
a reply if you see fit).


I'd note that the question of *versioning* mechanisms is a very very
special case of "when to deprecate stuff". So much so as to almost make
it a completely separate question altogether.

My own favourite application is littered with checks on
OPENSSL_VERSION_NUMBER, and the occasional call to SSLeay() to check
for things that were fixed at runtime without an ABI change.

http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/openssl.c
http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/openssl-dtls.c

A change to the versioning scheme is very much more than just another
thing that's been deprecated; it's a change to the very mechanism by
which we handle those deprecations. Changing that (and by extension,
rapidly deprecating it) requires a lot more work on the part of
application authors who want their code to build against whatever
version of OpenSSL may be present on the platforms they need to
support.





smime.p7s
Description: S/MIME cryptographic signature

Re: Deprecation of stuff

[Viktor Dukhovni]

On Wed, Sep 04, 2019 at 02:43:34PM +0200, Tomas Mraz wrote:

> > The dispute in PR https://github.com/openssl/openssl/pull/7853 has
> > made it quote obvious that we have some very different ideas on when
> > and why we should or shouldn't deprecate stuff.
> > 
> > What does deprecation mean?  Essentially, it's a warning that at some
> > point in the future, the deprecated functionality will be removed.  I
> > believe that part of the issue surrounding this is uncertainty about
> > when that removal will happen, so let me just remind you what's
> > written in our release strategy document:

Actually, my issue was not timing, but whether the particular feature
warrants eventual removal.  I don't believe it does.

> > 1. Why should we deprecate stuff
> 
> Because keeping every legacy API/feature/option/... increases the
> maintenance burden, attack surface, confuses users/developers, and in
> general hinders the development.
> 
> > 2. Why should we not deprecate stuff
> 
> If something does not really have an adequate replacement, it does not
> really increase the maintenance burden, does not significantly increase
> the attack surface, and is still used widely in applications, it should
> not be deprecated.

This is essentially the basis of my objection, with less emphasis
on "adequate replacement".  Just because we *can* ask users to
rewrite their code, does not mean we *should*.

-- 
Viktor.

Re: Deprecation of stuff

[Tomas Mraz]

On Mon, 2019-09-02 at 08:38 +0200, Richard Levitte wrote:
> The dispute in PR https://github.com/openssl/openssl/pull/7853 has
> made it quote obvious that we have some very different ideas on when
> and why we should or shouldn't deprecate stuff.
> 
> What does deprecation mean?  Essentially, it's a warning that at some
> point in the future, the deprecated functionality will be removed.  I
> believe that part of the issue surrounding this is uncertainty about
> when that removal will happen, so let me just remind you what's
> written in our release strategy document:
> 
>   * No existing public interface can be removed until its replacement
> has been in place in an LTS stable release. The original
> interface
> must also have been documented as deprecated for at least 5
> years.
> A public interface is any function, structure or macro declared
> in
> a public header file.
> 
> Ref: https://www.openssl.org/policies/releasestrat.html
> 
> I'd like to get this thread started with the following four
> questions,
> for as many of us to answer as possible:
> 
> 1. Why should we deprecate stuff

Because keeping every legacy API/feature/option/... increases the
maintenance burden, attack surface, confuses users/developers, and in
general hinders the development.

> 2. Why should we not deprecate stuff

If something does not really have an adequate replacement, it does not
really increase the maintenance burden, does not significantly increase
the attack surface, and is still used widely in applications, it should
not be deprecated.

> 3. When should we deprecate stuff

As soon as possible when there is a better replacement for the stuff. I
believe it is better to give the warning about future removal as soon
as possible rather than to plan deprecating and later removing
something anyway but delay the deprecation "to not scare someone
early".

> 4. When should we not deprecate stuff


-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]

Deprecation of stuff

[Richard Levitte]

The dispute in PR https://github.com/openssl/openssl/pull/7853 has
made it quote obvious that we have some very different ideas on when
and why we should or shouldn't deprecate stuff.

What does deprecation mean?  Essentially, it's a warning that at some
point in the future, the deprecated functionality will be removed.  I
believe that part of the issue surrounding this is uncertainty about
when that removal will happen, so let me just remind you what's
written in our release strategy document:

  * No existing public interface can be removed until its replacement
has been in place in an LTS stable release. The original interface
must also have been documented as deprecated for at least 5 years.
A public interface is any function, structure or macro declared in
a public header file.

Ref: https://www.openssl.org/policies/releasestrat.html

I'd like to get this thread started with the following four questions,
for as many of us to answer as possible:

1. Why should we deprecate stuff

2. Why should we not deprecate stuff

3. When should we deprecate stuff

4. When should we not deprecate stuff

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/