Re: Vote proposal: Technical items still to be done

2020-10-08 Thread Matt Caswell
Thanks for the feedback on this thread.

There was some feedback specifically on the vote text. I've made the
amendment suggested by Tomas, and numbered the items as suggested by
Nicola. I did not convert to markdown.

The other discussions on this thread I think are useful things to think
about when considering how to vote - but I don't think change the vote
text itself - so I've not made any changes in response to those things.

I'll shortly start this vote.

Matt


On 07/10/2020 12:35, Matt Caswell wrote:
> I had an action from the OTC meeting today to raise a vote on the OTC
> list of technical items still to be done. Here is my proposed vote text.
> There will be a subsequent vote on the "beta readiness checklist" which
> is a separate list.
> 
> Feedback please on the proposed vote text below.
> 
> The following items are required prerequisites for the first beta release:
> * EVP is the recommended API, it must be feature-complete compared with
> the functionality available using lower-level APIs.
>   - Anything that isn’t available must be put to an OTC vote to exclude.
>   - The apps are the minimum bar for this, subject to exceptions noted
> below.
> * Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_,
> RAND_METHOD_.
>   - Does not include macros defining useful constants (e.g.
> SHA512_DIGEST_LENGTH).
>   - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`.
>   - There might be some others.
>   - Review for exceptions.
>   - The apps are the minimum bar to measure feature completeness for the
> EVP interface: rewrite them so they do not use internal nor deprecated
> functions (except speed, engine, list, passwd -crypt and the code to
> handle the -engine CLI option).  That is, remove the suppression of
> deprecated define.
> - Proposal: drop passwd -crypt (OMC vote required)
>   - Compile and link 1.1.1 command line app against the master headers
> and library.  Run 1.1.1 app test cases against the chimera.  Treat this
> as an external test using a special 1.1.1 branch.
> Deprecated functions used by libssl should be moved to independent
> file(s), to limit the suppression of deprecated defines to the absolute
> minimum scope.
> * Draft documentation (contents but not pretty)
>   - Need a list of things we know are not present - including things we
> have removed.
>   - We need to have mapping tables for various d2i/i2d functions.
>   - We need to have a mapping table from “old names” for things into the
> OSSL_PARAMS names.
> - Documentation addition to old APIs to refer to new ones (man7).
> - Documentation needs to reference name mapping.
> - All the legacy interfaces need to have their documentation
> pointing to the replacement interfaces.
> * Review (and maybe clean up) legacy bridge code.
> * Review TODO(3.0) items #12224.
> * Source checksum script.
> * Review of functions previously named _with_libctx.
> * Encoder fixers (PKCS#8, PKCS#1, etc).
> * Encoder DER to PEM refactor.
> * Builds and passes tests on all primary, secondary and FIPS platforms.
> * Query provider parameters (name, version, …) from the command line.
> * Setup buildbot infrastructure and associated instructions.
> * Complete make fipsinstall.
> * More specific decoding selection (e.g. params or keys).
> * Example code covering replacements for deprecated APIs.
> * Drop C code output options from the apps (OMC approval required).
> * Address 3.0beta1 milestones.
> 
> 
> Matt
> 


Re: Vote proposal: Technical items still to be done

2020-10-08 Thread Salz, Rich
>Of course the 3.0 release is kind of special because we are defining a
completely new API - the providers API - with the intention to have it
stable for many years to come. Any bugs in the API design for providers
will have to live with us at least until the 4.0 release and so it is
reasonable goal to avoid them if at all possible.

There's two parts to this, bugs within the API, and errors in the API 
definition itself.  You're talking about the latter, which is the more 
important thing.

Nowhere in this thread, or elsewhere, has there been any mention of how to test 
that the provider API is correct.  We have the existence proof of OpenSSL, but 
that's it. There are beliefs that it works, concerns about things missing, but 
no other running code.  If the provider API is paramount, then perhaps 
additional proof-points are needed?

>Unfortunately the release requirements as defined in the proposal for
OTC vote come fairly naturally from the feature requirements set by OMC

Where can I find those?  If they were posted I missed it.  If it's the 3.0 
design document [1] then, for example, I would say that the deprecation 
requirements are met because it doesn't mandate "remove from the code" style of 
deprecation.  But maybe there is another list of 3.0 requirements that I am 
missing.  Any help

[1] https://www.openssl.org/docs/OpenSSL300Design.html




Re: Vote proposal: Technical items still to be done

2020-10-08 Thread Tomas Mraz
On Thu, 2020-10-08 at 09:47 +0100, Matt Caswell wrote:
> 
> On 07/10/2020 19:07, Kurt Roeckx wrote:
> > On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote:
> > > The following items are required prerequisites for the first beta
> > > release:
> > [...]
> > > * Address 3.0beta1 milestones.
> > 
> > So we now have a list here, with the last item pointing to a
> > different list. I suggest that we only have 1 list, and that we
> > make github issues for anything that doesn't have an issue yet,
> > and it to the milestone. It would at least be easier to track the
> > progress.
> 
> I agree, although its bit of a chicken-and-egg problem. I'd suggest
> voting on the list as is, and then if the vote passes converting all
> of
> those things into issues.

+1

> > We also talked about freezing the milestone at some point. We
> > intend to go over the 3.0.0 milestone PRs/issues, and see if any of
> > those need to be moved to the 3.0.0 beta1 milestone. I assume that
> > if we have done that, that we'll freeze it. We can then try to
> > estimate how long it will take before beta1.
> 
> Yes.
> 
> Matt
> 
-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]




Re: Vote proposal: Technical items still to be done

2020-10-08 Thread Matt Caswell



On 07/10/2020 19:07, Kurt Roeckx wrote:
> On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote:
>> The following items are required prerequisites for the first beta release:
> [...]
>> * Address 3.0beta1 milestones.
> 
> So we now have a list here, with the last item pointing to a
> different list. I suggest that we only have 1 list, and that we
> make github issues for anything that doesn't have an issue yet,
> and it to the milestone. It would at least be easier to track the
> progress.

I agree, although its bit of a chicken-and-egg problem. I'd suggest
voting on the list as is, and then if the vote passes converting all of
those things into issues.

> 
> We also talked about freezing the milestone at some point. We
> intend to go over the 3.0.0 milestone PRs/issues, and see if any of
> those need to be moved to the 3.0.0 beta1 milestone. I assume that
> if we have done that, that we'll freeze it. We can then try to
> estimate how long it will take before beta1.

Yes.

Matt



Re: Vote proposal: Technical items still to be done

2020-10-08 Thread Tomas Mraz
+1 to that. There is a reason why almost all the major projects
switched over to doing time based releases instead of feature
completeness based ones.

Of course the 3.0 release is kind of special because we are defining a
completely new API - the providers API - with the intention to have it
stable for many years to come. Any bugs in the API design for providers
will have to live with us at least until the 4.0 release and so it is
reasonable goal to avoid them if at all possible.

But yes, I am also feeling that the release requirements as defined in
the proposal are a little bit too much on the side of the perfect, the
enemy of the good and it would not be a major problem if some of them
were not there.

Unfortunately the release requirements as defined in the proposal for
OTC vote come fairly naturally from the feature requirements set by OMC
so if we would like to avoid some of them I think that OMC would have
to first amend the feature requirements for the 3.0 release.

On Wed, 2020-10-07 at 21:57 -0700, Benjamin Kaduk wrote:
> On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote:
> > I had an action from the OTC meeting today to raise a vote on the
> > OTC
> > list of technical items still to be done. Here is my proposed vote
> > text.
> > There will be a subsequent vote on the "beta readiness checklist"
> > which
> > is a separate list.
> 
> Thanks, Matt; this is a fine list of desirable things.  (I tried to
> resist
> commenting on specific items, but building the 1.1.1 apps+tests
> against 3.0
> libraries is a solid way to help meet our stability goals, and should
> arguably be something that run-checker just does, all the time.)
> 
> I've trimmed the list, though, to make a broader point, which could
> be
> summarized as "the perfect is the enemy of the good".
> 
> It's a natural consequence for a software project that has long-term
> supported releases, strong API stability guarantees, and infrequent
> releases, to feel that each major release is a critical threshold and
> that
> we need to do a bunch of tidying before the release to wrap it up
> nicely.
> Paying off technical debt is often a bit part of the tidying that is
> perceived necessary, as is attempting to be future looking -- missing
> the
> release, after all, is the difference between needing to support some
> bad/annoying thing for (say) 8 years instead of "only" 5.
> 
> There is value in doing this fixup, yes, but there is also cost in
> keeping
> all the good things (and other fixup) that is already done out of the
> hands
> of those who wish to consume it.  I've seen this phenomenon bite
> various
> projects over time with effects of different magnitude, varying from
> FreeBSD 5.0+SMP that had nearly existential consequences on the
> project, to
> OpenAFS 1.6.0 where release delays produced enough frustration to
> lead to a
> bit of a rush-job final release that was a bit unstable; I was lucky
> enough
> to have missed the worst of this effect for krb5, and managed to do a
> little better (but still not great) for OpenAFS 1.8.0.
> 
> Projects that get hit really badly by this phenomenon tend to correct
> for
> it and end up on a fixed time-based cadence of releases, and in order
> to
> stick to that cadence end up having to get comfortable shipping
> releases
> without a desired feature or that are known to have incomplete parts
> of one
> feature or another.  It also requires discipline to keep the main
> development branch always (or nearly so) in a releasable state, but
> in my
> opinion we are already doing a pretty good job of that with our
> policies
> for code review and unit tests.  (We could, of course, do better
> about
> monitoring the extended tests, run checker, and the like, but when we
> do
> have regressions getting them fixed is not an invasive mess.)
> 
> To list some examples:
> 
> FreeBSD aims for new major ("dot zero") releases every two years.
> MIT krb5 puts out new releases annually.
> Google Chrome puts out releases every 6 weeks.
> [Okay, I haven't exactly moved OpenAFS over to this schedule yet, but
> I did
> just today cut a 1.9.0 and am going to try to put out regular 1.9.x
> versions.]
> 
> I would urge the OTC (and OMC) to be careful around the pitfalls of
> making
> the perfect the enemy of the good, and be willing to accept some
> level of
> "uncleanliness" in the interest of getting all the good things we
> have
> already done out there in production releases.  (And also trying to
> not
> slip the published schedule too much.)  It is unpleasant, yes, but
> sometimes it is the best choice overall.
> 
> Thanks,
> 
> Ben
-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]




Re: Vote proposal: Technical items still to be done

2020-10-07 Thread Benjamin Kaduk
On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote:
> I had an action from the OTC meeting today to raise a vote on the OTC
> list of technical items still to be done. Here is my proposed vote text.
> There will be a subsequent vote on the "beta readiness checklist" which
> is a separate list.

Thanks, Matt; this is a fine list of desirable things.  (I tried to resist
commenting on specific items, but building the 1.1.1 apps+tests against 3.0
libraries is a solid way to help meet our stability goals, and should
arguably be something that run-checker just does, all the time.)

I've trimmed the list, though, to make a broader point, which could be
summarized as "the perfect is the enemy of the good".

It's a natural consequence for a software project that has long-term
supported releases, strong API stability guarantees, and infrequent
releases, to feel that each major release is a critical threshold and that
we need to do a bunch of tidying before the release to wrap it up nicely.
Paying off technical debt is often a bit part of the tidying that is
perceived necessary, as is attempting to be future looking -- missing the
release, after all, is the difference between needing to support some
bad/annoying thing for (say) 8 years instead of "only" 5.

There is value in doing this fixup, yes, but there is also cost in keeping
all the good things (and other fixup) that is already done out of the hands
of those who wish to consume it.  I've seen this phenomenon bite various
projects over time with effects of different magnitude, varying from
FreeBSD 5.0+SMP that had nearly existential consequences on the project, to
OpenAFS 1.6.0 where release delays produced enough frustration to lead to a
bit of a rush-job final release that was a bit unstable; I was lucky enough
to have missed the worst of this effect for krb5, and managed to do a
little better (but still not great) for OpenAFS 1.8.0.

Projects that get hit really badly by this phenomenon tend to correct for
it and end up on a fixed time-based cadence of releases, and in order to
stick to that cadence end up having to get comfortable shipping releases
without a desired feature or that are known to have incomplete parts of one
feature or another.  It also requires discipline to keep the main
development branch always (or nearly so) in a releasable state, but in my
opinion we are already doing a pretty good job of that with our policies
for code review and unit tests.  (We could, of course, do better about
monitoring the extended tests, run checker, and the like, but when we do
have regressions getting them fixed is not an invasive mess.)

To list some examples:

FreeBSD aims for new major ("dot zero") releases every two years.
MIT krb5 puts out new releases annually.
Google Chrome puts out releases every 6 weeks.
[Okay, I haven't exactly moved OpenAFS over to this schedule yet, but I did
just today cut a 1.9.0 and am going to try to put out regular 1.9.x
versions.]

I would urge the OTC (and OMC) to be careful around the pitfalls of making
the perfect the enemy of the good, and be willing to accept some level of
"uncleanliness" in the interest of getting all the good things we have
already done out there in production releases.  (And also trying to not
slip the published schedule too much.)  It is unpleasant, yes, but
sometimes it is the best choice overall.

Thanks,

Ben


Re: Vote proposal: Technical items still to be done

2020-10-07 Thread Kurt Roeckx
On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote:
> The following items are required prerequisites for the first beta release:
[...]
> * Address 3.0beta1 milestones.

So we now have a list here, with the last item pointing to a
different list. I suggest that we only have 1 list, and that we
make github issues for anything that doesn't have an issue yet,
and it to the milestone. It would at least be easier to track the
progress.

We also talked about freezing the milestone at some point. We
intend to go over the 3.0.0 milestone PRs/issues, and see if any of
those need to be moved to the 3.0.0 beta1 milestone. I assume that
if we have done that, that we'll freeze it. We can then try to
estimate how long it will take before beta1.


Kurt



Re: Vote proposal: Technical items still to be done

2020-10-07 Thread Nicola Tuveri
I support the edit proposed by Tomas.

First comment that I have is that I'd prefer the first-level items to
be actually numbered, as done in the drafts: we might put a disclaimer
that the numbers are not indicative of priority and just meant to be
used to address (equally important) tasks by index.

The second comment is just a quirk of mine: I prefer plain text emails
and Markdown formatting. If others shared the feeling (and nobody
objected to Markdown for our records) I'd volunteer to reformat the
draft of this vote text using Markdown syntax.

Nicola

On Wed, 7 Oct 2020 at 14:58, Tomas Mraz  wrote:
>
> On Wed, 2020-10-07 at 12:35 +0100, Matt Caswell wrote:
> > I had an action from the OTC meeting today to raise a vote on the OTC
> > list of technical items still to be done. Here is my proposed vote
> > text.
> > There will be a subsequent vote on the "beta readiness checklist"
> > which
> > is a separate list.
> >
> > Feedback please on the proposed vote text below.
> >
> > The following items are required prerequisites for the first beta
> > release:
> > * EVP is the recommended API, it must be feature-complete compared
> > with
> > the functionality available using lower-level APIs.
> >   - Anything that isn’t available must be put to an OTC vote to
> > exclude.
> >   - The apps are the minimum bar for this, subject to exceptions
> > noted
> > below.
> > * Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_,
> > RAND_METHOD_.
> >   - Does not include macros defining useful constants (e.g.
> > SHA512_DIGEST_LENGTH).
> >   - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`.
> >   - There might be some others.
> >   - Review for exceptions.
> >   - The apps are the minimum bar to measure feature completeness for
> > the
> > EVP interface: rewrite them so they do not use internal nor
> > deprecated
> > functions (except speed, engine, list, passwd -crypt and the code to
> > handle the -engine CLI option).  That is, remove the suppression of
> > deprecated define.
> > - Proposal: drop passwd -crypt (OMC vote required)
> >   - Compile and link 1.1.1 command line app against the master
> > headers
> > and library.  Run 1.1.1 app test cases against the chimera.  Treat
> > this
> > as an external test using a special 1.1.1 branch.
> > Deprecated functions used by libssl should be moved to independent
> > file(s), to limit the suppression of deprecated defines to the
> > absolute
> > minimum scope.
> > * Draft documentation (contents but not pretty)
> >   - Need a list of things we know are not present - including things
> > we
> > have removed.
> >   - We need to have mapping tables for various d2i/i2d functions.
> >   - We need to have a mapping table from “old names” for things into
> > the
> > OSSL_PARAMS names.
> > - Documentation addition to old APIs to refer to new ones (man7).
> > - Documentation needs to reference name mapping.
> > - All the legacy interfaces need to have their documentation
> > pointing to the replacement interfaces.
> > * Review (and maybe clean up) legacy bridge code.
> > * Review TODO(3.0) items #12224.
> > * Source checksum script.
> > * Review of functions previously named _with_libctx.
> > * Encoder fixers (PKCS#8, PKCS#1, etc).
> > * Encoder DER to PEM refactor.
> > * Builds and passes tests on all primary, secondary and FIPS
> > platforms.
> > * Query provider parameters (name, version, …) from the command line.
> > * Setup buildbot infrastructure and associated instructions.
> > * Complete make fipsinstall.
> > * More specific decoding selection (e.g. params or keys).
> > * Example code covering replacements for deprecated APIs.
> > * Drop C code output options from the apps (OMC approval required).
> > * Address 3.0beta1 milestones.
>
> Address issues and PRs in the 3.0beta1 milestone.
>
> --
> Tomáš Mráz
> No matter how far down the wrong road you've gone, turn back.
>   Turkish proverb
> [You'll know whether the road is wrong if you carefully listen to your
> conscience.]
>
>


Re: Vote proposal: Technical items still to be done

2020-10-07 Thread Tomas Mraz
On Wed, 2020-10-07 at 12:35 +0100, Matt Caswell wrote:
> I had an action from the OTC meeting today to raise a vote on the OTC
> list of technical items still to be done. Here is my proposed vote
> text.
> There will be a subsequent vote on the "beta readiness checklist"
> which
> is a separate list.
> 
> Feedback please on the proposed vote text below.
> 
> The following items are required prerequisites for the first beta
> release:
> * EVP is the recommended API, it must be feature-complete compared
> with
> the functionality available using lower-level APIs.
>   - Anything that isn’t available must be put to an OTC vote to
> exclude.
>   - The apps are the minimum bar for this, subject to exceptions
> noted
> below.
> * Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_,
> RAND_METHOD_.
>   - Does not include macros defining useful constants (e.g.
> SHA512_DIGEST_LENGTH).
>   - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`.
>   - There might be some others.
>   - Review for exceptions.
>   - The apps are the minimum bar to measure feature completeness for
> the
> EVP interface: rewrite them so they do not use internal nor
> deprecated
> functions (except speed, engine, list, passwd -crypt and the code to
> handle the -engine CLI option).  That is, remove the suppression of
> deprecated define.
> - Proposal: drop passwd -crypt (OMC vote required)
>   - Compile and link 1.1.1 command line app against the master
> headers
> and library.  Run 1.1.1 app test cases against the chimera.  Treat
> this
> as an external test using a special 1.1.1 branch.
> Deprecated functions used by libssl should be moved to independent
> file(s), to limit the suppression of deprecated defines to the
> absolute
> minimum scope.
> * Draft documentation (contents but not pretty)
>   - Need a list of things we know are not present - including things
> we
> have removed.
>   - We need to have mapping tables for various d2i/i2d functions.
>   - We need to have a mapping table from “old names” for things into
> the
> OSSL_PARAMS names.
> - Documentation addition to old APIs to refer to new ones (man7).
> - Documentation needs to reference name mapping.
> - All the legacy interfaces need to have their documentation
> pointing to the replacement interfaces.
> * Review (and maybe clean up) legacy bridge code.
> * Review TODO(3.0) items #12224.
> * Source checksum script.
> * Review of functions previously named _with_libctx.
> * Encoder fixers (PKCS#8, PKCS#1, etc).
> * Encoder DER to PEM refactor.
> * Builds and passes tests on all primary, secondary and FIPS
> platforms.
> * Query provider parameters (name, version, …) from the command line.
> * Setup buildbot infrastructure and associated instructions.
> * Complete make fipsinstall.
> * More specific decoding selection (e.g. params or keys).
> * Example code covering replacements for deprecated APIs.
> * Drop C code output options from the apps (OMC approval required).
> * Address 3.0beta1 milestones.

Address issues and PRs in the 3.0beta1 milestone.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]




Vote proposal: Technical items still to be done

2020-10-07 Thread Matt Caswell
I had an action from the OTC meeting today to raise a vote on the OTC
list of technical items still to be done. Here is my proposed vote text.
There will be a subsequent vote on the "beta readiness checklist" which
is a separate list.

Feedback please on the proposed vote text below.

The following items are required prerequisites for the first beta release:
* EVP is the recommended API, it must be feature-complete compared with
the functionality available using lower-level APIs.
  - Anything that isn’t available must be put to an OTC vote to exclude.
  - The apps are the minimum bar for this, subject to exceptions noted
below.
* Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_,
RAND_METHOD_.
  - Does not include macros defining useful constants (e.g.
SHA512_DIGEST_LENGTH).
  - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`.
  - There might be some others.
  - Review for exceptions.
  - The apps are the minimum bar to measure feature completeness for the
EVP interface: rewrite them so they do not use internal nor deprecated
functions (except speed, engine, list, passwd -crypt and the code to
handle the -engine CLI option).  That is, remove the suppression of
deprecated define.
- Proposal: drop passwd -crypt (OMC vote required)
  - Compile and link 1.1.1 command line app against the master headers
and library.  Run 1.1.1 app test cases against the chimera.  Treat this
as an external test using a special 1.1.1 branch.
Deprecated functions used by libssl should be moved to independent
file(s), to limit the suppression of deprecated defines to the absolute
minimum scope.
* Draft documentation (contents but not pretty)
  - Need a list of things we know are not present - including things we
have removed.
  - We need to have mapping tables for various d2i/i2d functions.
  - We need to have a mapping table from “old names” for things into the
OSSL_PARAMS names.
- Documentation addition to old APIs to refer to new ones (man7).
- Documentation needs to reference name mapping.
- All the legacy interfaces need to have their documentation
pointing to the replacement interfaces.
* Review (and maybe clean up) legacy bridge code.
* Review TODO(3.0) items #12224.
* Source checksum script.
* Review of functions previously named _with_libctx.
* Encoder fixers (PKCS#8, PKCS#1, etc).
* Encoder DER to PEM refactor.
* Builds and passes tests on all primary, secondary and FIPS platforms.
* Query provider parameters (name, version, …) from the command line.
* Setup buildbot infrastructure and associated instructions.
* Complete make fipsinstall.
* More specific decoding selection (e.g. params or keys).
* Example code covering replacements for deprecated APIs.
* Drop C code output options from the apps (OMC approval required).
* Address 3.0beta1 milestones.


Matt