Re: Vote proposal: Technical items still to be done
On Thu, 2020-10-08 at 12:00 +, Salz, Rich wrote: > >Of course the 3.0 release is kind of special because we are > > defining a > completely new API - the providers API - with the intention to > have it > stable for many years to come. Any bugs in the API design for > providers > will have to live with us at least until the 4.0 release and so > it is > reasonable goal to avoid them if at all possible. > > There's two parts to this, bugs within the API, and errors in the API > definition itself. You're talking about the latter, which is the > more important thing. > > Nowhere in this thread, or elsewhere, has there been any mention of > how to test that the provider API is correct. We have the existence > proof of OpenSSL, but that's it. There are beliefs that it works, > concerns about things missing, but no other running code. If the > provider API is paramount, then perhaps additional proof-points are > needed? > > >Unfortunately the release requirements as defined in the > > proposal for > OTC vote come fairly naturally from the feature requirements set > by OMC > > Where can I find those? If they were posted I missed it. If it's > the 3.0 design document [1] then, for example, I would say that the > deprecation requirements are met because it doesn't mandate "remove > from the code" style of deprecation. But maybe there is another list > of 3.0 requirements that I am missing. Any help Yes, we are talking here about the 3.0 design document. And no, there is no "remove from the code" deprecation kind of. There is no such thing on this list that we are voting upon either, except for two things - C code output and passwd -crypt - and both of them are explicitly mentioned to require OMC vote to drop completely. So this is misunderstanding. The problem is that the current state of the API does not deprecate (as in add deprecation warnings and disable in no- deprecated build) many low level API things that are supposed to be deprecated. And we list them here in this list as a requirement to be finished before the 3.0 can be declared feature complete. > [1] https://www.openssl.org/docs/OpenSSL300Design.html -- Tomáš Mráz No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.]
Re: Vote proposal: Technical items still to be done
Thanks for the feedback on this thread. There was some feedback specifically on the vote text. I've made the amendment suggested by Tomas, and numbered the items as suggested by Nicola. I did not convert to markdown. The other discussions on this thread I think are useful things to think about when considering how to vote - but I don't think change the vote text itself - so I've not made any changes in response to those things. I'll shortly start this vote. Matt On 07/10/2020 12:35, Matt Caswell wrote: > I had an action from the OTC meeting today to raise a vote on the OTC > list of technical items still to be done. Here is my proposed vote text. > There will be a subsequent vote on the "beta readiness checklist" which > is a separate list. > > Feedback please on the proposed vote text below. > > The following items are required prerequisites for the first beta release: > * EVP is the recommended API, it must be feature-complete compared with > the functionality available using lower-level APIs. > - Anything that isn’t available must be put to an OTC vote to exclude. > - The apps are the minimum bar for this, subject to exceptions noted > below. > * Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_, > RAND_METHOD_. > - Does not include macros defining useful constants (e.g. > SHA512_DIGEST_LENGTH). > - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`. > - There might be some others. > - Review for exceptions. > - The apps are the minimum bar to measure feature completeness for the > EVP interface: rewrite them so they do not use internal nor deprecated > functions (except speed, engine, list, passwd -crypt and the code to > handle the -engine CLI option). That is, remove the suppression of > deprecated define. > - Proposal: drop passwd -crypt (OMC vote required) > - Compile and link 1.1.1 command line app against the master headers > and library. Run 1.1.1 app test cases against the chimera. Treat this > as an external test using a special 1.1.1 branch. > Deprecated functions used by libssl should be moved to independent > file(s), to limit the suppression of deprecated defines to the absolute > minimum scope. > * Draft documentation (contents but not pretty) > - Need a list of things we know are not present - including things we > have removed. > - We need to have mapping tables for various d2i/i2d functions. > - We need to have a mapping table from “old names” for things into the > OSSL_PARAMS names. > - Documentation addition to old APIs to refer to new ones (man7). > - Documentation needs to reference name mapping. > - All the legacy interfaces need to have their documentation > pointing to the replacement interfaces. > * Review (and maybe clean up) legacy bridge code. > * Review TODO(3.0) items #12224. > * Source checksum script. > * Review of functions previously named _with_libctx. > * Encoder fixers (PKCS#8, PKCS#1, etc). > * Encoder DER to PEM refactor. > * Builds and passes tests on all primary, secondary and FIPS platforms. > * Query provider parameters (name, version, …) from the command line. > * Setup buildbot infrastructure and associated instructions. > * Complete make fipsinstall. > * More specific decoding selection (e.g. params or keys). > * Example code covering replacements for deprecated APIs. > * Drop C code output options from the apps (OMC approval required). > * Address 3.0beta1 milestones. > > > Matt >
Re: Vote proposal: Technical items still to be done
>Of course the 3.0 release is kind of special because we are defining a completely new API - the providers API - with the intention to have it stable for many years to come. Any bugs in the API design for providers will have to live with us at least until the 4.0 release and so it is reasonable goal to avoid them if at all possible. There's two parts to this, bugs within the API, and errors in the API definition itself. You're talking about the latter, which is the more important thing. Nowhere in this thread, or elsewhere, has there been any mention of how to test that the provider API is correct. We have the existence proof of OpenSSL, but that's it. There are beliefs that it works, concerns about things missing, but no other running code. If the provider API is paramount, then perhaps additional proof-points are needed? >Unfortunately the release requirements as defined in the proposal for OTC vote come fairly naturally from the feature requirements set by OMC Where can I find those? If they were posted I missed it. If it's the 3.0 design document [1] then, for example, I would say that the deprecation requirements are met because it doesn't mandate "remove from the code" style of deprecation. But maybe there is another list of 3.0 requirements that I am missing. Any help [1] https://www.openssl.org/docs/OpenSSL300Design.html
Re: Vote proposal: Technical items still to be done
On Thu, 2020-10-08 at 09:47 +0100, Matt Caswell wrote: > > On 07/10/2020 19:07, Kurt Roeckx wrote: > > On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote: > > > The following items are required prerequisites for the first beta > > > release: > > [...] > > > * Address 3.0beta1 milestones. > > > > So we now have a list here, with the last item pointing to a > > different list. I suggest that we only have 1 list, and that we > > make github issues for anything that doesn't have an issue yet, > > and it to the milestone. It would at least be easier to track the > > progress. > > I agree, although its bit of a chicken-and-egg problem. I'd suggest > voting on the list as is, and then if the vote passes converting all > of > those things into issues. +1 > > We also talked about freezing the milestone at some point. We > > intend to go over the 3.0.0 milestone PRs/issues, and see if any of > > those need to be moved to the 3.0.0 beta1 milestone. I assume that > > if we have done that, that we'll freeze it. We can then try to > > estimate how long it will take before beta1. > > Yes. > > Matt > -- Tomáš Mráz No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.]
Re: Vote proposal: Technical items still to be done
On 07/10/2020 19:07, Kurt Roeckx wrote: > On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote: >> The following items are required prerequisites for the first beta release: > [...] >> * Address 3.0beta1 milestones. > > So we now have a list here, with the last item pointing to a > different list. I suggest that we only have 1 list, and that we > make github issues for anything that doesn't have an issue yet, > and it to the milestone. It would at least be easier to track the > progress. I agree, although its bit of a chicken-and-egg problem. I'd suggest voting on the list as is, and then if the vote passes converting all of those things into issues. > > We also talked about freezing the milestone at some point. We > intend to go over the 3.0.0 milestone PRs/issues, and see if any of > those need to be moved to the 3.0.0 beta1 milestone. I assume that > if we have done that, that we'll freeze it. We can then try to > estimate how long it will take before beta1. Yes. Matt
Re: Vote proposal: Technical items still to be done
+1 to that. There is a reason why almost all the major projects switched over to doing time based releases instead of feature completeness based ones. Of course the 3.0 release is kind of special because we are defining a completely new API - the providers API - with the intention to have it stable for many years to come. Any bugs in the API design for providers will have to live with us at least until the 4.0 release and so it is reasonable goal to avoid them if at all possible. But yes, I am also feeling that the release requirements as defined in the proposal are a little bit too much on the side of the perfect, the enemy of the good and it would not be a major problem if some of them were not there. Unfortunately the release requirements as defined in the proposal for OTC vote come fairly naturally from the feature requirements set by OMC so if we would like to avoid some of them I think that OMC would have to first amend the feature requirements for the 3.0 release. On Wed, 2020-10-07 at 21:57 -0700, Benjamin Kaduk wrote: > On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote: > > I had an action from the OTC meeting today to raise a vote on the > > OTC > > list of technical items still to be done. Here is my proposed vote > > text. > > There will be a subsequent vote on the "beta readiness checklist" > > which > > is a separate list. > > Thanks, Matt; this is a fine list of desirable things. (I tried to > resist > commenting on specific items, but building the 1.1.1 apps+tests > against 3.0 > libraries is a solid way to help meet our stability goals, and should > arguably be something that run-checker just does, all the time.) > > I've trimmed the list, though, to make a broader point, which could > be > summarized as "the perfect is the enemy of the good". > > It's a natural consequence for a software project that has long-term > supported releases, strong API stability guarantees, and infrequent > releases, to feel that each major release is a critical threshold and > that > we need to do a bunch of tidying before the release to wrap it up > nicely. > Paying off technical debt is often a bit part of the tidying that is > perceived necessary, as is attempting to be future looking -- missing > the > release, after all, is the difference between needing to support some > bad/annoying thing for (say) 8 years instead of "only" 5. > > There is value in doing this fixup, yes, but there is also cost in > keeping > all the good things (and other fixup) that is already done out of the > hands > of those who wish to consume it. I've seen this phenomenon bite > various > projects over time with effects of different magnitude, varying from > FreeBSD 5.0+SMP that had nearly existential consequences on the > project, to > OpenAFS 1.6.0 where release delays produced enough frustration to > lead to a > bit of a rush-job final release that was a bit unstable; I was lucky > enough > to have missed the worst of this effect for krb5, and managed to do a > little better (but still not great) for OpenAFS 1.8.0. > > Projects that get hit really badly by this phenomenon tend to correct > for > it and end up on a fixed time-based cadence of releases, and in order > to > stick to that cadence end up having to get comfortable shipping > releases > without a desired feature or that are known to have incomplete parts > of one > feature or another. It also requires discipline to keep the main > development branch always (or nearly so) in a releasable state, but > in my > opinion we are already doing a pretty good job of that with our > policies > for code review and unit tests. (We could, of course, do better > about > monitoring the extended tests, run checker, and the like, but when we > do > have regressions getting them fixed is not an invasive mess.) > > To list some examples: > > FreeBSD aims for new major ("dot zero") releases every two years. > MIT krb5 puts out new releases annually. > Google Chrome puts out releases every 6 weeks. > [Okay, I haven't exactly moved OpenAFS over to this schedule yet, but > I did > just today cut a 1.9.0 and am going to try to put out regular 1.9.x > versions.] > > I would urge the OTC (and OMC) to be careful around the pitfalls of > making > the perfect the enemy of the good, and be willing to accept some > level of > "uncleanliness" in the interest of getting all the good things we > have > already done out there in production releases. (And also trying to > not > slip the published schedule too much.) It is unpleasant, yes, but > sometimes it is the best choice overall. > > Thanks, > > Ben -- Tomáš Mráz No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.]
Re: Vote proposal: Technical items still to be done
On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote: > I had an action from the OTC meeting today to raise a vote on the OTC > list of technical items still to be done. Here is my proposed vote text. > There will be a subsequent vote on the "beta readiness checklist" which > is a separate list. Thanks, Matt; this is a fine list of desirable things. (I tried to resist commenting on specific items, but building the 1.1.1 apps+tests against 3.0 libraries is a solid way to help meet our stability goals, and should arguably be something that run-checker just does, all the time.) I've trimmed the list, though, to make a broader point, which could be summarized as "the perfect is the enemy of the good". It's a natural consequence for a software project that has long-term supported releases, strong API stability guarantees, and infrequent releases, to feel that each major release is a critical threshold and that we need to do a bunch of tidying before the release to wrap it up nicely. Paying off technical debt is often a bit part of the tidying that is perceived necessary, as is attempting to be future looking -- missing the release, after all, is the difference between needing to support some bad/annoying thing for (say) 8 years instead of "only" 5. There is value in doing this fixup, yes, but there is also cost in keeping all the good things (and other fixup) that is already done out of the hands of those who wish to consume it. I've seen this phenomenon bite various projects over time with effects of different magnitude, varying from FreeBSD 5.0+SMP that had nearly existential consequences on the project, to OpenAFS 1.6.0 where release delays produced enough frustration to lead to a bit of a rush-job final release that was a bit unstable; I was lucky enough to have missed the worst of this effect for krb5, and managed to do a little better (but still not great) for OpenAFS 1.8.0. Projects that get hit really badly by this phenomenon tend to correct for it and end up on a fixed time-based cadence of releases, and in order to stick to that cadence end up having to get comfortable shipping releases without a desired feature or that are known to have incomplete parts of one feature or another. It also requires discipline to keep the main development branch always (or nearly so) in a releasable state, but in my opinion we are already doing a pretty good job of that with our policies for code review and unit tests. (We could, of course, do better about monitoring the extended tests, run checker, and the like, but when we do have regressions getting them fixed is not an invasive mess.) To list some examples: FreeBSD aims for new major ("dot zero") releases every two years. MIT krb5 puts out new releases annually. Google Chrome puts out releases every 6 weeks. [Okay, I haven't exactly moved OpenAFS over to this schedule yet, but I did just today cut a 1.9.0 and am going to try to put out regular 1.9.x versions.] I would urge the OTC (and OMC) to be careful around the pitfalls of making the perfect the enemy of the good, and be willing to accept some level of "uncleanliness" in the interest of getting all the good things we have already done out there in production releases. (And also trying to not slip the published schedule too much.) It is unpleasant, yes, but sometimes it is the best choice overall. Thanks, Ben
Re: Vote proposal: Technical items still to be done
On Wed, Oct 07, 2020 at 12:35:28PM +0100, Matt Caswell wrote: > The following items are required prerequisites for the first beta release: [...] > * Address 3.0beta1 milestones. So we now have a list here, with the last item pointing to a different list. I suggest that we only have 1 list, and that we make github issues for anything that doesn't have an issue yet, and it to the milestone. It would at least be easier to track the progress. We also talked about freezing the milestone at some point. We intend to go over the 3.0.0 milestone PRs/issues, and see if any of those need to be moved to the 3.0.0 beta1 milestone. I assume that if we have done that, that we'll freeze it. We can then try to estimate how long it will take before beta1. Kurt
Re: Vote proposal: Technical items still to be done
I support the edit proposed by Tomas. First comment that I have is that I'd prefer the first-level items to be actually numbered, as done in the drafts: we might put a disclaimer that the numbers are not indicative of priority and just meant to be used to address (equally important) tasks by index. The second comment is just a quirk of mine: I prefer plain text emails and Markdown formatting. If others shared the feeling (and nobody objected to Markdown for our records) I'd volunteer to reformat the draft of this vote text using Markdown syntax. Nicola On Wed, 7 Oct 2020 at 14:58, Tomas Mraz wrote: > > On Wed, 2020-10-07 at 12:35 +0100, Matt Caswell wrote: > > I had an action from the OTC meeting today to raise a vote on the OTC > > list of technical items still to be done. Here is my proposed vote > > text. > > There will be a subsequent vote on the "beta readiness checklist" > > which > > is a separate list. > > > > Feedback please on the proposed vote text below. > > > > The following items are required prerequisites for the first beta > > release: > > * EVP is the recommended API, it must be feature-complete compared > > with > > the functionality available using lower-level APIs. > > - Anything that isn’t available must be put to an OTC vote to > > exclude. > > - The apps are the minimum bar for this, subject to exceptions > > noted > > below. > > * Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_, > > RAND_METHOD_. > > - Does not include macros defining useful constants (e.g. > > SHA512_DIGEST_LENGTH). > > - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`. > > - There might be some others. > > - Review for exceptions. > > - The apps are the minimum bar to measure feature completeness for > > the > > EVP interface: rewrite them so they do not use internal nor > > deprecated > > functions (except speed, engine, list, passwd -crypt and the code to > > handle the -engine CLI option). That is, remove the suppression of > > deprecated define. > > - Proposal: drop passwd -crypt (OMC vote required) > > - Compile and link 1.1.1 command line app against the master > > headers > > and library. Run 1.1.1 app test cases against the chimera. Treat > > this > > as an external test using a special 1.1.1 branch. > > Deprecated functions used by libssl should be moved to independent > > file(s), to limit the suppression of deprecated defines to the > > absolute > > minimum scope. > > * Draft documentation (contents but not pretty) > > - Need a list of things we know are not present - including things > > we > > have removed. > > - We need to have mapping tables for various d2i/i2d functions. > > - We need to have a mapping table from “old names” for things into > > the > > OSSL_PARAMS names. > > - Documentation addition to old APIs to refer to new ones (man7). > > - Documentation needs to reference name mapping. > > - All the legacy interfaces need to have their documentation > > pointing to the replacement interfaces. > > * Review (and maybe clean up) legacy bridge code. > > * Review TODO(3.0) items #12224. > > * Source checksum script. > > * Review of functions previously named _with_libctx. > > * Encoder fixers (PKCS#8, PKCS#1, etc). > > * Encoder DER to PEM refactor. > > * Builds and passes tests on all primary, secondary and FIPS > > platforms. > > * Query provider parameters (name, version, …) from the command line. > > * Setup buildbot infrastructure and associated instructions. > > * Complete make fipsinstall. > > * More specific decoding selection (e.g. params or keys). > > * Example code covering replacements for deprecated APIs. > > * Drop C code output options from the apps (OMC approval required). > > * Address 3.0beta1 milestones. > > Address issues and PRs in the 3.0beta1 milestone. > > -- > Tomáš Mráz > No matter how far down the wrong road you've gone, turn back. > Turkish proverb > [You'll know whether the road is wrong if you carefully listen to your > conscience.] > >
Re: Vote proposal: Technical items still to be done
On Wed, 2020-10-07 at 12:35 +0100, Matt Caswell wrote: > I had an action from the OTC meeting today to raise a vote on the OTC > list of technical items still to be done. Here is my proposed vote > text. > There will be a subsequent vote on the "beta readiness checklist" > which > is a separate list. > > Feedback please on the proposed vote text below. > > The following items are required prerequisites for the first beta > release: > * EVP is the recommended API, it must be feature-complete compared > with > the functionality available using lower-level APIs. > - Anything that isn’t available must be put to an OTC vote to > exclude. > - The apps are the minimum bar for this, subject to exceptions > noted > below. > * Deprecation List Proposal: DH_, DSA_, ECDH_, ECDSA_, EC_KEY_, RSA_, > RAND_METHOD_. > - Does not include macros defining useful constants (e.g. > SHA512_DIGEST_LENGTH). > - Excluded from Deprecation: `EC_`, `DSA_SIG_`, `ECDSA_SIG_`. > - There might be some others. > - Review for exceptions. > - The apps are the minimum bar to measure feature completeness for > the > EVP interface: rewrite them so they do not use internal nor > deprecated > functions (except speed, engine, list, passwd -crypt and the code to > handle the -engine CLI option). That is, remove the suppression of > deprecated define. > - Proposal: drop passwd -crypt (OMC vote required) > - Compile and link 1.1.1 command line app against the master > headers > and library. Run 1.1.1 app test cases against the chimera. Treat > this > as an external test using a special 1.1.1 branch. > Deprecated functions used by libssl should be moved to independent > file(s), to limit the suppression of deprecated defines to the > absolute > minimum scope. > * Draft documentation (contents but not pretty) > - Need a list of things we know are not present - including things > we > have removed. > - We need to have mapping tables for various d2i/i2d functions. > - We need to have a mapping table from “old names” for things into > the > OSSL_PARAMS names. > - Documentation addition to old APIs to refer to new ones (man7). > - Documentation needs to reference name mapping. > - All the legacy interfaces need to have their documentation > pointing to the replacement interfaces. > * Review (and maybe clean up) legacy bridge code. > * Review TODO(3.0) items #12224. > * Source checksum script. > * Review of functions previously named _with_libctx. > * Encoder fixers (PKCS#8, PKCS#1, etc). > * Encoder DER to PEM refactor. > * Builds and passes tests on all primary, secondary and FIPS > platforms. > * Query provider parameters (name, version, …) from the command line. > * Setup buildbot infrastructure and associated instructions. > * Complete make fipsinstall. > * More specific decoding selection (e.g. params or keys). > * Example code covering replacements for deprecated APIs. > * Drop C code output options from the apps (OMC approval required). > * Address 3.0beta1 milestones. Address issues and PRs in the 3.0beta1 milestone. -- Tomáš Mráz No matter how far down the wrong road you've gone, turn back. Turkish proverb [You'll know whether the road is wrong if you carefully listen to your conscience.]