Re: Backports to 1.1.1 and what is allowed

2020-06-25 Thread Nicola Tuveri
Sorry yes, I meant to refer to the open PR with the s390 support, I picked
the wrong number!

On Thu, Jun 25, 2020, 17:54 Matt Caswell  wrote:

>
>
> On 25/06/2020 15:33, Nicola Tuveri wrote:
> > In light of how the discussion evolved I would say that not only there
> > is consensus on supporting the definition of a detailed policy on
> > backports and the definitions of what are the requirements for regular
> > releases vs LTS releases (other than the longer support timeframe),
> > but also highlights a need to do it sooner rather than later!
> >
> > This seems a job for the OMC, as it falls under:
> >
> >> makes all decisions regarding management and strategic direction of the
> project; including:
> >> - business requirements;
> >> - feature requirements;
> >> - platform requirements;
> >> - roadmap requirements and priority;
> >> - end-of-life decisions;
> >> - release timing and requirement decisions;
> >
> > My contribution to the discussion is to ask if the OMC has plans for
> > addressing this in the very short term.
>
> I think its unlikely we are going to get to a policy in the short term.
> It seems to me we are still some way away from a consensus here.
>
> > If working on a policy is going to be a medium-term effort, maybe it
> > would be opportune to call an OTC vote specific to #11968 under the
> > current release requirements defined by the OMC (or lack thereof).
>
> 11968 is already merged and, AFAIK, no one has proposed reverting it. If
> such a PR was raised then a vote might be a way forward for it.
>
> 11188 is the more pressing problem because that is currently unmerged
> and stuck. That has an OTC hold on it (placed there by me), so an OTC
> vote seems appropriate. If a vote is held it should be to decide whether
> backporting it is consistent with our current understanding of the
> policy such as it is. It is for the OMC to decide whether a different
> policy should be introduced at some point in the future.
>
> Matt
>
>
> >
> > We already saw a few comments in favor of evaluating backporting
> > #11968 as an exception, in light of the supporting arguments, even if
> > it was in conflict with the current policy understanding or the
> > upcoming policy formulation.
> > So if we could swiftly agree on this being an OTC or OMC vote, I would
> > urge to have a dedicated discussion/vote specific to #11968, while
> > more detailed policies and definitions are in the process of being
> > formulated.
> >
> > - What is the consensus on splitting the 2 discussions?
> > - If splitting the discussions, is deciding on #11968 an OTC or OMC
> matter?
> >
> >
> >
> > Cheers,
> >
> > Nicola
> >
>


Re: Backports to 1.1.1 and what is allowed

2020-06-25 Thread Matt Caswell



On 25/06/2020 15:33, Nicola Tuveri wrote:
> In light of how the discussion evolved I would say that not only there
> is consensus on supporting the definition of a detailed policy on
> backports and the definitions of what are the requirements for regular
> releases vs LTS releases (other than the longer support timeframe),
> but also highlights a need to do it sooner rather than later!
> 
> This seems a job for the OMC, as it falls under:
> 
>> makes all decisions regarding management and strategic direction of the 
>> project; including:
>> - business requirements;
>> - feature requirements;
>> - platform requirements;
>> - roadmap requirements and priority;
>> - end-of-life decisions;
>> - release timing and requirement decisions;
> 
> My contribution to the discussion is to ask if the OMC has plans for
> addressing this in the very short term.

I think its unlikely we are going to get to a policy in the short term.
It seems to me we are still some way away from a consensus here.

> If working on a policy is going to be a medium-term effort, maybe it
> would be opportune to call an OTC vote specific to #11968 under the
> current release requirements defined by the OMC (or lack thereof).

11968 is already merged and, AFAIK, no one has proposed reverting it. If
such a PR was raised then a vote might be a way forward for it.

11188 is the more pressing problem because that is currently unmerged
and stuck. That has an OTC hold on it (placed there by me), so an OTC
vote seems appropriate. If a vote is held it should be to decide whether
backporting it is consistent with our current understanding of the
policy such as it is. It is for the OMC to decide whether a different
policy should be introduced at some point in the future.

Matt


> 
> We already saw a few comments in favor of evaluating backporting
> #11968 as an exception, in light of the supporting arguments, even if
> it was in conflict with the current policy understanding or the
> upcoming policy formulation.
> So if we could swiftly agree on this being an OTC or OMC vote, I would
> urge to have a dedicated discussion/vote specific to #11968, while
> more detailed policies and definitions are in the process of being
> formulated.
> 
> - What is the consensus on splitting the 2 discussions?
> - If splitting the discussions, is deciding on #11968 an OTC or OMC matter?
> 
> 
> 
> Cheers,
> 
> Nicola
> 


Re: Backports to 1.1.1 and what is allowed

2020-06-25 Thread Tomas Mraz
On Thu, 2020-06-25 at 14:49 +, Dr. Matthias St. Pierre wrote:
> Since already a few backporting requests to 1.1.1 have accumulated
> which are controversial
> or not allowed for an LTS release, would it make sense to consider
> creating a new 1.1.2 STS
> branch which could then receive such changes?

In my opinion that would open a can of worms. Such branch would
inevitably mean there will be request for the 1.1.2 release and I do
not think it would be a good idea to have it. It would seriously
stretch the team resources IMHO.

It would be much better to have the rules not tightened for 1.1.1 right
now (i.e. still allow for example the performance improvements and
support for newer hardware) and tighten them only after the 3.0 release
is final.

Still, I do not think allowing any truly new features other than the
performance and HW support on 1.1.1 is worth it.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]




RE: Backports to 1.1.1 and what is allowed

2020-06-25 Thread Dr. Matthias St. Pierre
Since already a few backporting requests to 1.1.1 have accumulated which are 
controversial
or not allowed for an LTS release, would it make sense to consider creating a 
new 1.1.2 STS
branch which could then receive such changes?

Matthias



Re: Backports to 1.1.1 and what is allowed

2020-06-25 Thread Nicola Tuveri
In light of how the discussion evolved I would say that not only there
is consensus on supporting the definition of a detailed policy on
backports and the definitions of what are the requirements for regular
releases vs LTS releases (other than the longer support timeframe),
but also highlights a need to do it sooner rather than later!

This seems a job for the OMC, as it falls under:

> makes all decisions regarding management and strategic direction of the 
> project; including:
> - business requirements;
> - feature requirements;
> - platform requirements;
> - roadmap requirements and priority;
> - end-of-life decisions;
> - release timing and requirement decisions;

My contribution to the discussion is to ask if the OMC has plans for
addressing this in the very short term.
If working on a policy is going to be a medium-term effort, maybe it
would be opportune to call an OTC vote specific to #11968 under the
current release requirements defined by the OMC (or lack thereof).

We already saw a few comments in favor of evaluating backporting
#11968 as an exception, in light of the supporting arguments, even if
it was in conflict with the current policy understanding or the
upcoming policy formulation.
So if we could swiftly agree on this being an OTC or OMC vote, I would
urge to have a dedicated discussion/vote specific to #11968, while
more detailed policies and definitions are in the process of being
formulated.

- What is the consensus on splitting the 2 discussions?
- If splitting the discussions, is deciding on #11968 an OTC or OMC matter?



Cheers,

Nicola


Re: Backports to 1.1.1 and what is allowed

2020-06-22 Thread Patrick Steuer

From the view of an application dev ..

1 the reasons to chose OpenSSL over other crypto libraries are its
  rich feature set and wide portability paired with good performance
  across a wide range of platforms plus having a customizable
  backend (engines/providers).

2 the reasons to not move away from OpenSSL trusting in LTS releases
  actually being stable i.e., not causing any disruptions.

(For both 1 and 2, a good security record is a matter of course.)

IMO the current release policy of infrequent "big bang" releases
serves 2 well, but largely neglects 1 (think e.g., about
time-to-market for new processor features).

That doesnt mean that we should take all features into LTS
but i think performance improvements is an area where you
should think about it, at least in individual cases.

Patrick


Re: Backports to 1.1.1 and what is allowed

2020-06-22 Thread Salz, Rich
  *   I suggest everyone takes a read through  
https://en.wikipedia.org/wiki/Long-term_support
 as to what LTS is actually meant to be focused on.

For what it’s worth, I think Tim’s comments are spot-on.  An LTS release should 
not change unless there are serious bugs or new security flaws.

It is tempting to add new hardware support, nor improved performance.  “Hey, 
everyone will get this in their long-term offering.”  But you’re thinking the 
wrong way.  Imagine that each new patch to an LTS release requires massive 
effort, such as changing the firmware on a plane. Does the change justify that? 
 Almost always, the answer is no.




Re: Backports to 1.1.1 and what is allowed

2020-06-22 Thread Matt Caswell



On 20/06/2020 01:11, Tim Hudson wrote:
> I suggest everyone takes a read through 
> https://en.wikipedia.org/wiki/Long-term_support as to what LTS is
> actually meant to be focused on.
> 
> What you (Ben and Matt) are both describing is not LTS but STS ... these
> are different concepts.
> 
> For LTS the focus is *stability *and *reduced risk of disruption *which
> alters the judgement on what is appropriate to put into a release.
> It then becomes a test of "is this bug fix worth the risk" with the
> general focus on lowest possible risk which stops this sort of thing
> happening unless a critical feature is broken.
> 
> All of the "edge case" examples presented all involve substantial
> changes to implementations and carry an inherent risk of breaking
> something else - and that is the issue.
> It would be different if we had a full regression test suite across all
> the platforms and variations on platforms that our users are operating
> on - but we don't.
> 
> We don't compare performance between releases or for any updates in our
> test suite so it isn't part of our scope for release readiness - if this
> is such a critical thing that must be fixed then it is something that we
> should be measuring and checking as a release condition - but we don't -
> because it actually isn't that critical. 
> 

I read the page and I don't think it is contrary to anything that I
said. From the characteristics section:

"At the beginning of a long-term support period, the software developers
impose a feature freeze: They make patches to correct software bugs and
vulnerabilities, but do not introduce new features that may cause
regression. The software maintainer either distributes patches
individually, or packages them in maintenance releases, point releases,
or service packs. At the conclusion of the support period, the product
either reaches end-of-life, or receives a reduced level of support for a
period of time (e.g., high-priority security patches only).[2]"

AFAIK this is exactly what we do. We fix bugs and don't add new
features. After 4 years we transition to security fixes only.

Later, in the Rational section, it says this:

"Long-term support addresses this by assuring users and administrators
that the software will be maintained for a specific period of time, and
that updates selected for publication will carry a significantly reduced
risk of regression.[2] The maintainers of LTS software only publish
updates that either have low IT risk or that reduce IT risk (such as
security patches). Patches for LTS software are published with the
understanding that installing them is less risky than not installing them."

This, IMO, is exactly what we do.

In none of this does it say that we shouldn't fix less serious bugs or
performance regressions. It talks about risk - but that does not
preclude those kinds of fixes. In fact as I said in my answer to Kurt
earlier in this thread I do believe that risk is an important factor in
deciding what things get backported. However I don't believe that,
generally speaking, most minor bug fixes represent a significant source
of risk. I also think the same *might* be true of *some* types of
performance regressions.

Matt


Re: Backports to 1.1.1 and what is allowed

2020-06-22 Thread Tomas Mraz
On Sun, 2020-06-21 at 14:34 -0700, Benjamin Kaduk wrote:
> Hi Tim,
> 
> On Sat, Jun 20, 2020 at 10:11:04AM +1000, Tim Hudson wrote:
> > I suggest everyone takes a read through
> > https://en.wikipedia.org/wiki/Long-term_support as to what LTS is
> > actually
> > meant to be focused on.
> 
> You may have hit on a key aspect of how we are disconnected, here.
> 
> I was under the impression that (as is the case for many OSS
> projects), the
> fact that 1.1.1 is an LTS release means that we enter a separate "LTS
> mode"
> at the "beginning of a long-term support period" (as Wikipedia puts
> it) but
> that there is some period prior to the start of the long-term support
> period for which the STS policies apply.
> 
> So, are you considering that 1.1.1 is now, and has always been, in
> LTS mode
> because it is marked as an "LTS release"?  Or is there a separate STS
> period before it transitions to "LTS mode"?

In my opinion the 1.1.1 should not enter such strongly restricted phase
earlier than after the 3.0 is released.

Ideally, if we had releases based on some predictable cadence and not
rather feature based ones, we should have 3 active branches - master
for development of new features, one branch with bug fixes and even
non-invasive performance fixes, and one branch for LTS where only
security fixes with above than low severity and severe bug fixes would
be applied.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]




Re: Backports to 1.1.1 and what is allowed

2020-06-21 Thread Benjamin Kaduk
Hi Tim,

On Sat, Jun 20, 2020 at 10:11:04AM +1000, Tim Hudson wrote:
> I suggest everyone takes a read through
> https://en.wikipedia.org/wiki/Long-term_support as to what LTS is actually
> meant to be focused on.

You may have hit on a key aspect of how we are disconnected, here.

I was under the impression that (as is the case for many OSS projects), the
fact that 1.1.1 is an LTS release means that we enter a separate "LTS mode"
at the "beginning of a long-term support period" (as Wikipedia puts it) but
that there is some period prior to the start of the long-term support
period for which the STS policies apply.

So, are you considering that 1.1.1 is now, and has always been, in LTS mode
because it is marked as an "LTS release"?  Or is there a separate STS
period before it transitions to "LTS mode"?

> What you (Ben and Matt) are both describing is not LTS but STS ... these
> are different concepts.

AFAICT the thread started with just talk of merging to stable release
branches; LTS didn't come up until Kurt's note.  I certainly was not
considering the LTS policy in my earlier comments.

> For LTS the focus is *stability *and *reduced risk of disruption *which
> alters the judgement on what is appropriate to put into a release.
> It then becomes a test of "is this bug fix worth the risk" with the general
> focus on lowest possible risk which stops this sort of thing happening
> unless a critical feature is broken.
> 
> All of the "edge case" examples presented all involve substantial changes
> to implementations and carry an inherent risk of breaking something else -
> and that is the issue.

IMO, if we're in LTS mode, if any of the proposed edge cases came up that
would indicate that we failed at the LTS policy in the first place.

> It would be different if we had a full regression test suite across all the
> platforms and variations on platforms that our users are operating on - but
> we don't.
> 
> We don't compare performance between releases or for any updates in our
> test suite so it isn't part of our scope for release readiness - if this is
> such a critical thing that must be fixed then it is something that we
> should be measuring and checking as a release condition - but we don't -
> because it actually isn't that critical.

It can translate to real monetary impact in some cases (e.g., at scale).
But I can't directly refute your point, it is true.

-Ben


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Tim Hudson
I suggest everyone takes a read through
https://en.wikipedia.org/wiki/Long-term_support as to what LTS is actually
meant to be focused on.

What you (Ben and Matt) are both describing is not LTS but STS ... these
are different concepts.

For LTS the focus is *stability *and *reduced risk of disruption *which
alters the judgement on what is appropriate to put into a release.
It then becomes a test of "is this bug fix worth the risk" with the general
focus on lowest possible risk which stops this sort of thing happening
unless a critical feature is broken.

All of the "edge case" examples presented all involve substantial changes
to implementations and carry an inherent risk of breaking something else -
and that is the issue.
It would be different if we had a full regression test suite across all the
platforms and variations on platforms that our users are operating on - but
we don't.

We don't compare performance between releases or for any updates in our
test suite so it isn't part of our scope for release readiness - if this is
such a critical thing that must be fixed then it is something that we
should be measuring and checking as a release condition - but we don't -
because it actually isn't that critical.

Tim.


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Benjamin Kaduk
On Fri, Jun 19, 2020 at 11:46:15PM +0100, Matt Caswell wrote:
> 
> 
> On 19/06/2020 23:34, Tim Hudson wrote:
> > 
> > 
> > On Sat, 20 Jun 2020, 8:14 am Benjamin Kaduk,  > > wrote:
> > 
> > On Sat, Jun 20, 2020 at 08:11:16AM +1000, Tim Hudson wrote:
> > > The general concept is to only fix serious bugs in stable releases.
> > > Increasing performance is not fixing a bug - it is a feature.
> > 
> > Is remediating a significant performance regression fixing a bug?
> > 
> > 
> > It would be a bug - but not a serious bug. So no.
> > It works. It was released. 
> 
> I don't recall us ever saying that we would only fix serious bugs.
> AFAIK, if its a bug then we will fix it. IMO a serious performance
> regression would qualify, within reason (e.g. major rewrites of the
> assembler would not be ok).
> 
> > Wholesale replacement of implementations of algorithms should not be
> > happening in LTS releases.
> 
> This I agree with.

Both of Matt's paragraphs match up with my understanding/sentiment.

-Ben


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Matt Caswell



On 19/06/2020 22:58, Kurt Roeckx wrote:
> On Fri, Jun 19, 2020 at 10:29:24PM +0100, Matt Caswell wrote:
>>
>> My immediate reaction to that is no - it shouldn't go to 1.1.1. That
>> would impact a very high proportion of our user base.
> 
> So is risk an important factor? How do you judge the risk? By the
> size of the change?

Yes, I do believe risk is an important factor although putting hard
rules around it is very difficult. There are bound to be some fuzzy
lines here between what is and isn't acceptable.

Matt


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Matt Caswell



On 19/06/2020 23:34, Tim Hudson wrote:
> 
> 
> On Sat, 20 Jun 2020, 8:14 am Benjamin Kaduk,  > wrote:
> 
> On Sat, Jun 20, 2020 at 08:11:16AM +1000, Tim Hudson wrote:
> > The general concept is to only fix serious bugs in stable releases.
> > Increasing performance is not fixing a bug - it is a feature.
> 
> Is remediating a significant performance regression fixing a bug?
> 
> 
> It would be a bug - but not a serious bug. So no.
> It works. It was released. 

I don't recall us ever saying that we would only fix serious bugs.
AFAIK, if its a bug then we will fix it. IMO a serious performance
regression would qualify, within reason (e.g. major rewrites of the
assembler would not be ok).

> Wholesale replacement of implementations of algorithms should not be
> happening in LTS releases.

This I agree with.

Matt


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Tim Hudson
On Sat, 20 Jun 2020, 8:14 am Benjamin Kaduk,  wrote:

> On Sat, Jun 20, 2020 at 08:11:16AM +1000, Tim Hudson wrote:
> > The general concept is to only fix serious bugs in stable releases.
> > Increasing performance is not fixing a bug - it is a feature.
>
> Is remediating a significant performance regression fixing a bug?
>

It would be a bug - but not a serious bug. So no.
It works. It was released.
Wholesale replacement of implementations of algorithms should not be
happening in LTS releases.

We make no performance guarantees or statements in our releases (in
general).

And performance isn't an issue for the vast majority of our users.

Those for whom performance is critical also tend to be building their own
releases in my experience.

Tim.


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Benjamin Kaduk
On Sat, Jun 20, 2020 at 08:11:16AM +1000, Tim Hudson wrote:
> The general concept is to only fix serious bugs in stable releases.
> Increasing performance is not fixing a bug - it is a feature.

Is remediating a significant performance regression fixing a bug?

-Ben


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Tim Hudson
The general concept is to only fix serious bugs in stable releases.
Increasing performance is not fixing a bug - it is a feature.

Swapping out one implementation of algorithm for another is a significant
change and isn't something that should go into an LTS in my view.

It would be less of an issue for users if our release cadence was more
frequent - but the principal applies - stability is what an LTS is aimed
at. We should be fixing significant bugs only.

Arguing that slower performance is a bug is specious - it works - and
performance is not something that we document and guarantee. You don't find
our release notes stating performance=X for a release - because such a
statement makes little sense for the vast majority of users.

Tim.


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Kurt Roeckx
On Fri, Jun 19, 2020 at 10:29:24PM +0100, Matt Caswell wrote:
> 
> My immediate reaction to that is no - it shouldn't go to 1.1.1. That
> would impact a very high proportion of our user base.

So is risk an important factor? How do you judge the risk? By the
size of the change?


Kurt



Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Matt Caswell



On 19/06/2020 21:42, Kurt Roeckx wrote:
> I think one other thing that has come up is adding support for a
> new target, which can just be some small change to configuration
> files. Is that something we want to accept?

I think previously we have said that a new target is a new feature and
therefore haven't allowed it. But even this I think is a grey area. If a
target could be added simply by adding some lines to Configure, probably
the risk to our existing users is very low. OTOH, if adding a platform
means adding lots of ifdef hackery throughout the codebase then the risk
of something going wrong is significantly higher.

> So we currently also have PR #12201 that adds a new constant time
> P-384 implementation. Do you think we should backport that to the
> 1.1.1 branch? If the answer is different than for the assembler,
> why?

My immediate reaction to that is no - it shouldn't go to 1.1.1. That
would impact a very high proportion of our user base.

Matt


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Kurt Roeckx
I think one other thing that has come up is adding support for a
new target, which can just be some small change to configuration
files. Is that something we want to accept?


Kurt


Re: Backports to 1.1.1 and what is allowed

2020-06-19 Thread Kurt Roeckx
So we currently also have PR #12201 that adds a new constant time
P-384 implementation. Do you think we should backport that to the
1.1.1 branch? If the answer is different than for the assembler,
why?

Does 1.1.1 being an LTS release have any effect on the answer?
For instance, if we add some assembler during the 3.1 development,
and 3.0 is not an LTS release, but 1.1.1 is, and is still supported,
do we backport it to 1.1.1 and not 3.0?

We've at least talked about not adding a feature to the stable
branch. At least one way of looking at that is that we don't add new
functionality. You could argue that new assembler is not a new
feature, just a new implementation of an existing feature.


Kurt



Re: Backports to 1.1.1 and what is allowed

2020-06-16 Thread Dr Paul Dale
I’d also support 11968 being merged.  I find the recent discussion and the 
likelihood of all major distros (supporting the S390x) picking up the patches 
fairly compelling, especially since the same people will be maintaining it.

I would need to go and double check the non-S390x specific parts for possible 
breakage but the bulk of the changes are S390x specific.


I support formalising the rules better than we have at the moment.  Even if 
this is in conflict with the above.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




> On 17 Jun 2020, at 1:43 pm, Benjamin Kaduk  wrote:
> 
> On Tue, Jun 16, 2020 at 02:03:58PM +0100, Matt Caswell wrote:
>> PR 11188 proposes to backport a series of s390x patches to the 1.1.1
>> branch. IIUC it includes performance improvements as well as support for
>> new hardware instructions.
>> 
>> I think we need to have a much clearer and more explicit policy about
>> exactly what is allowed to be backported to a stable branch. For example
>> PR #11968 was a significant recent PR that backported AES CTR-DRBG
>> performance improvements to the 1.1.1 branch and was allowed (should it
>> have been?).
> 
> I am happy to see 11968 landed; some informal testing that I hope to make
> more formal soon seems to show a ~20% slowdown/regression for large RNG
> requests going from 1.1.1d to 1.1.g, and 11968 provided substantially more
> significant of a speedup (again, very informal testing, though).
> 
> In this case you could say that the "bug" is that we're only calling AES on
> a block at a time despite having much more effective backends available for
> multi-block calls.
> 
>> We have always said that the stable releases should only receive bug and
>> security fixes. Performance improvements have been a bit of a grey area,
>> e.g. a few lines of code that significantly improves the performance of
>> a particular algorithm or codepath could reasonably be justified as
>> "fixing a performance bug". OTOH bringing in whole new files of
>> assembler seems to go way beyond that definition.
> 
> There's probably some semi-subtle distinction to make along the lines of
> "making the algorithm more efficient" vs "using a new algorithm, that
> happens to run faster", with the latter counting as a feature.
> 
>> However, when I tried to find a reference to the policy which says
>> exactly what we are allowed to backport I could find one. Do we have
>> such a thing?
>> 
>> In any case I think we should develop a much more explicit policy that
>> will enable us to decide whether PRs such as 11188 are reasonable or
>> not. See for example Ubuntu's Stable Release Updates policy:
> 
> I agree that having something written down will be very useful, even if
> just as a fixed benchmark against which to think about how exceptional any
> given "exception case" is where we want to deviate from it.
> 
> -Ben
> 
>> https://wiki.ubuntu.com/StableReleaseUpdates
>> 
>> 
>> Matt



Re: Backports to 1.1.1 and what is allowed

2020-06-16 Thread Benjamin Kaduk
On Tue, Jun 16, 2020 at 02:03:58PM +0100, Matt Caswell wrote:
> PR 11188 proposes to backport a series of s390x patches to the 1.1.1
> branch. IIUC it includes performance improvements as well as support for
> new hardware instructions.
> 
> I think we need to have a much clearer and more explicit policy about
> exactly what is allowed to be backported to a stable branch. For example
> PR #11968 was a significant recent PR that backported AES CTR-DRBG
> performance improvements to the 1.1.1 branch and was allowed (should it
> have been?).

I am happy to see 11968 landed; some informal testing that I hope to make
more formal soon seems to show a ~20% slowdown/regression for large RNG
requests going from 1.1.1d to 1.1.g, and 11968 provided substantially more
significant of a speedup (again, very informal testing, though).

In this case you could say that the "bug" is that we're only calling AES on
a block at a time despite having much more effective backends available for
multi-block calls.

> We have always said that the stable releases should only receive bug and
> security fixes. Performance improvements have been a bit of a grey area,
> e.g. a few lines of code that significantly improves the performance of
> a particular algorithm or codepath could reasonably be justified as
> "fixing a performance bug". OTOH bringing in whole new files of
> assembler seems to go way beyond that definition.

There's probably some semi-subtle distinction to make along the lines of
"making the algorithm more efficient" vs "using a new algorithm, that
happens to run faster", with the latter counting as a feature.

> However, when I tried to find a reference to the policy which says
> exactly what we are allowed to backport I could find one. Do we have
> such a thing?
> 
> In any case I think we should develop a much more explicit policy that
> will enable us to decide whether PRs such as 11188 are reasonable or
> not. See for example Ubuntu's Stable Release Updates policy:

I agree that having something written down will be very useful, even if
just as a fixed benchmark against which to think about how exceptional any
given "exception case" is where we want to deviate from it.

-Ben

> https://wiki.ubuntu.com/StableReleaseUpdates
> 
> 
> Matt