I don’t have a big stake in, however, one opinion is if a large commercial enterprise was using a specific release that is working the desire is often to only upgrade if necessary. Necessary can be for a number of reasons including new features; however if a new feature is not needed the compelling reason to upgrade is to fix a specific problem that is causing issues. Thus keeping a maintenance release stable is very important and reducing the chance for, while fixing one problem, introducing another.
Often a clear classification of severity of the problem would dictate the need to make a change. (yes these can be subjective, but some guidance would be better than nothing). It might be good to give committers guidance on back porting things that have a high impact on improving a problem. Fixing a crashing bug, fixing a degenerative performance issue, etc, where these issues have no easy/viable work around. Nice to have fixes aren’t, always, worth updating to. There can be an argument to respond with a “then don’t upgrade” but if changing the release with “nice to have’s” and several point releases later when a critical bug is fixed then the org if forced to accept the risk of the nice to have’s. just an opinion. …larry > On Jul 17, 2018, at 3:00 PM, Chun-Hung Hsiao <chhs...@mesosphere.io> wrote: > > I just have a comment on a special case: > If a fix for a flaky test is easy to backport, > IMO we probably should backport it, > otherwise if someone backports another critical fix in the future, > it would take them extra effort to check all CI failures. > > On Mon, Jul 16, 2018 at 11:39 AM Vinod Kone <vinodk...@apache.org > <mailto:vinodk...@apache.org>> wrote: > I like how you summarized it Greg and I would vote for leaving the decision > to the committer too. In addition to what others mentioned, I think > committer should've the responsibility because if things break in a point > release (after it is released), it is the committer and contributor who are > on the hook to triage and fix it and not the release manager. > > Having said that, if "during" the release process (i.e., cutting an RC) > these backports cause delays for a release manager in getting the release > out (e.g., CI flakiness introduced due to backports), release manager could > be the ultimate arbiter on whether such a backport should be reverted or > fixed by the committer/contributor. Hopefully such issues are caught much > before a release process is started (e.g., CI running against release > branches). > > On Mon, Jul 16, 2018 at 1:28 PM Jie Yu <yujie....@gmail.com > <mailto:yujie....@gmail.com>> wrote: > > > Greg, I like your idea of adding a prescriptive "policy" when evaluating > > whether a bug fix should be backported, and leave the decision to committer > > (because they have the most context, and avoid a bottleneck in the > > process). > > > > - Jie > > > > On Mon, Jul 16, 2018 at 11:24 AM, Greg Mann <g...@mesosphere.io > > <mailto:g...@mesosphere.io>> wrote: > > > > > My impression is that we have two opposing schools of thought here: > > > > > > 1. Backport as little as possible, to avoid unforeseen consequences > > > 2. Backport as much as proves practical, to eliminate bugs in > > > supported versions > > > > > > Do other people agree with this assessment? > > > > > > If so, how can we find common ground? One possible solution would be to > > > leave the decision on backporting up to the committer, without > > specifying a > > > project-wide policy. This seems to be the status quo, and would lead to > > > some variation across committers regarding what types of fixes are > > > backported. We could also choose to delegate the decision to the release > > > manager; I favor leaving the decision with the committer, to eliminate > > the > > > burden on release managers. > > > > > > Here's a thought: rather than defining a prescriptive "policy" that we > > > expect committers to abide by, we could enumerate in the documentation > > the > > > competing concerns that we expect committers to consider when making > > > decisions on backports. The committing docs could read something like: > > > > > > "When bug fixes are committed to master, the committer should evaluate > > the > > > fix to determine whether or not it should be backported to supported > > > versions. This is left to the committer, but they are expected to weigh > > the > > > following concerns when making the decision: > > > > > > - Every backported change comes with a risk of unintended > > > consequences. The change should be carefully evaluated to ensure that > > such > > > side-effects are highly unlikely. > > > - As the complexity of applying a backport increases due to merge > > > conflicts, the likelihood of unintended consequences also increases. > > Bug > > > fixes which require extensive rebasing should only be backported when > > the > > > bug is critical enough to warrant the risk. > > > - Users of supported versions benefit greatly from the resolution of > > > bugs in point releases. Thus, whenever concerns #1 and #2 can be > > allayed > > > for a given bug fix, it should be backported." > > > > > > > > > Cheers, > > > Greg > > > > > > > > > On Mon, Jul 16, 2018 at 3:06 AM, Alex Rukletsov <a...@mesosphere.com > > > <mailto:a...@mesosphere.com>> > > > wrote: > > > > > >> Back porting as little as possible is the ultimate goal for me. My > > >> reasons are closely aligned with what Andrew wrote above. > > >> > > >> If we agree on this strategy, the next question is how to enforce it. My > > >> intuition is that committers will lean towards back porting their > > patches > > >> in arguable cases, because humans tend to overestimate the importance of > > >> their personal work. Delegating the decision in such cases to a release > > >> manager in my opinion will help us enforce the strategy of minimal > > number > > >> backports. As a bonus, the release manager will have a much better > > >> understanding of what's going on with the release, keyword: "more > > >> ownership". > > >> > > >> On Sat, Jul 14, 2018 at 12:07 AM, Andrew Schwartzmeyer < > > >> and...@schwartzmeyer.com <mailto:and...@schwartzmeyer.com>> wrote: > > >> > > >>> I believe I fall somewhere between Alex and Ben. > > >>> > > >>> As for deciding what to backport or not, I lean toward Alex's view of > > >>> backporting as little as possible (and agree with his criteria). My > > >>> reasoning is that all changes can have unforeseen consequences, which I > > >>> believe is something to be actively avoided in already released > > versions. > > >>> The reason for backporting patches to fix regressions is the same as > > the > > >>> reason to avoid backporting as much as possible: keep behavior > > consistent > > >>> (and safe) within a release. With that as the goal of a branch in > > >>> maintenance mode, it makes sense to fix regressions, and make > > exceptions to > > >>> fix CVEs and other critical/blocking issues. > > >>> > > >>> As for who should decide what to backport, I lean toward Ben's view of > > >>> the burden being on the committer. I don't think we should add more > > work > > >>> for release managers, and I think the committer/shepherd obviously has > > the > > >>> most understanding of the context around changes proposed for backport. > > >>> > > >>> Here's an example of a recent bugfix which I backported: > > >>> https://reviews.apache.org/r/67587/ > > >>> <https://reviews.apache.org/r/67587/> (for MESOS-3790) > > >>> > > >>> While normally I believe this change falls under "avoid due to > > >>> unforeseen consequences," I made an exception as the bug was old, circa > > >>> 2015, (indicating it had been an issue for others), and was causing > > >>> recurring failures in testing. The fix itself was very small, meaning > > it > > >>> was easier to evaluate for possible side effects, so I felt a little > > safer > > >>> in that regard. The effect of not having the fix was a fatal and > > undesired > > >>> crash, which furthermore left troublesome side effects on the system > > (you > > >>> couldn't bring the agent back up). And lastly, a dependent project > > (DC/OS) > > >>> wanted it in their next bump, which necessitated backporting to the > > release > > >>> they were pulling in. > > >>> > > >>> I think in general we should backport only as necessary, and leave it > > on > > >>> the committers to decide if backporting a particular change is > > necessary. > > >>> > > >>> > > >>> On 07/13/2018 12:54 am, Alex Rukletsov wrote: > > >>> > > >>>> This is exactly where our views differ, Ben : ) > > >>>> > > >>>> Ideally, I would like a release manager to have more ownership and > > less > > >>>> manual work. In my imagination, a release manager has more power and > > >>>> control about dates, features, backports and everything that is > > related > > >>>> to > > >>>> "their" branch. I would also like us to back port as little as > > >>>> possible, to > > >>>> simplify testing and releasing patch versions. > > >>>> > > >>>> On Fri, Jul 13, 2018 at 1:17 AM, Benjamin Mahler <bmah...@apache.org > > >>>> <mailto:bmah...@apache.org>> > > >>>> wrote: > > >>>> > > >>>> +user, I probably it would be good to hear from users as well. > > >>>>> > > >>>>> Please see the original proposal as well as Alex's proposal and let > > us > > >>>>> know > > >>>>> your thoughts. > > >>>>> > > >>>>> To continue the discussion from where Alex left off: > > >>>>> > > >>>>> > Other bugs and significant improvements, e.g., performance, may be > > >>>>> back > > >>>>> ported, > > >>>>> the release manager should ideally be the one who decides on this. > > >>>>> > > >>>>> I'm a little puzzled by this, why is the release manager involved? As > > >>>>> we > > >>>>> already document, backports occur when the bug is fixed, so this > > >>>>> happens in > > >>>>> the steady state of development, not at release time. The release > > >>>>> manager > > >>>>> only comes in at the time of the release itself, at which point all > > >>>>> backports have already happened and the release manager handles the > > >>>>> release > > >>>>> process. Only blocker level issues can stop the release and while the > > >>>>> release manager has a strong say, we should generally agree on what > > >>>>> consists of a release blocking issue. > > >>>>> > > >>>>> Just to clarify my workflow, I generally backport every bug fix I > > >>>>> commit > > >>>>> that applies cleanly, right after I commit it to master (with the > > >>>>> exceptions I listed below). > > >>>>> > > >>>>> On Thu, Jul 12, 2018 at 8:39 AM, Alex Rukletsov <a...@mesosphere.com > > >>>>> <mailto:a...@mesosphere.com> > > > > > >>>>> wrote: > > >>>>> > > >>>>> > I would like to back port as little as possible. I suggest the > > >>>>> following > > >>>>> > criteria: > > >>>>> > > > >>>>> > * By default, regressions are back ported to existing release > > >>>>> branches. A > > >>>>> > bug is considered a regression if the functionality is present in > > the > > >>>>> > previous minor or patch version and is not affected by the bug > > there. > > >>>>> > > > >>>>> > * Critical and blocker issues, e.g., a CVE, can be back ported. > > >>>>> > > > >>>>> > * Other bugs and significant improvements, e.g., performance, may > > be > > >>>>> back > > >>>>> > ported, the release manager should ideally be the one who decides > > on > > >>>>> this. > > >>>>> > > > >>>>> > On Thu, Jul 12, 2018 at 12:25 AM, Vinod Kone <vinodk...@apache.org > > >>>>> > <mailto:vinodk...@apache.org> > > > > > >>>>> wrote: > > >>>>> > > > >>>>> > > Ben, thanks for the clarification. I'm in agreement with the > > >>>>> points you > > >>>>> > > made. > > >>>>> > > > > >>>>> > > Once we have consensus, would you mind updating the doc? > > >>>>> > > > > >>>>> > > On Wed, Jul 11, 2018 at 5:15 PM Benjamin Mahler < > > >>>>> bmah...@apache.org <mailto:bmah...@apache.org>> > > >>>>> > > wrote: > > >>>>> > > > > >>>>> > > > I realized recently that we aren't all on the same page with > > >>>>> > backporting. > > >>>>> > > > We currently only document the following: > > >>>>> > > > > > >>>>> > > > "Typically the fix for an issue that is affecting supported > > >>>>> releases > > >>>>> > > lands > > >>>>> > > > on the master branch and is then backported to the release > > >>>>> branch(es). > > >>>>> > In > > >>>>> > > > rare cases, the fix might directly go into a release branch > > >>>>> without > > >>>>> > > landing > > >>>>> > > > on master (e.g., fix / issue is not applicable to master)." [1] > > >>>>> > > > > > >>>>> > > > This leaves room for interpretation about what lies outside of > > >>>>> > "typical". > > >>>>> > > > Here's the simplest way I can explain what I stick to, and I'd > > >>>>> like > > >>>>> to > > >>>>> > > hear > > >>>>> > > > what others have in mind: > > >>>>> > > > > > >>>>> > > > * By default, bug fixes at any level should be backported to > > >>>>> existing > > >>>>> > > > release branches if it affects those releases. Especially > > >>>>> important: > > >>>>> > > > crashes, bugs in non-experimental features. > > >>>>> > > > > > >>>>> > > > * Exceptional cases that can omit backporting: difficult to > > >>>>> backport > > >>>>> > > fixes > > >>>>> > > > (especially if the bugs are deemed of low priority), bugs in > > >>>>> > experimental > > >>>>> > > > features. > > >>>>> > > > > > >>>>> > > > * Exceptional non-bug cases that can be backported: performance > > >>>>> > > > improvements. > > >>>>> > > > > > >>>>> > > > I realize that there is a ton of subtlety here (even in terms > > of > > >>>>> which > > >>>>> > > > things are defined as bugs). But I hope we can lay down a > > policy > > >>>>> that > > >>>>> > > gives > > >>>>> > > > everyone the right mindset for common cases and then discuss > > >>>>> corner > > >>>>> > cases > > >>>>> > > > on-demand in the future. > > >>>>> > > > > > >>>>> > > > [1] http://mesos.apache.org/documentation/latest/versioning/ > > >>>>> > > > <http://mesos.apache.org/documentation/latest/versioning/> > > >>>>> > > > > > >>>>> > > > > >>>>> > > > >>>>> > > >>>>> > > >>> > > >> > > > > >
