Hi, I've been thinking a lot about maintenance and lifecycle policies and defect classification recently - I'm very grateful this is being raised. I believe establishing such policies will prove instrumental to enable adoption of Arrow for a number of use cases that prioritize stability over innovation.
On Wed, Oct 19, 2022 at 5:42 AM Antoine Pitrou <anto...@python.org> wrote: > > Hi Kou, > > Le 19/10/2022 à 06:29, Sutou Kouhei a écrit : > > > > My proposal: We maintain the last major release: > > * We maintain 9.Y.Z when the latest major release is 9.0.0 > > * We may release 9.Y.Z when we find a problem such as a > > security vulnerability in 9.Y.Z > > * We drop support for 9.Y.Z when we release 10.0.0 > > That sounds ok to me, but is there a more precise criterion than "we > find a problem"? > > In the past, we have from time to time done maintenance releases based > on annoying bugs/regressions. But not always. > I very much agree, and actually think there are multiple questions to answer here: 1. Which class of defects should be allowed to be merged into a maintenance branch? 2. Which class of defects must be fixed in a supported maintenance branch? 3. Which class of defects should trigger a maintenance release once a fix is made to the branch? 4. Which versions should be targeted in backporting a defect fix? How long will a release receive maintenance support? 5. Which class of defects can be batched into a future maintenance release, and which need immediate release? 6. What delivery artifacts are needed for maintenance releases? Can some things be source-only? Today, any fix may be a candidate for backporting to a maintenance branch if there's support for doing so in a vote. I believe it might be useful to more formally triage defects in part to establish policy answering these questions. For example: * How severe is the defect? Does it produce wrong results? Cause crashes? Or is it an annoying spelling error in a log message? * How widespread is the impact? Is everybody who uses Arrow going to be affected by this? Or is it only triggered by some very obscure use case? * How accessible is any workaround? * How much risk is involved in a fix? Having a common framework to classify those elements above would enable policy that clearly defines which defects can (or should, eventually) get what attention. If there is interest in the community, I'll continue a draft proposal I'm working on to formalize triage to capture these aspects. Any such triage process would be entirely optional for work done against master/main, but could be required for assessing potential backports as needed. I'll also note that I recognize Arrow may not currently see a need to answer all the questions about maintenance/lifecycle policy today, or may not have the resources needed to deliver what may be desired. It takes a lot of work to generate a release today. I think it's completely appropriate to commit only to what can be delivered today, with an eye towards incremental improvement. For example, an entirely acceptable policy might be: * Only the most recently-released minor version is eligible for defect fixes. * Security vulnerabilities with CVSS 3.0 score >= 7.0 (High) should trigger a maintenance release. * Fixes for defects of any nature may be backported if it reaches established thresholds (TBD) for severity, widespread impact, workaround accessibility and risk. Such fixes will be incorporated into the release maintenance release, made available via source, but no release will be produced unless triggered by a subsequent security vulnerability fix. > > I think that we can maintain multiple major releases with > > not high release cost by implementing the followings: > > * Green nightly CI > > * Nightly CI for all maintained branches (maint-X.Y.Z) > > * We need to reduce the time taken to CI > > * ... > > I'm afraid "green nightly CI" is more of an ideal than a reality given > the breadth and complexity of our fleet of CI jobs. We still seem to > have stability problems in some areas (perhaps Acero?) but there are > also regularly regressions due to changes in third-party packages. > Would this still be true if executed against a maintenance release branch? I understand why this would drift for main/master, but if a version branch is green when first released, and only accepts limited, qualified backported fixes, it should be much easier to "keep" green, I'd think. Thanks, Todd
