Hi, > Also, I know we have high release costs for new versions, but is that also > true for backporting fixes? Unlike new releases, if we were creating a > bugfix release, we are presumably starting from a much more stable point, > right?
Right. Additionally, I've improved/added shell scripts for some post release tasks (updating MSYS2/Homebrew/vcpkg packages) in the 10.0.0 release. Release costs will be reduced in future releases. Thanks, -- kou In <CAH6BBm2__JZ+dxvagRT0K1t=ttv-sbxo93p3md70zz1wxh0...@mail.gmail.com> "Re: [DISCUSS] Maintenance policy" on Wed, 19 Oct 2022 10:47:51 -0700, Will Jones <will.jones...@gmail.com> wrote: > One particular type of defect we might want to consider backporting to > supported versions are ones that silently produce incorrect data. Unlike > ones that cause a crash, it's not easy for a user to know they are affected. > > Here are a few examples: > > * ARROW-17453: [Go][C++][Parquet] Inconsistent Data with Repetition Levels > [1] (fixed in 10.0.0) > * ARROW-17995: [C++] Fix json decimals not being rescaled based on the > explicit schema [2] (fixed in 10.0.0) > * ARROW-14523: [C++] Fix potential data loss in S3 multipart upload [3] > (fixed in 7.0.0) > > Also, I know we have high release costs for new versions, but is that also > true for backporting fixes? Unlike new releases, if we were creating a > bugfix release, we are presumably starting from a much more stable point, > right? > > Thanks, > > Will Jones > > [1] https://issues.apache.org/jira/browse/ARROW-17453 > [2] https://issues.apache.org/jira/browse/ARROW-17995 > [3] https://issues.apache.org/jira/browse/ARROW-14523 > > On Wed, Oct 19, 2022 at 9:32 AM Todd Farmer <t...@voltrondata.com.invalid> > wrote: > >> Hi, >> >> I've been thinking a lot about maintenance and lifecycle policies and >> defect classification recently - I'm very grateful this is being raised. I >> believe establishing such policies will prove instrumental to enable >> adoption of Arrow for a number of use cases that prioritize stability over >> innovation. >> >> On Wed, Oct 19, 2022 at 5:42 AM Antoine Pitrou <anto...@python.org> wrote: >> >> > >> > Hi Kou, >> > >> > Le 19/10/2022 à 06:29, Sutou Kouhei a écrit : >> > > >> > > My proposal: We maintain the last major release: >> > > * We maintain 9.Y.Z when the latest major release is 9.0.0 >> > > * We may release 9.Y.Z when we find a problem such as a >> > > security vulnerability in 9.Y.Z >> > > * We drop support for 9.Y.Z when we release 10.0.0 >> > >> > That sounds ok to me, but is there a more precise criterion than "we >> > find a problem"? >> > >> > In the past, we have from time to time done maintenance releases based >> > on annoying bugs/regressions. But not always. >> > >> >> I very much agree, and actually think there are multiple questions to >> answer here: >> >> 1. Which class of defects should be allowed to be merged into a maintenance >> branch? >> 2. Which class of defects must be fixed in a supported maintenance branch? >> 3. Which class of defects should trigger a maintenance release once a fix >> is made to the branch? >> 4. Which versions should be targeted in backporting a defect fix? How long >> will a release receive maintenance support? >> 5. Which class of defects can be batched into a future maintenance release, >> and which need immediate release? >> 6. What delivery artifacts are needed for maintenance releases? Can some >> things be source-only? >> >> Today, any fix may be a candidate for backporting to a maintenance branch >> if there's support for doing so in a vote. I believe it might be useful to >> more formally triage defects in part to establish policy answering these >> questions. For example: >> >> * How severe is the defect? Does it produce wrong results? Cause crashes? >> Or is it an annoying spelling error in a log message? >> * How widespread is the impact? Is everybody who uses Arrow going to be >> affected by this? Or is it only triggered by some very obscure use case? >> * How accessible is any workaround? >> * How much risk is involved in a fix? >> >> Having a common framework to classify those elements above would enable >> policy that clearly defines which defects can (or should, eventually) get >> what attention. >> >> If there is interest in the community, I'll continue a draft proposal I'm >> working on to formalize triage to capture these aspects. Any such triage >> process would be entirely optional for work done against master/main, but >> could be required for assessing potential backports as needed. >> >> I'll also note that I recognize Arrow may not currently see a need to >> answer all the questions about maintenance/lifecycle policy today, or may >> not have the resources needed to deliver what may be desired. It takes a >> lot of work to generate a release today. I think it's completely >> appropriate to commit only to what can be delivered today, with an eye >> towards incremental improvement. For example, an entirely acceptable policy >> might be: >> >> * Only the most recently-released minor version is eligible for defect >> fixes. >> * Security vulnerabilities with CVSS 3.0 score >= 7.0 (High) should trigger >> a maintenance release. >> * Fixes for defects of any nature may be backported if it reaches >> established thresholds (TBD) for severity, widespread impact, workaround >> accessibility and risk. Such fixes will be incorporated into the release >> maintenance release, made available via source, but no release will be >> produced unless triggered by a subsequent security vulnerability fix. >> >> >> > > I think that we can maintain multiple major releases with >> > > not high release cost by implementing the followings: >> > > * Green nightly CI >> > > * Nightly CI for all maintained branches (maint-X.Y.Z) >> > > * We need to reduce the time taken to CI >> > > * ... >> > >> > I'm afraid "green nightly CI" is more of an ideal than a reality given >> > the breadth and complexity of our fleet of CI jobs. We still seem to >> > have stability problems in some areas (perhaps Acero?) but there are >> > also regularly regressions due to changes in third-party packages. >> > >> >> Would this still be true if executed against a maintenance release branch? >> I understand why this would drift for main/master, but if a version branch >> is green when first released, and only accepts limited, qualified >> backported fixes, it should be much easier to "keep" green, I'd think. >> >> Thanks, >> >> Todd >>
