So, under the new version release policy, in principle, long-term maintenance branches, such as branch-4.x, are not allowed to have dependency changes. For example, if the Scala version 2.13.17 is used when branch-4.x is created, then throughout the entire lifecycle of branch-4.x, it should not, in principle, be updated to a newer Scala version. Is my understanding correct?
On 2025/12/09 01:30:43 Wenchen Fan wrote: > I like this idea, but we should call out the impact to Spark development. > IIUC, we need the following changes: > > - If a feature is known to take a long time to complete, we should add a > feature flag at the beginning, to avoid releasing a half-baked feature with > this faster relase cadence. > - We have defined behavior change in > https://spark.apache.org/contributing.html. According to it, new APIs, > new features, bug fixes, etc. are all behavior changes because they are > user-visible. We should be more clear about what can go to minor releases. > I think we can follow the same standard for writing migration guide: if the > behavior change needs user action. > - We will merge most PRs to both master and a new .x branch, and we > should avoid merge conflicts as possible as we can. For example, behavior > changes should also be merged into the .x branch, with flag off. > > An alternative idea is to still cut next release from the master branch, > but we only allow a 3-month window per year (right before the next major > release) to make breaking changes in the master branch, such as dependency > upgrade, Java/Scala version upgrade, etc. Of cource we can still have > exceptions with voting. The drawback is that sometimes 3 months is not > sufficient to make major upgrades, and then the .x branch will be useful. > > On Tue, Dec 9, 2025 at 9:02 AM L. C. Hsieh <[email protected]> wrote: > > > I see. > > > > In the "Code Merging Principle" section, > > > For non-behavior changes, always merge to master and the latest .x > > branch. This change will be released with the next minor release > > > > Is this .x branch meaning a branch of a major branch like branch-4.x? > > > > Also, looks like master and the latest .x branch basically have the > > same codebase? > > > > On Mon, Dec 8, 2025 at 4:37 PM Hyukjin Kwon <[email protected]> wrote: > > > > > > I actually intentionally disabled the commenter access so the discussion > > can happen here :-). Otherwise, we would end up with multiple places to > > discuss this. > > > > > > On Tue, 9 Dec 2025 at 09:33, L. C. Hsieh <[email protected]> wrote: > > >> > > >> Can you open comment access to the google doc? > > >> So it will be easier to ask questions directly on the SPIP doc. > > >> > > >> On Sun, Dec 7, 2025 at 1:53 PM Hyukjin Kwon <[email protected]> > > wrote: > > >> > > > >> > Hi all, > > >> > > > >> > I would like to start a discussion on accelerating the Apache Spark > > release cadence. Over the past four months, we have been running preview > > releases, and the process has been smooth and effective. As mentioned in > > the preview release discussion thread, I’d now like to extend this approach > > to official releases. > > >> > > > >> > During this period, I also looked into how other large projects, such > > as Kubernetes and Python, manage their release timelines. Based on that > > research and our own recent experience, I’ve drafted a proposal for an > > updated Apache Spark release plan. > > >> > > > >> > TL;DR: > > >> > > > >> > Introduce a predictable release schedule: annual major releases and > > quarterly minor releases, so users can benefit from new features earlier. > > >> > With a faster cadence for minor releases, we should take a more > > conservative approach toward behavior changes in minor versions, while > > still allowing new features and improvements. > > >> > > > >> > I’d love to hear your thoughts and feedback. > > >> > > > >> > More details can be found in SPIP: Accelerating Apache Spark Release > > Cadence > > >> > > > >> > Thanks! > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe e-mail: [email protected] > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > > > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
