On the other hand, regarding long-term maintenance branches, such as branch-4.x, under the new version release policy, is it necessary to fix the versions of Java/Python? For instance, Java `17.0.16` or Python 3.11.9. Previously, even for branch-4.1, GitHub Actions tested the latest versions of Java 17 and Python 3.11 rather than being fixed to a specific version.
On 2025/12/09 03:13:58 Yang Jie wrote: > So, under the new version release policy, in principle, long-term maintenance > branches, such as branch-4.x, are not allowed to have dependency changes. For > example, if the Scala version 2.13.17 is used when branch-4.x is created, > then throughout the entire lifecycle of branch-4.x, it should not, in > principle, be updated to a newer Scala version. Is my understanding correct? > > On 2025/12/09 01:30:43 Wenchen Fan wrote: > > I like this idea, but we should call out the impact to Spark development. > > IIUC, we need the following changes: > > > > - If a feature is known to take a long time to complete, we should add a > > feature flag at the beginning, to avoid releasing a half-baked feature > > with > > this faster relase cadence. > > - We have defined behavior change in > > https://spark.apache.org/contributing.html. According to it, new APIs, > > new features, bug fixes, etc. are all behavior changes because they are > > user-visible. We should be more clear about what can go to minor > > releases. > > I think we can follow the same standard for writing migration guide: if > > the > > behavior change needs user action. > > - We will merge most PRs to both master and a new .x branch, and we > > should avoid merge conflicts as possible as we can. For example, behavior > > changes should also be merged into the .x branch, with flag off. > > > > An alternative idea is to still cut next release from the master branch, > > but we only allow a 3-month window per year (right before the next major > > release) to make breaking changes in the master branch, such as dependency > > upgrade, Java/Scala version upgrade, etc. Of cource we can still have > > exceptions with voting. The drawback is that sometimes 3 months is not > > sufficient to make major upgrades, and then the .x branch will be useful. > > > > On Tue, Dec 9, 2025 at 9:02 AM L. C. Hsieh <[email protected]> wrote: > > > > > I see. > > > > > > In the "Code Merging Principle" section, > > > > For non-behavior changes, always merge to master and the latest .x > > > branch. This change will be released with the next minor release > > > > > > Is this .x branch meaning a branch of a major branch like branch-4.x? > > > > > > Also, looks like master and the latest .x branch basically have the > > > same codebase? > > > > > > On Mon, Dec 8, 2025 at 4:37 PM Hyukjin Kwon <[email protected]> wrote: > > > > > > > > I actually intentionally disabled the commenter access so the discussion > > > can happen here :-). Otherwise, we would end up with multiple places to > > > discuss this. > > > > > > > > On Tue, 9 Dec 2025 at 09:33, L. C. Hsieh <[email protected]> wrote: > > > >> > > > >> Can you open comment access to the google doc? > > > >> So it will be easier to ask questions directly on the SPIP doc. > > > >> > > > >> On Sun, Dec 7, 2025 at 1:53 PM Hyukjin Kwon <[email protected]> > > > wrote: > > > >> > > > > >> > Hi all, > > > >> > > > > >> > I would like to start a discussion on accelerating the Apache Spark > > > release cadence. Over the past four months, we have been running preview > > > releases, and the process has been smooth and effective. As mentioned in > > > the preview release discussion thread, I’d now like to extend this > > > approach > > > to official releases. > > > >> > > > > >> > During this period, I also looked into how other large projects, such > > > as Kubernetes and Python, manage their release timelines. Based on that > > > research and our own recent experience, I’ve drafted a proposal for an > > > updated Apache Spark release plan. > > > >> > > > > >> > TL;DR: > > > >> > > > > >> > Introduce a predictable release schedule: annual major releases and > > > quarterly minor releases, so users can benefit from new features earlier. > > > >> > With a faster cadence for minor releases, we should take a more > > > conservative approach toward behavior changes in minor versions, while > > > still allowing new features and improvements. > > > >> > > > > >> > I’d love to hear your thoughts and feedback. > > > >> > > > > >> > More details can be found in SPIP: Accelerating Apache Spark Release > > > Cadence > > > >> > > > > >> > Thanks! > > > >> > > > >> --------------------------------------------------------------------- > > > >> To unsubscribe e-mail: [email protected] > > > >> > > > > > > --------------------------------------------------------------------- > > > To unsubscribe e-mail: [email protected] > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
