Hi all, One more item I would like the list to weigh in on: the Spark 4 upgrade. Do we want it in 0.5.0, or should it move to 0.6.0?
Where we are today. main builds against Spark 3.4.2 and Scala 2.12, with a Java 8 compiler target. Delta is pinned at 2.4.0 and Iceberg at 1.9.2. We never moved the default to Spark 3.5 either. PR #671 was the last attempt at that and I closed it in April as stale. What the upgrade actually involves. Spark 4.0 only ships for Scala 2.13 and needs Java 17, so this is not a one-line version bump. It means: - flipping the default build to Scala 2.13. Our scala-2.13 profile is still marked "once Hudi supports Scala 2.13" and still passes -target:jvm-1.8. - raising the compiler target off Java 8. - moving Delta from 2.4.0 to 4.0.0 and Iceberg from 1.9.2 to 1.10.0 or later, since those are the first releases with Spark 4.0 artifacts. The upstream artifacts all exist already: hudi-spark4.0-bundle_2.13 1.2.0, iceberg-spark-runtime-4.0_2.13 1.10.0+, delta-spark_2.13 4.0.0, and paimon-spark-4.0 1.3.1. Nothing external blocks us. My own read is that it does not fit 0.5.0. We already have six items against a 30 September cutoff, and the Delta 2.4 to 4.0 jump lands directly on top of "Delta Kernel as the default Delta path". Doing both in one release makes it hard to tell which change broke what. I would rather land Kernel-as-default in 0.5.0, and then make Spark 4 a headline item for 0.6.0 with the 3.5 step folded into it. But I am happy to be argued out of this if someone wants to pick it up early. So the question for the community is just this one: Spark 4 in 0.5.0, or 0.6.0? If there is agreement on the direction I will file a tracking issue for it. Thanks, Vinish On Mon, Aug 17, 2026 03:58 PM, Vinish <[email protected]> wrote: > Thanks Tim, and thanks for taking it on. I have noted you as the release > manager for 0.5.0-incubating. > > Thanks, > Vinish > > On Thu, Aug 13, 2026 04:58 AM, Tim Brown <[email protected]> wrote: > >> Thanks for kicking off the discussion Vinish. I can be the release manager >> for 0.5.0. >> >> -Tim >> >> On Mon, Aug 10, 2026 at 9:01 PM Vinish <[email protected]> >> wrote: >> >> > Hi all, >> > >> > I have gone through the project's open issues and the earlier [DISCUSS] >> > threads on this list. Based on those, the following six seem like the >> > natural scope for 0.5.0-incubating, each with a tracking issue >> > 1. Hudi table version 9 - >> > https://github.com/apache/incubator-xtable/issues/834 >> > 2. Iceberg pluggable table format - >> > https://github.com/apache/incubator-xtable/issues/722 >> > 3. Delta Kernel as the default Delta path - >> > https://github.com/apache/incubator-xtable/issues/886 >> > 4. xtable-spark-runtime for Hudi 1.x - >> > https://github.com/apache/incubator-xtable/issues/836 >> > 5. Cross-format indexing support - >> > https://github.com/apache/incubator-xtable/issues/887 >> > 6. AI skills and tooling - >> > https://github.com/apache/incubator-xtable/issues/888 >> > >> > What else should we pick up? If you are working on something you would >> like >> > in 0.5.0-incubating, please reply here. >> > >> > Timeline. I think we should time-box these releases and fit the scope to >> > the date. If the community agrees, the cutoffs would be: >> > - 0.5.0: 30 September 2026 >> > - 0.6.0: 15 November 2026 >> > - 0.7.0: 5 January 2027 >> > >> > Realistically, some of these may not be ready by the first cutoff. If >> that >> > happens, my preference would be to hold the date and let the scope >> move, so >> > anything not quite ready is picked up in the next release a few weeks >> > later. That said, these dates are only a starting point, and we can >> adjust >> > them if they do not work for the community. >> > >> > Release manager. Stamatis started a thread on rotating the roll, and >> Vinoth >> > kindly offered to take 0.6.0 or 0.7.0, which leaves 0.5.0 open. Any >> > volunteers for 0.5.0 release manager? >> > >> > Thanks, >> > Vinish >> > >> >
