On Wed, Nov 18, 2026 at 11:11 AM Russell Spitzer <[email protected]> wrote:
> We don't ship binaries for 4.2 so there is no package target for them. A > user could just build their own lib and use it. > OK, but could a Spark 4.2 user use the 4.1 binaries unmodified? Would they see a dependency error at build time, and/or a principled refusal to operate at runtime, and/or some kind of LinkageError at runtime? Thanks! -0xe1a > > On Wed, Nov 18, 2026 at 12:06 PM Alex Cruise <[email protected]> wrote: > >> I apologize if this should be obvious, but does the lack of official >> Iceberg support for Spark 4.2 mean that Spark 4.2 users can't use Iceberg >> at all, or merely that they can't benefit from enhancements available in >> Spark 4.2? >> >> Thanks! >> >> -0xe1a >> >> On Wed, Nov 18, 2026 at 2:44 AM Cheng Pan <[email protected]> wrote: >> >>> Thanks for reviving this discussion and sharing your thoughts. But I >>> think we had reached a consensus on the Spark version support policy when >>> adding Spark 4.2 support: the latest LTS + 2 latest minors, and the >>> situation still hold - Spark community runs with the planned version policy >>> and has no change since we made decision. >>> >>> And for "We have not released Spark 4.2 binaries yet.”, this was raised >>> by Anton, I believe it should be up to him to assess whether the current >>> Spark 4.2 connector meets his expectations for a binary release, and to >>> determine whether it should be included in the next 1.12.x patch release or >>> in 1.13.0. >>> >>> Thanks, >>> Cheng Pan >>> >>> >>> >>> On Nov 18, 2026, at 16:41, Anurag Mantripragada via dev < >>> [email protected]> wrote: >>> >>> Hi all, >>> >>> Reviving this thread because Spark 4.3.0 is out and Cheng opened #18749 >>> <https://github.com/apache/iceberg/pull/18749> to add it. We need to >>> decide whether to add 4.3 or skip this version entirely. >>> >>> Where we are: >>> >>> - We have not released Spark 4.2 binaries yet. >>> - Spark 4.3 is a non-LTS minor with a 6-month maintenance window. #18749 >>> <https://github.com/apache/iceberg/pull/18749>shows the diff is >>> small. >>> - Spark 4.4 is expected around January (branch is cut >>> <https://lists.apache.org/thread.html/8l3sv3ywtyk9j86r92gfmc3cj5xx1dyy> >>> in Spark already). >>> >>> I lean towards skipping 4.3 entirely and focusing on completing the pending >>> items in Spark 4.2 <https://github.com/apache/iceberg/issues/17959>. >>> Depending on the timeline, we can look into adding Spark 4.4 after the 1.13 >>> release. >>> >>> What do you all think? >>> >>> On Wed, Jun 17, 2026 at 3:56 PM Anton Okolnychyi <[email protected]> >>> wrote: >>> >>>> Given that we still rely on internals of Spark in some places, I think >>>> we will have to continue to have a module per Spark version. LTS + rolling >>>> window of 2 minors seems reasonable to me too. >>>> >>>> +1 on getting 4.2 support in to unlock development and dropping 4.0 >>>> after that. >>>> >>>> - Anton >>>> >>>> пн, 15 черв. 2026 р. о 07:30 Anurag Mantripragada < >>>> [email protected]> пише: >>>> >>>>> And I think whether to dropping Spark 4.0 support does not block >>>>>> adding Spark 4.2 support (with current RC state) >>>>> >>>>> >>>>> +1, we can review and merge Spark 4.2 support first. I will start a >>>>> vote to remove Spark 4.0 after the community sync tomorrow. >>>>> >>>>> Thanks, >>>>> Anurag >>>>> >>>>> On Mon, Jun 15, 2026 at 1:08 AM Cheng Pan <[email protected]> wrote: >>>>> >>>>>> Keep support for Spark last LTS + 2 latest minors sounds reasonable, >>>>>> if Iceberg could have a roughly 3 months release periods, which is >>>>>> similar >>>>>> to Spark and Flink, otherwise, it requires a wider version range to avoid >>>>>> version jumping when upgrading Iceberg. >>>>>> >>>>>> And I think whether to dropping Spark 4.0 support does not block >>>>>> adding Spark 4.2 support (with current RC state), I see at least two >>>>>> committer expressed the willingness on merging the spark 4.2 connector >>>>>> code >>>>>> as-is to unblock subsequent developing, and no objections, so, merge >>>>>> Spark >>>>>> 4.2 support first? And then start a vote thread for deleting Spark 4.0 >>>>>> support, if necessary? >>>>>> >>>>>> Thanks, >>>>>> Cheng Pan >>>>>> >>>>>> >>>
