Hi Alex, Each Iceberg runtime jar is built for one Spark minor, and running it on a different minor fails at runtime (See this thread <https://github.com/apache/iceberg/issues/17959#issuecomment-6121779365> for example). Until we publish 4.2 jars, production Spark 4.2 users would need to build their own from source.
~ Anurag On Wed, Nov 18, 2026 at 10:33 AM Alex Cruise <[email protected]> wrote: > On Wed, Nov 18, 2026 at 11:11 AM Russell Spitzer < > [email protected]> wrote: > >> We don't ship binaries for 4.2 so there is no package target for them. A >> user could just build their own lib and use it. >> > > OK, but could a Spark 4.2 user use the 4.1 binaries unmodified? Would they > see a dependency error at build time, and/or a principled refusal to > operate at runtime, and/or some kind of LinkageError at runtime? > > Thanks! > > -0xe1a > > >> >> On Wed, Nov 18, 2026 at 12:06 PM Alex Cruise <[email protected]> wrote: >> >>> I apologize if this should be obvious, but does the lack of official >>> Iceberg support for Spark 4.2 mean that Spark 4.2 users can't use Iceberg >>> at all, or merely that they can't benefit from enhancements available in >>> Spark 4.2? >>> >>> Thanks! >>> >>> -0xe1a >>> >>> On Wed, Nov 18, 2026 at 2:44 AM Cheng Pan <[email protected]> wrote: >>> >>>> Thanks for reviving this discussion and sharing your thoughts. But I >>>> think we had reached a consensus on the Spark version support policy when >>>> adding Spark 4.2 support: the latest LTS + 2 latest minors, and the >>>> situation still hold - Spark community runs with the planned version policy >>>> and has no change since we made decision. >>>> >>>> And for "We have not released Spark 4.2 binaries yet.”, this was raised >>>> by Anton, I believe it should be up to him to assess whether the current >>>> Spark 4.2 connector meets his expectations for a binary release, and to >>>> determine whether it should be included in the next 1.12.x patch release or >>>> in 1.13.0. >>>> >>>> Thanks, >>>> Cheng Pan >>>> >>>> >>>> >>>> On Nov 18, 2026, at 16:41, Anurag Mantripragada via dev < >>>> [email protected]> wrote: >>>> >>>> Hi all, >>>> >>>> Reviving this thread because Spark 4.3.0 is out and Cheng opened #18749 >>>> <https://github.com/apache/iceberg/pull/18749> to add it. We need to >>>> decide whether to add 4.3 or skip this version entirely. >>>> >>>> Where we are: >>>> >>>> - We have not released Spark 4.2 binaries yet. >>>> - Spark 4.3 is a non-LTS minor with a 6-month maintenance window. #18749 >>>> <https://github.com/apache/iceberg/pull/18749>shows the diff is >>>> small. >>>> - Spark 4.4 is expected around January (branch is cut >>>> <https://lists.apache.org/thread.html/8l3sv3ywtyk9j86r92gfmc3cj5xx1dyy> >>>> in Spark already). >>>> >>>> I lean towards skipping 4.3 entirely and focusing on completing the pending >>>> items in Spark 4.2 <https://github.com/apache/iceberg/issues/17959>. >>>> Depending on the timeline, we can look into adding Spark 4.4 after the 1.13 >>>> release. >>>> >>>> What do you all think? >>>> >>>> On Wed, Jun 17, 2026 at 3:56 PM Anton Okolnychyi <[email protected]> >>>> wrote: >>>> >>>>> Given that we still rely on internals of Spark in some places, I think >>>>> we will have to continue to have a module per Spark version. LTS + rolling >>>>> window of 2 minors seems reasonable to me too. >>>>> >>>>> +1 on getting 4.2 support in to unlock development and dropping 4.0 >>>>> after that. >>>>> >>>>> - Anton >>>>> >>>>> пн, 15 черв. 2026 р. о 07:30 Anurag Mantripragada < >>>>> [email protected]> пише: >>>>> >>>>>> And I think whether to dropping Spark 4.0 support does not block >>>>>>> adding Spark 4.2 support (with current RC state) >>>>>> >>>>>> >>>>>> +1, we can review and merge Spark 4.2 support first. I will start a >>>>>> vote to remove Spark 4.0 after the community sync tomorrow. >>>>>> >>>>>> Thanks, >>>>>> Anurag >>>>>> >>>>>> On Mon, Jun 15, 2026 at 1:08 AM Cheng Pan <[email protected]> wrote: >>>>>> >>>>>>> Keep support for Spark last LTS + 2 latest minors sounds reasonable, >>>>>>> if Iceberg could have a roughly 3 months release periods, which is >>>>>>> similar >>>>>>> to Spark and Flink, otherwise, it requires a wider version range to >>>>>>> avoid >>>>>>> version jumping when upgrading Iceberg. >>>>>>> >>>>>>> And I think whether to dropping Spark 4.0 support does not block >>>>>>> adding Spark 4.2 support (with current RC state), I see at least two >>>>>>> committer expressed the willingness on merging the spark 4.2 connector >>>>>>> code >>>>>>> as-is to unblock subsequent developing, and no objections, so, merge >>>>>>> Spark >>>>>>> 4.2 support first? And then start a vote thread for deleting Spark 4.0 >>>>>>> support, if necessary? >>>>>>> >>>>>>> Thanks, >>>>>>> Cheng Pan >>>>>>> >>>>>>> >>>>
