On Wed, Nov 18, 2026 at 11:11 AM Russell Spitzer <[email protected]>
wrote:

> We don't ship binaries for 4.2 so there is no package target for them. A
> user could just build their own lib and use it.
>

OK, but could a Spark 4.2 user use the 4.1 binaries unmodified? Would they
see a dependency error at build time, and/or a principled refusal to
operate at runtime, and/or some kind of LinkageError at runtime?

Thanks!

-0xe1a


>
> On Wed, Nov 18, 2026 at 12:06 PM Alex Cruise <[email protected]> wrote:
>
>> I apologize if this should be obvious, but does the lack of official
>> Iceberg support for Spark 4.2 mean that Spark 4.2 users can't use Iceberg
>> at all, or merely that they can't benefit from enhancements available in
>> Spark 4.2?
>>
>> Thanks!
>>
>> -0xe1a
>>
>> On Wed, Nov 18, 2026 at 2:44 AM Cheng Pan <[email protected]> wrote:
>>
>>> Thanks for reviving this discussion and sharing your thoughts. But I
>>> think we had reached a consensus on the Spark version support policy when
>>> adding Spark 4.2 support: the latest LTS + 2 latest minors, and the
>>> situation still hold - Spark community runs with the planned version policy
>>> and has no change since we made decision.
>>>
>>> And for "We have not released Spark 4.2 binaries yet.”, this was raised
>>> by Anton, I believe it should be up to him to assess whether the current
>>> Spark 4.2 connector meets his expectations for a binary release, and to
>>> determine whether it should be included in the next 1.12.x patch release or
>>> in 1.13.0.
>>>
>>> Thanks,
>>> Cheng Pan
>>>
>>>
>>>
>>> On Nov 18, 2026, at 16:41, Anurag Mantripragada via dev <
>>> [email protected]> wrote:
>>>
>>> Hi all,
>>>
>>> Reviving this thread because Spark 4.3.0 is out and Cheng opened #18749
>>> <https://github.com/apache/iceberg/pull/18749> to add it. We need to
>>> decide whether to add 4.3 or skip this version entirely.
>>>
>>> Where we are:
>>>
>>>    - We have not released Spark 4.2 binaries yet.
>>>    - Spark 4.3 is a non-LTS minor with a 6-month maintenance window. #18749
>>>    <https://github.com/apache/iceberg/pull/18749>shows the diff is
>>>    small.
>>>    - Spark 4.4 is expected around January (branch is cut
>>>    <https://lists.apache.org/thread.html/8l3sv3ywtyk9j86r92gfmc3cj5xx1dyy>
>>>    in Spark already).
>>>
>>> I lean towards skipping 4.3 entirely and focusing on completing the pending
>>> items in Spark 4.2 <https://github.com/apache/iceberg/issues/17959>.
>>> Depending on the timeline, we can look into adding Spark 4.4 after the 1.13
>>> release.
>>>
>>> What do you all think?
>>>
>>> On Wed, Jun 17, 2026 at 3:56 PM Anton Okolnychyi <[email protected]>
>>> wrote:
>>>
>>>> Given that we still rely on internals of Spark in some places, I think
>>>> we will have to continue to have a module per Spark version. LTS + rolling
>>>> window of 2 minors seems reasonable to me too.
>>>>
>>>> +1 on getting 4.2 support in to unlock development and dropping 4.0
>>>> after that.
>>>>
>>>> - Anton
>>>>
>>>> пн, 15 черв. 2026 р. о 07:30 Anurag Mantripragada <
>>>> [email protected]> пише:
>>>>
>>>>> And I think whether to dropping Spark 4.0 support does not block
>>>>>> adding Spark 4.2 support (with current RC state)
>>>>>
>>>>>
>>>>> +1, we can review and merge Spark 4.2 support first. I will start a
>>>>> vote to remove Spark 4.0 after the community sync tomorrow.
>>>>>
>>>>> Thanks,
>>>>> Anurag
>>>>>
>>>>> On Mon, Jun 15, 2026 at 1:08 AM Cheng Pan <[email protected]> wrote:
>>>>>
>>>>>> Keep support for Spark last LTS + 2 latest minors sounds reasonable,
>>>>>> if Iceberg could have a roughly 3 months release periods, which is 
>>>>>> similar
>>>>>> to Spark and Flink, otherwise, it requires a wider version range to avoid
>>>>>> version jumping when upgrading Iceberg.
>>>>>>
>>>>>> And I think whether to dropping Spark 4.0 support does not block
>>>>>> adding Spark 4.2 support (with current RC state), I see at least two
>>>>>> committer expressed the willingness on merging the spark 4.2 connector 
>>>>>> code
>>>>>> as-is to unblock subsequent developing, and no objections, so, merge 
>>>>>> Spark
>>>>>> 4.2 support first? And then start a vote thread for deleting Spark 4.0
>>>>>> support, if necessary?
>>>>>>
>>>>>> Thanks,
>>>>>> Cheng Pan
>>>>>>
>>>>>>
>>>

Reply via email to