We got the final Spark backport, #16303 <https://github.com/apache/iceberg/pull/16303>, merged late last night. Thanks to everyone who helped make that happen. At this point, there are no remaining 1.11.0 blockers listed in the milestone <https://github.com/apache/iceberg/milestone/59> .
Thanks, Steve, for raising the SerializableFileIOWithSize concern—it's a valid point from a stabilization perspective to introduce #15470. As I understand from the conversion (please chime in if you have more context), it's not considered a regression and I think we should move forward and cut the next release candidate. Please let me know your thoughts. On Thu, May 14, 2026 at 6:08 AM Steve Loughran <[email protected]> wrote: > > fixing SerializableFileIOWithSize, > https://github.com/apache/iceberg/pull/16284, makes that file length more > of an issue, especially with the GCS file io, which uses the supplied file > length to limit its range. > > But that #15470 is still stabilising, and last minute fixes are always > dangerous as devs don't get enough time to play with them. It makes for a > dangerous last-minute patch > > On Wed, 13 May 2026 at 05:46, huaxin gao <[email protected]> wrote: > >> This is a correctness issue, but it is not a new regression; the same >> issue already exists in 1.10. The PR description lists Trino, Impala, >> Comet, and iceberg-rust as affected readers. I wouldn’t call it a hard >> blocker for 1.11.0 because it is a pre-existing problem. >> >> Thanks, >> Huaxin >> >> On Tue, May 12, 2026 at 8:44 PM Aihua Xu <[email protected]> wrote: >> >>> Huaxin, how critical is this one? The bug has been reported for a while >>> and it's still being worked on. Can you add to the milestone 1.11.0 if it's >>> a blocker so we can track? >>> >>> One more PR to consider: https://github.com/apache/iceberg/pull/15470 fixes >>>> a correctness issue in rewriteTablePath, where manifests can record >>>> stale file_size_in_bytes values for rewritten position delete files. >>>> We probably want to include this fix in 1.11 too. >>> >>> >>> On Tue, May 12, 2026 at 7:33 PM Aihua Xu <[email protected]> wrote: >>> >>>> If it’s needed, I will wait for it. Thanks for working on them. >>>> >>>> On May 12, 2026, at 7:17 PM, Kevin Liu <[email protected]> wrote: >>>> >>>> >>>> 3 of the PRs have been merged. Thank you Huaxin for the review. I >>>> merged it since it was mostly clean backports and only targets spark 3.4. >>>> The last PR is pending CI and also a clean backport, >>>> https://github.com/apache/iceberg/pull/16311 >>>> >>>> >>>> >>>> On Tue, May 12, 2026 at 6:52 PM Kevin Liu <[email protected]> >>>> wrote: >>>> >>>>> I'll take a look at https://github.com/apache/iceberg/pull/15470 >>>>> >>>>> Here's the Spark 3.4 PRs. I only backported PRs with relevant code >>>>> changes: >>>>> https://github.com/apache/iceberg/pull/16306 (Backport of #14483 + >>>>> #14497) >>>>> https://github.com/apache/iceberg/pull/16307 (Backport of #15683 + >>>>> #16284) >>>>> https://github.com/apache/iceberg/pull/16308 (Backport of #15832) >>>>> https://github.com/apache/iceberg/pull/16311 (Backport of #15992) >>>>> This one needs to rebase #16307 above >>>>> >>>>> They are mostly clean backports, some with minimal change. The first 3 >>>>> already passed CI. >>>>> I would like to have these in. But will defer to Aihua (RM) for the >>>>> final call. >>>>> >>>>> Best, >>>>> Kevin Liu >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, May 12, 2026 at 6:39 PM Manu Zhang <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Aihua, >>>>>> >>>>>> Since we plan to drop Spark 3.4 after 1.11.0, let's get the back-port >>>>>> PRs in. Otherwise, it will be left in a broken state. >>>>>> >>>>>> Thanks, >>>>>> Manu >>>>>> >>>>>> On Wed, May 13, 2026 at 9:16 AM Aihua Xu <[email protected]> wrote: >>>>>> >>>>>>> Thanks everyone for driving these blockers to closure. >>>>>>> >>>>>>> Kevin, since this isn’t blocking and Spark 3.4 is deprecated, I’d >>>>>>> like to go ahead and cut the next release candidate tonight so we can >>>>>>> move >>>>>>> forward—unless anyone disagrees. If we end up needing another RC, we can >>>>>>> consider adding them in. What do you think? >>>>>>> >>>>>>> >>>>>>> On Tue, May 12, 2026 at 4:48 PM Kevin Liu <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Ajay's email was stuck in webmod, i just unblocked it. >>>>>>>> >>>>>>>> Looks like all the issues in this email chain have been resolved. >>>>>>>> - first row ID https://github.com/apache/iceberg/pull/16263 >>>>>>>> - analyticscore https://github.com/apache/iceberg/pull/16258 >>>>>>>> - SerializableFileIOWithSize >>>>>>>> https://github.com/apache/iceberg/pull/16284 >>>>>>>> >>>>>>>> Thanks everyone for contributing to the fix! >>>>>>>> >>>>>>>> The 1.11.0 milestone is 100% complete at this time, >>>>>>>> https://github.com/apache/iceberg/milestone/59 >>>>>>>> >>>>>>>> One _last_ thing, I went over the potential feature parity gap >>>>>>>> between the four different Spark versions we currently support. It >>>>>>>> looks >>>>>>>> like there are a couple of PRs that can be backported to Spark 3.4 but >>>>>>>> haven't been. Since this is the last release that supports Spark 3.4, >>>>>>>> I'd >>>>>>>> like to backport them and close the parity gap. This is completely >>>>>>>> optional >>>>>>>> since we've already marked Spark 3.4 as deprecated, but I think it's a >>>>>>>> good >>>>>>>> gesture for its final release. >>>>>>>> >>>>>>>> Best, >>>>>>>> Kevin Liu >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 12, 2026 at 3:57 PM Ajay Yadav <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Everyone, >>>>>>>>> >>>>>>>>> I would like to report a performance regression we've identified >>>>>>>>> in Spark queries on Iceberg tables stored in cloud storage (tested >>>>>>>>> with >>>>>>>>> GCS), which I believe should be addressed in the 1.11.0 release. >>>>>>>>> >>>>>>>>> Current SerializableFileIOWithSize drops file length, causing >>>>>>>>> performance regression due to excessive metadata calls in Cloud >>>>>>>>> Storage: >>>>>>>>> https://github.com/apache/iceberg/ssues/16283. The fix overrides >>>>>>>>> InputFile newInputFile(String path, long length) to preserve file >>>>>>>>> length and avoid unwanted metadata calls >>>>>>>>> https://github.com/apache/iceberg/pull/16284 >>>>>>>>> >>>>>>>>> On 2026/05/08 15:27:05 Péter Váry wrote: >>>>>>>>> > Just to clarify: >>>>>>>>> > >>>>>>>>> > The following PRs are already merged to 1.11.0: >>>>>>>>> > >>>>>>>>> > - https://github.com/apache/iceberg/pull/14297 - Spark: >>>>>>>>> Support writing >>>>>>>>> > shredded variant in Iceberg-Spark >>>>>>>>> > - https://github.com/apache/iceberg/pull/15512 - Spark: fix >>>>>>>>> delete from >>>>>>>>> > branch for canDeleteWhere where it does not resolve to the >>>>>>>>> correct branch - >>>>>>>>> > WAP fix >>>>>>>>> > - https://github.com/apache/iceberg/pull/15475 - Flink: Add >>>>>>>>> Nanosecond >>>>>>>>> > Precision Support for Flink-Iceberg Integration >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > The missing ones are the ones backporting those to other engine >>>>>>>>> versions: >>>>>>>>> > >>>>>>>>> > - For: 14297 <https://github.com/apache/iceberg/pull/14297>: >>>>>>>>> > - 16241 <https://github.com/apache/iceberg/pull/16241> - >>>>>>>>> Backport for >>>>>>>>> > variant shredding in Spark 4.0 >>>>>>>>> > - For: 15512 <https://github.com/apache/iceberg/pull/15512>: >>>>>>>>> > - 16245 <https://github.com/apache/iceberg/pull/16245> - >>>>>>>>> Spark: >>>>>>>>> > backport PR #15512 to v3.4, v3.5, v4.0 for WAP branch >>>>>>>>> delete fix >>>>>>>>> > - For: 15475 <https://github.com/apache/iceberg/pull/15475>: >>>>>>>>> > - #16183 <https://github.com/apache/iceberg/pull/16183>, >>>>>>>>> #16239 >>>>>>>>> > <https://github.com/apache/iceberg/pull/16239>, #16240 >>>>>>>>> > <https://github.com/apache/iceberg/pull/16240> - Backport >>>>>>>>> for Nano >>>>>>>>> > timestamps for Flink 2.0/1.20 >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > So the PRs needed on 1.11.0 are: >>>>>>>>> > https://github.com/apache/iceberg/pull/16241 >>>>>>>>> > https://github.com/apache/iceberg/pull/16245 >>>>>>>>> > https://github.com/apache/iceberg/pull/16183 >>>>>>>>> > https://github.com/apache/iceberg/pull/16239 >>>>>>>>> > https://github.com/apache/iceberg/pull/16240 >>>>>>>>> > https://github.com/apache/iceberg/pull/16186 >>>>>>>>> > >>>>>>>>> > Aihua Xu <[email protected]> ezt írta (időpont: 2026. máj. 8., P, >>>>>>>>> 17:13): >>>>>>>>> > >>>>>>>>> > > Thank you all for the feedback and for verifying the release >>>>>>>>> candidate. >>>>>>>>> > > Based on the issues identified above, we will include the >>>>>>>>> following fixes >>>>>>>>> > > and cut RC2 with a new vote: >>>>>>>>> > > >>>>>>>>> > > https://github.com/apache/iceberg/pull/14297 >>>>>>>>> > > https://github.com/apache/iceberg/pull/15512 >>>>>>>>> > > https://github.com/apache/iceberg/pull/15475 >>>>>>>>> > > https://github.com/apache/iceberg/pull/16186 >>>>>>>>> > > >>>>>>>>> > > Please let me know if you have any questions or identified >>>>>>>>> additional >>>>>>>>> > > issues. >>>>>>>>> > > >>>>>>>>> > > Thanks, >>>>>>>>> > > Aihua >>>>>>>>> > > >>>>>>>>> > > On Thu, May 7, 2026 at 10:09 PM Aihua Xu <[email protected]> >>>>>>>>> wrote: >>>>>>>>> > > >>>>>>>>> > >> I also looked into this. There is a configuration >>>>>>>>> > >> gcs.analytics-core.enabled to enable/disable GCS Analytics >>>>>>>>> Core. The >>>>>>>>> > >> current implementation always requires runtime dependency of >>>>>>>>> GCS Analytics >>>>>>>>> > >> Core even if the configuration is off. Ideally we can lazy >>>>>>>>> load such >>>>>>>>> > >> dependency so the dependency is only required when the >>>>>>>>> feature is >>>>>>>>> > >> explicitly enabled. But since GCP is likely to enable GCS >>>>>>>>> Analytics Core by >>>>>>>>> > >> default, I feel it's reasonable for downstream projects using >>>>>>>>> non-bundle >>>>>>>>> > >> jars to add this dependency. >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> > >> On Thu, May 7, 2026 at 6:54 PM Steven Wu <[email protected]> >>>>>>>>> wrote: >>>>>>>>> > >> >>>>>>>>> > >>> Looked a little more. >>>>>>>>> > >>> >>>>>>>>> > >>> So Iceberg's cloud modules consistently use compileOnly for >>>>>>>>> vendor SDKs >>>>>>>>> > >>> and rely on either the bundle artifact or downstream >>>>>>>>> coordination for >>>>>>>>> > >>> runtime. So, both changes are expected for downstream >>>>>>>>> consumers using the >>>>>>>>> > >>> non-bundle jars. Maybe we don't need to change anything. >>>>>>>>> > >>> >>>>>>>>> > >>> iceberg-gcp module >>>>>>>>> > >>> >>>>>>>>> > >>> compileOnly platform(libs.google.libraries.bom) >>>>>>>>> > >>> compileOnly "com.google.cloud:google-cloud-storage" >>>>>>>>> > >>> compileOnly "com.google.cloud:google-cloud-kms" >>>>>>>>> > >>> compileOnly(libs.gcs.analytics.core) >>>>>>>>> > >>> >>>>>>>>> > >>> >>>>>>>>> > >>> On Thu, May 7, 2026 at 6:16 PM Steven Wu <[email protected]> >>>>>>>>> wrote: >>>>>>>>> > >>> >>>>>>>>> > >>>> Yuya, thanks for reporting the discovery. >>>>>>>>> > >>>> >>>>>>>>> > >>>> Azure: I approved your PR and can merge it soon: >>>>>>>>> > >>>> https://github.com/apache/iceberg/pull/16186 >>>>>>>>> > >>>> GCP: the new dependency is marked as compileOnly in PR 14333 >>>>>>>>> > >>>> <https://github.com/apache/iceberg/pull/14333>, as it is >>>>>>>>> an opt-in >>>>>>>>> > >>>> feature. we need to either change the dep to implementation >>>>>>>>> or update the >>>>>>>>> > >>>> code similar to the Azure fix above. >>>>>>>>> > >>>> >>>>>>>>> > >>>> >>>>>>>>> > >>>> On Thu, May 7, 2026 at 4:07 PM Yuya Ebihara < >>>>>>>>> > >>>> [email protected]> wrote: >>>>>>>>> > >>>> >>>>>>>>> > >>>>> Hi Aihua, >>>>>>>>> > >>>>> >>>>>>>>> > >>>>> Thanks for leading the release! >>>>>>>>> > >>>>> >>>>>>>>> > >>>>> Just a quick reminder about two dependency-related items >>>>>>>>> from a >>>>>>>>> > >>>>> downstream perspective: >>>>>>>>> > >>>>> * Azure module users will require >>>>>>>>> azure-security-keyvault-keys, even >>>>>>>>> > >>>>> when table encryption is not used, as noted in >>>>>>>>> > >>>>> https://github.com/apache/iceberg/pull/16186 >>>>>>>>> > >>>>> * GCS module users will require gcs-analytics-core >>>>>>>>> > >>>>> >>>>>>>>> > >>>>> I ran into CI failures with 1.11.0 in Trino because the >>>>>>>>> project does >>>>>>>>> > >>>>> not use the azure-bundle or gcp-bundle modules. >>>>>>>>> > >>>>> The CI passed once we explicitly added these two >>>>>>>>> dependencies. >>>>>>>>> > >>>>> >>>>>>>>> > >>>>> Thanks, >>>>>>>>> > >>>>> Yuya Ebihara >>>>>>>>> > >>>>> >>>>>>>>> > >>>>> On Fri, May 8, 2026 at 4:58 AM Péter Váry <[email protected] >>>>>>>>> > >>>>>>>>> > >>>>> wrote: >>>>>>>>> > >>>>> >>>>>>>>> > >>>>>> First of all, thanks to everyone for the effort put into >>>>>>>>> preparing >>>>>>>>> > >>>>>> this release! >>>>>>>>> > >>>>>> >>>>>>>>> > >>>>>> I would like to highlight that RC1 is built from a branch >>>>>>>>> where the >>>>>>>>> > >>>>>> following features have not been backported to all engine >>>>>>>>> versions: >>>>>>>>> > >>>>>> - Spark: Support writing shredded variant in >>>>>>>>> Iceberg-Spark ( >>>>>>>>> > >>>>>> https://github.com/apache/iceberg/pull/14297) - >>>>>>>>> Available in Spark >>>>>>>>> > >>>>>> 4.1, but not in Spark 4.0 >>>>>>>>> > >>>>>> - Spark: fix delete from branch for canDeleteWhere where >>>>>>>>> it does not >>>>>>>>> > >>>>>> resolve to the correct branch ( >>>>>>>>> > >>>>>> https://github.com/apache/iceberg/pull/15512) - >>>>>>>>> Available in Spark >>>>>>>>> > >>>>>> 4.1, but not in Spark 4.0, 3.5, or 3.4 >>>>>>>>> > >>>>>> - Flink: Add Nanosecond Precision Support for >>>>>>>>> Flink-Iceberg >>>>>>>>> > >>>>>> Integration (https://github.com/apache/iceberg/pull/15475) >>>>>>>>> - >>>>>>>>> > >>>>>> Available in Flink 2.1, but not in Flink 2.0 or 1.20 >>>>>>>>> > >>>>>> >>>>>>>>> > >>>>>> It is up to the community to decide whether these missing >>>>>>>>> backports >>>>>>>>> > >>>>>> should be considered release blockers. Most of the >>>>>>>>> corresponding PRs have >>>>>>>>> > >>>>>> already been merged to main (except #15512), and >>>>>>>>> including them in the >>>>>>>>> > >>>>>> release should be relatively straightforward. >>>>>>>>> > >>>>>> >>>>>>>>> > >>>>>> From my perspective, I would prefer not to release with >>>>>>>>> these gaps. >>>>>>>>> > >>>>>> That said, I understand the urgency and the need for a >>>>>>>>> release, and I am >>>>>>>>> > >>>>>> happy to go with the community’s decision. >>>>>>>>> > >>>>>> >>>>>>>>> > >>>>>> Peter >>>>>>>>> > >>>>>> >>>>>>>>> > >>>>>> Aihua Xu <[email protected]> ezt írta (időpont: 2026. máj. >>>>>>>>> 7., Cs, >>>>>>>>> > >>>>>> 18:26): >>>>>>>>> > >>>>>> >>>>>>>>> > >>>>>>> Hi Everyone, >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> I propose that we release the following RC as the >>>>>>>>> official Apache >>>>>>>>> > >>>>>>> Iceberg 1.11.0 release. >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> The commit ID is 0f657edf12dc29f8487a679bfdd4210e9588d014 >>>>>>>>> > >>>>>>> * This corresponds to the tag: apache-iceberg-1.11.0-rc1 >>>>>>>>> > >>>>>>> * >>>>>>>>> > >>>>>>> >>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.11.0-rc1 >>>>>>>>> > >>>>>>> * >>>>>>>>> > >>>>>>> >>>>>>>>> https://github.com/apache/iceberg/tree/0f657edf12dc29f8487a679bfdd4210e9588d014 >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> The release tarball, signature, and checksums are here: >>>>>>>>> > >>>>>>> * >>>>>>>>> > >>>>>>> >>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.11.0-rc1 >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> You can find the KEYS file here: >>>>>>>>> > >>>>>>> * https://downloads.apache.org/iceberg/KEYS >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> Convenience binary artifacts are staged on Nexus. The >>>>>>>>> Maven >>>>>>>>> > >>>>>>> repository URL is: >>>>>>>>> > >>>>>>> * >>>>>>>>> > >>>>>>> >>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1278/ >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> Please download, verify, and test. >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> Instructions for verifying a release can be found here: >>>>>>>>> > >>>>>>> * >>>>>>>>> https://iceberg.apache.org/how-to-release/#how-to-verify-a-release >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> Please vote in the next 72 hours. >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> [ ] +1 Release this as Apache Iceberg 1.11.0 >>>>>>>>> > >>>>>>> [ ] +0 >>>>>>>>> > >>>>>>> [ ] -1 Do not release this because... >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> Only PMC members have binding votes, but other community >>>>>>>>> members are >>>>>>>>> > >>>>>>> encouraged to cast >>>>>>>>> > >>>>>>> non-binding votes. This vote will pass if there are 3 >>>>>>>>> binding +1 >>>>>>>>> > >>>>>>> votes and more binding >>>>>>>>> > >>>>>>> +1 votes than -1 votes. >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>> >>>>>>>>> > >>>>>>>>> >>>>>>>>
