> But that #15470 is still stabilising, and last minute fixes are always dangerous as devs don't get enough time to play with them. It makes for a dangerous last-minute patch
+1 I don't think we should wait for this. But i'll defer to the RM to make the call On Thu, May 14, 2026 at 9:41 AM Aihua Xu <[email protected]> wrote: > We got the final Spark backport, #16303 > <https://github.com/apache/iceberg/pull/16303>, merged late last night. > Thanks to everyone who helped make that happen. > At this point, there are no remaining 1.11.0 blockers listed in the > milestone <https://github.com/apache/iceberg/milestone/59> . > > Thanks, Steve, for raising the SerializableFileIOWithSize concern—it's a > valid point from a stabilization perspective to introduce #15470. As I > understand from the conversion (please chime in if you have more context), > it's not considered a regression and I think we should move forward and cut > the next release candidate. Please let me know your thoughts. > > > On Thu, May 14, 2026 at 6:08 AM Steve Loughran <[email protected]> > wrote: > >> >> fixing SerializableFileIOWithSize, >> https://github.com/apache/iceberg/pull/16284, makes that file length >> more of an issue, especially with the GCS file io, which uses the supplied >> file length to limit its range. >> >> But that #15470 is still stabilising, and last minute fixes are always >> dangerous as devs don't get enough time to play with them. It makes for a >> dangerous last-minute patch >> >> On Wed, 13 May 2026 at 05:46, huaxin gao <[email protected]> wrote: >> >>> This is a correctness issue, but it is not a new regression; the same >>> issue already exists in 1.10. The PR description lists Trino, Impala, >>> Comet, and iceberg-rust as affected readers. I wouldn’t call it a hard >>> blocker for 1.11.0 because it is a pre-existing problem. >>> >>> Thanks, >>> Huaxin >>> >>> On Tue, May 12, 2026 at 8:44 PM Aihua Xu <[email protected]> wrote: >>> >>>> Huaxin, how critical is this one? The bug has been reported for a while >>>> and it's still being worked on. Can you add to the milestone 1.11.0 if it's >>>> a blocker so we can track? >>>> >>>> One more PR to consider: https://github.com/apache/iceberg/pull/15470 fixes >>>>> a correctness issue in rewriteTablePath, where manifests can record >>>>> stale file_size_in_bytes values for rewritten position delete files. >>>>> We probably want to include this fix in 1.11 too. >>>> >>>> >>>> On Tue, May 12, 2026 at 7:33 PM Aihua Xu <[email protected]> wrote: >>>> >>>>> If it’s needed, I will wait for it. Thanks for working on them. >>>>> >>>>> On May 12, 2026, at 7:17 PM, Kevin Liu <[email protected]> wrote: >>>>> >>>>> >>>>> 3 of the PRs have been merged. Thank you Huaxin for the review. I >>>>> merged it since it was mostly clean backports and only targets spark 3.4. >>>>> The last PR is pending CI and also a clean backport, >>>>> https://github.com/apache/iceberg/pull/16311 >>>>> >>>>> >>>>> >>>>> On Tue, May 12, 2026 at 6:52 PM Kevin Liu <[email protected]> >>>>> wrote: >>>>> >>>>>> I'll take a look at https://github.com/apache/iceberg/pull/15470 >>>>>> >>>>>> Here's the Spark 3.4 PRs. I only backported PRs with relevant code >>>>>> changes: >>>>>> https://github.com/apache/iceberg/pull/16306 (Backport of #14483 + >>>>>> #14497) >>>>>> https://github.com/apache/iceberg/pull/16307 (Backport of #15683 + >>>>>> #16284) >>>>>> https://github.com/apache/iceberg/pull/16308 (Backport of #15832) >>>>>> https://github.com/apache/iceberg/pull/16311 (Backport of #15992) >>>>>> This one needs to rebase #16307 above >>>>>> >>>>>> They are mostly clean backports, some with minimal change. The first >>>>>> 3 already passed CI. >>>>>> I would like to have these in. But will defer to Aihua (RM) for the >>>>>> final call. >>>>>> >>>>>> Best, >>>>>> Kevin Liu >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, May 12, 2026 at 6:39 PM Manu Zhang <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Aihua, >>>>>>> >>>>>>> Since we plan to drop Spark 3.4 after 1.11.0, let's get the >>>>>>> back-port PRs in. Otherwise, it will be left in a broken state. >>>>>>> >>>>>>> Thanks, >>>>>>> Manu >>>>>>> >>>>>>> On Wed, May 13, 2026 at 9:16 AM Aihua Xu <[email protected]> wrote: >>>>>>> >>>>>>>> Thanks everyone for driving these blockers to closure. >>>>>>>> >>>>>>>> Kevin, since this isn’t blocking and Spark 3.4 is deprecated, I’d >>>>>>>> like to go ahead and cut the next release candidate tonight so we can >>>>>>>> move >>>>>>>> forward—unless anyone disagrees. If we end up needing another RC, we >>>>>>>> can >>>>>>>> consider adding them in. What do you think? >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 12, 2026 at 4:48 PM Kevin Liu <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Ajay's email was stuck in webmod, i just unblocked it. >>>>>>>>> >>>>>>>>> Looks like all the issues in this email chain have been resolved. >>>>>>>>> - first row ID https://github.com/apache/iceberg/pull/16263 >>>>>>>>> - analyticscore https://github.com/apache/iceberg/pull/16258 >>>>>>>>> - SerializableFileIOWithSize >>>>>>>>> https://github.com/apache/iceberg/pull/16284 >>>>>>>>> >>>>>>>>> Thanks everyone for contributing to the fix! >>>>>>>>> >>>>>>>>> The 1.11.0 milestone is 100% complete at this time, >>>>>>>>> https://github.com/apache/iceberg/milestone/59 >>>>>>>>> >>>>>>>>> One _last_ thing, I went over the potential feature parity gap >>>>>>>>> between the four different Spark versions we currently support. It >>>>>>>>> looks >>>>>>>>> like there are a couple of PRs that can be backported to Spark 3.4 but >>>>>>>>> haven't been. Since this is the last release that supports Spark 3.4, >>>>>>>>> I'd >>>>>>>>> like to backport them and close the parity gap. This is completely >>>>>>>>> optional >>>>>>>>> since we've already marked Spark 3.4 as deprecated, but I think it's >>>>>>>>> a good >>>>>>>>> gesture for its final release. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Kevin Liu >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, May 12, 2026 at 3:57 PM Ajay Yadav <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Everyone, >>>>>>>>>> >>>>>>>>>> I would like to report a performance regression we've identified >>>>>>>>>> in Spark queries on Iceberg tables stored in cloud storage (tested >>>>>>>>>> with >>>>>>>>>> GCS), which I believe should be addressed in the 1.11.0 release. >>>>>>>>>> >>>>>>>>>> Current SerializableFileIOWithSize drops file length, causing >>>>>>>>>> performance regression due to excessive metadata calls in Cloud >>>>>>>>>> Storage: >>>>>>>>>> https://github.com/apache/iceberg/ssues/16283. The fix overrides >>>>>>>>>> InputFile newInputFile(String path, long length) to preserve >>>>>>>>>> file length and avoid unwanted metadata calls >>>>>>>>>> https://github.com/apache/iceberg/pull/16284 >>>>>>>>>> >>>>>>>>>> On 2026/05/08 15:27:05 Péter Váry wrote: >>>>>>>>>> > Just to clarify: >>>>>>>>>> > >>>>>>>>>> > The following PRs are already merged to 1.11.0: >>>>>>>>>> > >>>>>>>>>> > - https://github.com/apache/iceberg/pull/14297 - Spark: >>>>>>>>>> Support writing >>>>>>>>>> > shredded variant in Iceberg-Spark >>>>>>>>>> > - https://github.com/apache/iceberg/pull/15512 - Spark: fix >>>>>>>>>> delete from >>>>>>>>>> > branch for canDeleteWhere where it does not resolve to the >>>>>>>>>> correct branch - >>>>>>>>>> > WAP fix >>>>>>>>>> > - https://github.com/apache/iceberg/pull/15475 - Flink: Add >>>>>>>>>> Nanosecond >>>>>>>>>> > Precision Support for Flink-Iceberg Integration >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > The missing ones are the ones backporting those to other engine >>>>>>>>>> versions: >>>>>>>>>> > >>>>>>>>>> > - For: 14297 <https://github.com/apache/iceberg/pull/14297>: >>>>>>>>>> > - 16241 <https://github.com/apache/iceberg/pull/16241> - >>>>>>>>>> Backport for >>>>>>>>>> > variant shredding in Spark 4.0 >>>>>>>>>> > - For: 15512 <https://github.com/apache/iceberg/pull/15512>: >>>>>>>>>> > - 16245 <https://github.com/apache/iceberg/pull/16245> - >>>>>>>>>> Spark: >>>>>>>>>> > backport PR #15512 to v3.4, v3.5, v4.0 for WAP branch >>>>>>>>>> delete fix >>>>>>>>>> > - For: 15475 <https://github.com/apache/iceberg/pull/15475>: >>>>>>>>>> > - #16183 <https://github.com/apache/iceberg/pull/16183>, >>>>>>>>>> #16239 >>>>>>>>>> > <https://github.com/apache/iceberg/pull/16239>, #16240 >>>>>>>>>> > <https://github.com/apache/iceberg/pull/16240> - >>>>>>>>>> Backport for Nano >>>>>>>>>> > timestamps for Flink 2.0/1.20 >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > So the PRs needed on 1.11.0 are: >>>>>>>>>> > https://github.com/apache/iceberg/pull/16241 >>>>>>>>>> > https://github.com/apache/iceberg/pull/16245 >>>>>>>>>> > https://github.com/apache/iceberg/pull/16183 >>>>>>>>>> > https://github.com/apache/iceberg/pull/16239 >>>>>>>>>> > https://github.com/apache/iceberg/pull/16240 >>>>>>>>>> > https://github.com/apache/iceberg/pull/16186 >>>>>>>>>> > >>>>>>>>>> > Aihua Xu <[email protected]> ezt írta (időpont: 2026. máj. 8., >>>>>>>>>> P, 17:13): >>>>>>>>>> > >>>>>>>>>> > > Thank you all for the feedback and for verifying the release >>>>>>>>>> candidate. >>>>>>>>>> > > Based on the issues identified above, we will include the >>>>>>>>>> following fixes >>>>>>>>>> > > and cut RC2 with a new vote: >>>>>>>>>> > > >>>>>>>>>> > > https://github.com/apache/iceberg/pull/14297 >>>>>>>>>> > > https://github.com/apache/iceberg/pull/15512 >>>>>>>>>> > > https://github.com/apache/iceberg/pull/15475 >>>>>>>>>> > > https://github.com/apache/iceberg/pull/16186 >>>>>>>>>> > > >>>>>>>>>> > > Please let me know if you have any questions or identified >>>>>>>>>> additional >>>>>>>>>> > > issues. >>>>>>>>>> > > >>>>>>>>>> > > Thanks, >>>>>>>>>> > > Aihua >>>>>>>>>> > > >>>>>>>>>> > > On Thu, May 7, 2026 at 10:09 PM Aihua Xu <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> > > >>>>>>>>>> > >> I also looked into this. There is a configuration >>>>>>>>>> > >> gcs.analytics-core.enabled to enable/disable GCS Analytics >>>>>>>>>> Core. The >>>>>>>>>> > >> current implementation always requires runtime dependency of >>>>>>>>>> GCS Analytics >>>>>>>>>> > >> Core even if the configuration is off. Ideally we can lazy >>>>>>>>>> load such >>>>>>>>>> > >> dependency so the dependency is only required when the >>>>>>>>>> feature is >>>>>>>>>> > >> explicitly enabled. But since GCP is likely to enable GCS >>>>>>>>>> Analytics Core by >>>>>>>>>> > >> default, I feel it's reasonable for downstream projects >>>>>>>>>> using non-bundle >>>>>>>>>> > >> jars to add this dependency. >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> > >> On Thu, May 7, 2026 at 6:54 PM Steven Wu <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> > >> >>>>>>>>>> > >>> Looked a little more. >>>>>>>>>> > >>> >>>>>>>>>> > >>> So Iceberg's cloud modules consistently use compileOnly for >>>>>>>>>> vendor SDKs >>>>>>>>>> > >>> and rely on either the bundle artifact or downstream >>>>>>>>>> coordination for >>>>>>>>>> > >>> runtime. So, both changes are expected for downstream >>>>>>>>>> consumers using the >>>>>>>>>> > >>> non-bundle jars. Maybe we don't need to change anything. >>>>>>>>>> > >>> >>>>>>>>>> > >>> iceberg-gcp module >>>>>>>>>> > >>> >>>>>>>>>> > >>> compileOnly platform(libs.google.libraries.bom) >>>>>>>>>> > >>> compileOnly "com.google.cloud:google-cloud-storage" >>>>>>>>>> > >>> compileOnly "com.google.cloud:google-cloud-kms" >>>>>>>>>> > >>> compileOnly(libs.gcs.analytics.core) >>>>>>>>>> > >>> >>>>>>>>>> > >>> >>>>>>>>>> > >>> On Thu, May 7, 2026 at 6:16 PM Steven Wu <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> > >>> >>>>>>>>>> > >>>> Yuya, thanks for reporting the discovery. >>>>>>>>>> > >>>> >>>>>>>>>> > >>>> Azure: I approved your PR and can merge it soon: >>>>>>>>>> > >>>> https://github.com/apache/iceberg/pull/16186 >>>>>>>>>> > >>>> GCP: the new dependency is marked as compileOnly in PR >>>>>>>>>> 14333 >>>>>>>>>> > >>>> <https://github.com/apache/iceberg/pull/14333>, as it is >>>>>>>>>> an opt-in >>>>>>>>>> > >>>> feature. we need to either change the dep to >>>>>>>>>> implementation or update the >>>>>>>>>> > >>>> code similar to the Azure fix above. >>>>>>>>>> > >>>> >>>>>>>>>> > >>>> >>>>>>>>>> > >>>> On Thu, May 7, 2026 at 4:07 PM Yuya Ebihara < >>>>>>>>>> > >>>> [email protected]> wrote: >>>>>>>>>> > >>>> >>>>>>>>>> > >>>>> Hi Aihua, >>>>>>>>>> > >>>>> >>>>>>>>>> > >>>>> Thanks for leading the release! >>>>>>>>>> > >>>>> >>>>>>>>>> > >>>>> Just a quick reminder about two dependency-related items >>>>>>>>>> from a >>>>>>>>>> > >>>>> downstream perspective: >>>>>>>>>> > >>>>> * Azure module users will require >>>>>>>>>> azure-security-keyvault-keys, even >>>>>>>>>> > >>>>> when table encryption is not used, as noted in >>>>>>>>>> > >>>>> https://github.com/apache/iceberg/pull/16186 >>>>>>>>>> > >>>>> * GCS module users will require gcs-analytics-core >>>>>>>>>> > >>>>> >>>>>>>>>> > >>>>> I ran into CI failures with 1.11.0 in Trino because the >>>>>>>>>> project does >>>>>>>>>> > >>>>> not use the azure-bundle or gcp-bundle modules. >>>>>>>>>> > >>>>> The CI passed once we explicitly added these two >>>>>>>>>> dependencies. >>>>>>>>>> > >>>>> >>>>>>>>>> > >>>>> Thanks, >>>>>>>>>> > >>>>> Yuya Ebihara >>>>>>>>>> > >>>>> >>>>>>>>>> > >>>>> On Fri, May 8, 2026 at 4:58 AM Péter Váry < >>>>>>>>>> [email protected]> >>>>>>>>>> > >>>>> wrote: >>>>>>>>>> > >>>>> >>>>>>>>>> > >>>>>> First of all, thanks to everyone for the effort put into >>>>>>>>>> preparing >>>>>>>>>> > >>>>>> this release! >>>>>>>>>> > >>>>>> >>>>>>>>>> > >>>>>> I would like to highlight that RC1 is built from a >>>>>>>>>> branch where the >>>>>>>>>> > >>>>>> following features have not been backported to all >>>>>>>>>> engine versions: >>>>>>>>>> > >>>>>> - Spark: Support writing shredded variant in >>>>>>>>>> Iceberg-Spark ( >>>>>>>>>> > >>>>>> https://github.com/apache/iceberg/pull/14297) - >>>>>>>>>> Available in Spark >>>>>>>>>> > >>>>>> 4.1, but not in Spark 4.0 >>>>>>>>>> > >>>>>> - Spark: fix delete from branch for canDeleteWhere where >>>>>>>>>> it does not >>>>>>>>>> > >>>>>> resolve to the correct branch ( >>>>>>>>>> > >>>>>> https://github.com/apache/iceberg/pull/15512) - >>>>>>>>>> Available in Spark >>>>>>>>>> > >>>>>> 4.1, but not in Spark 4.0, 3.5, or 3.4 >>>>>>>>>> > >>>>>> - Flink: Add Nanosecond Precision Support for >>>>>>>>>> Flink-Iceberg >>>>>>>>>> > >>>>>> Integration ( >>>>>>>>>> https://github.com/apache/iceberg/pull/15475) - >>>>>>>>>> > >>>>>> Available in Flink 2.1, but not in Flink 2.0 or 1.20 >>>>>>>>>> > >>>>>> >>>>>>>>>> > >>>>>> It is up to the community to decide whether these >>>>>>>>>> missing backports >>>>>>>>>> > >>>>>> should be considered release blockers. Most of the >>>>>>>>>> corresponding PRs have >>>>>>>>>> > >>>>>> already been merged to main (except #15512), and >>>>>>>>>> including them in the >>>>>>>>>> > >>>>>> release should be relatively straightforward. >>>>>>>>>> > >>>>>> >>>>>>>>>> > >>>>>> From my perspective, I would prefer not to release with >>>>>>>>>> these gaps. >>>>>>>>>> > >>>>>> That said, I understand the urgency and the need for a >>>>>>>>>> release, and I am >>>>>>>>>> > >>>>>> happy to go with the community’s decision. >>>>>>>>>> > >>>>>> >>>>>>>>>> > >>>>>> Peter >>>>>>>>>> > >>>>>> >>>>>>>>>> > >>>>>> Aihua Xu <[email protected]> ezt írta (időpont: 2026. >>>>>>>>>> máj. 7., Cs, >>>>>>>>>> > >>>>>> 18:26): >>>>>>>>>> > >>>>>> >>>>>>>>>> > >>>>>>> Hi Everyone, >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> I propose that we release the following RC as the >>>>>>>>>> official Apache >>>>>>>>>> > >>>>>>> Iceberg 1.11.0 release. >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> The commit ID is >>>>>>>>>> 0f657edf12dc29f8487a679bfdd4210e9588d014 >>>>>>>>>> > >>>>>>> * This corresponds to the tag: apache-iceberg-1.11.0-rc1 >>>>>>>>>> > >>>>>>> * >>>>>>>>>> > >>>>>>> >>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.11.0-rc1 >>>>>>>>>> > >>>>>>> * >>>>>>>>>> > >>>>>>> >>>>>>>>>> https://github.com/apache/iceberg/tree/0f657edf12dc29f8487a679bfdd4210e9588d014 >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> The release tarball, signature, and checksums are here: >>>>>>>>>> > >>>>>>> * >>>>>>>>>> > >>>>>>> >>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.11.0-rc1 >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> You can find the KEYS file here: >>>>>>>>>> > >>>>>>> * https://downloads.apache.org/iceberg/KEYS >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> Convenience binary artifacts are staged on Nexus. The >>>>>>>>>> Maven >>>>>>>>>> > >>>>>>> repository URL is: >>>>>>>>>> > >>>>>>> * >>>>>>>>>> > >>>>>>> >>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1278/ >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> Please download, verify, and test. >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> Instructions for verifying a release can be found here: >>>>>>>>>> > >>>>>>> * >>>>>>>>>> https://iceberg.apache.org/how-to-release/#how-to-verify-a-release >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> Please vote in the next 72 hours. >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> [ ] +1 Release this as Apache Iceberg 1.11.0 >>>>>>>>>> > >>>>>>> [ ] +0 >>>>>>>>>> > >>>>>>> [ ] -1 Do not release this because... >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> Only PMC members have binding votes, but other >>>>>>>>>> community members are >>>>>>>>>> > >>>>>>> encouraged to cast >>>>>>>>>> > >>>>>>> non-binding votes. This vote will pass if there are 3 >>>>>>>>>> binding +1 >>>>>>>>>> > >>>>>>> votes and more binding >>>>>>>>>> > >>>>>>> +1 votes than -1 votes. >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>
