Hey Russell, Thanks for driving this. Talking about making the security scanner happy: https://github.com/RussellSpitzer/iceberg/pull/5
JB, let me know if you need any help with the Avro release! Kind regards, Fokko Op vr 16 mei 2025 om 19:56 schreef Driesprong, Fokko <fo...@driesprong.frl>: > Hey Russell, > > Thanks for driving this. Talking about making the security scanner happy: > https://github.com/RussellSpitzer/iceberg/pull/5 > > JB, let me know if you need any help with the Avro release! > > Kind regards, > Fokko > > Op vr 16 mei 2025 om 19:39 schreef Russell Spitzer < > russell.spit...@gmail.com>: > >> Steven explained the Flink issue to me, Flink 2.0 isn't in 1.9.0 so not >> an issue. >> >> On Fri, May 16, 2025 at 12:20 PM Russell Spitzer < >> russell.spit...@gmail.com> wrote: >> >>> Ok so far of the lists proposed above I only picked 2 fixes that apply >>> cleanly and (we double checked) >>> actually apply to 1.9.0. Some of the fixes above need other commits >>> which aren't in 1.9.0 so aren't an >>> issue. If anyone else has any other issues let me know. >>> >>> The only one i'm not sure about is the Flink 2.0 Lock code, if someone >>> with flink expertise can ping me I >>> would appreciate it since I can't figure out how the patch applies to >>> 1.9.0. >>> >>> https://github.com/apache/iceberg/pull/13081 >>> >>> On Tue, May 13, 2025 at 11:04 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>> >>>> The new Avro release will content security improvement (and only this). >>>> So even if not strictly required (as iceberg is not impacted), it would be >>>> interesting to have security scanner happy ;) >>>> >>>> Regards >>>> JB >>>> >>>> Le mar. 13 mai 2025 à 23:24, Péter Váry <peter.vary.apa...@gmail.com> >>>> a écrit : >>>> >>>>> Do we really want to include a new lib version in a maintenance >>>>> release? In the past, we have seen issues when upgrading libs. Avro is >>>>> very >>>>> important, as it is used for metadata files. I would rather not include a >>>>> new version, unless it is absolutely necessary. >>>>> >>>>> On Tue, May 13, 2025, 06:42 Jean-Baptiste Onofré <j...@nanthrax.net> >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> I did a fix/improvement on Avro. I will propose to do new Avro >>>>>> releases. >>>>>> Maybe worth to include in Iceberg 1.9.1 if the timing is ok. >>>>>> >>>>>> Regards >>>>>> JB >>>>>> >>>>>> Le lun. 12 mai 2025 à 20:03, Russell Spitzer < >>>>>> russell.spit...@gmail.com> a écrit : >>>>>> >>>>>>> I'd rather we didn't get any "feature" sorts of things in like >>>>>>> * Enable HTTP proxy support for the client used by REST Catalog >>>>>>> #12406 <https://github.com/apache/iceberg/pull/12406> >>>>>>> * GCP: Support multiple storage credential prefixes #12881 >>>>>>> <https://github.com/apache/iceberg/pull/12881> >>>>>>> >>>>>>> These seem like refactors (no-ops for end users) >>>>>>> * Flink: Fix typo in JdbcLockFactory #12940 >>>>>>> <https://github.com/apache/iceberg/pull/12940> >>>>>>> * Flink: Change Preconditions import from flink util to guava #12939 >>>>>>> <https://github.com/apache/iceberg/pull/12939> >>>>>>> >>>>>>> >>>>>>> I think these are real bugs we should fix : >>>>>>> * Core: Ensure reactivated view version uses correct timestamp >>>>>>> #12821 <https://github.com/apache/iceberg/pull/12821> >>>>>>> * Flink: Add lockFactory open in LockRemover for table maintenance >>>>>>> #12900 <https://github.com/apache/iceberg/pull/12900> >>>>>>> >>>>>>> Low Priority : >>>>>>> * Core: Broaden exception handling in writer clean up logic #12863 >>>>>>> <https://github.com/apache/iceberg/pull/12863> >>>>>>> * Core: Disallow creation of invalid PartitionSpec #12887 >>>>>>> <https://github.com/apache/iceberg/pull/12887> >>>>>>> * Core: Fix Kryo ser/de with StorageCredential config #12882 >>>>>>> <https://github.com/apache/iceberg/pull/12882> >>>>>>> * Build, Core: Let RevAPI compare against 1.9.0 / Fix API breakage >>>>>>> around StorageCredential #12930 >>>>>>> <https://github.com/apache/iceberg/pull/12930> // Is this a >>>>>>> dangerous time to change this? We are only doing a point release >>>>>>> >>>>>>> >>>>>>> The goal should be to just get in bug fixes for 1.9.0 >>>>>>> >>>>>>> For me the priority goes >>>>>>> Highest - >>>>>>> Regressions - anything breaking the previous release (1.8.x) >>>>>>> Serious Bug Fixes - Correctness issues or major performance bugs >>>>>>> Minor Bug Fixes - Typos/ build things >>>>>>> --- Red Line >>>>>>> New Functionality / parameters ect >>>>>>> Lowest >>>>>>> >>>>>>> So I would avoid any "nice-to-have" items if we can and minimize the >>>>>>> changeset. >>>>>>> >>>>>>> On Mon, May 12, 2025 at 12:22 PM Yufei Gu <flyrain...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks Kevin for the list! That looks good to me. >>>>>>>> Looking forward to getting these fixes out! >>>>>>>> >>>>>>>> Yufei >>>>>>>> >>>>>>>> >>>>>>>> On Mon, May 12, 2025 at 10:19 AM Kevin Liu <kevinjq...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Russell, >>>>>>>>> >>>>>>>>> I went through the commits since 1.9.x release, >>>>>>>>> https://github.com/apache/iceberg/compare/1.9.x...main >>>>>>>>> >>>>>>>>> Here are some possible candidates for 1.9.1 patch release, >>>>>>>>> * Core: Fix Kryo ser/de with StorageCredential config #12882 >>>>>>>>> <https://github.com/apache/iceberg/pull/12882> >>>>>>>>> * Core: Ensure reactivated view version uses correct timestamp >>>>>>>>> #12821 <https://github.com/apache/iceberg/pull/12821> >>>>>>>>> * Flink: Add lockFactory open in LockRemover for table >>>>>>>>> maintenance #12900 <https://github.com/apache/iceberg/pull/12900> >>>>>>>>> * Flink: Fix typo in JdbcLockFactory #12940 >>>>>>>>> <https://github.com/apache/iceberg/pull/12940> >>>>>>>>> * Flink: Change Preconditions import from flink util to guava >>>>>>>>> #12939 <https://github.com/apache/iceberg/pull/12939> >>>>>>>>> * Core: Broaden exception handling in writer clean up logic #12863 >>>>>>>>> <https://github.com/apache/iceberg/pull/12863> >>>>>>>>> * Enable HTTP proxy support for the client used by REST Catalog >>>>>>>>> #12406 <https://github.com/apache/iceberg/pull/12406> >>>>>>>>> * Core: Disallow creation of invalid PartitionSpec #12887 >>>>>>>>> <https://github.com/apache/iceberg/pull/12887> >>>>>>>>> * GCP: Support multiple storage credential prefixes #12881 >>>>>>>>> <https://github.com/apache/iceberg/pull/12881> >>>>>>>>> * Build, Core: Let RevAPI compare against 1.9.0 / Fix API >>>>>>>>> breakage around StorageCredential #12930 >>>>>>>>> <https://github.com/apache/iceberg/pull/12930> >>>>>>>>> >>>>>>>>> IMO most of these are "nice to have" as part of 1.9.1. >>>>>>>>> Let me know what you think! >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Kevin Liu >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, May 12, 2025 at 9:41 AM Russell Spitzer < >>>>>>>>> russell.spit...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> I haven't gotten any other issues for 1.9.1 on the milestone and >>>>>>>>>> no one has responded here. >>>>>>>>>> I think it's important that we get a version of Iceberg out with >>>>>>>>>> a working Version function >>>>>>>>>> so I'll start a release today or tomorrow for a vote. >>>>>>>>>> >>>>>>>>>> On Sat, May 3, 2025 at 1:22 AM Jean-Baptiste Onofré < >>>>>>>>>> j...@nanthrax.net> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Russ >>>>>>>>>>> >>>>>>>>>>> Yes, agree. Your PR is good and already merged. >>>>>>>>>>> >>>>>>>>>>> I don't have anything blocker for 1.9.1 (still working on >>>>>>>>>>> source-ids, >>>>>>>>>>> but definitely not for 1.9.1). >>>>>>>>>>> >>>>>>>>>>> Thanks ! >>>>>>>>>>> Regards >>>>>>>>>>> JB >>>>>>>>>>> >>>>>>>>>>> On Fri, May 2, 2025 at 11:47 PM Russell Spitzer >>>>>>>>>>> <russell.spit...@gmail.com> wrote: >>>>>>>>>>> > >>>>>>>>>>> > Hey y'all! >>>>>>>>>>> > >>>>>>>>>>> > Thanks to @suilis we have learned that IcebergBuild.version() >>>>>>>>>>> is returning unspecified for Iceberg 1.9.0. I have a PR up >>>>>>>>>>> > to fix this and I think this is a clear reason to do a 1.9.1 >>>>>>>>>>> as soon as possible. I know we have a few other issues that >>>>>>>>>>> > may need to be fixed as well so let's make sure we get all >>>>>>>>>>> those listed and I can do a release when they are ready. >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > Please respond if you have any concerns or you have any issues >>>>>>>>>>> that need to go into a 1.9.1, >>>>>>>>>>> > Russ >>>>>>>>>>> >>>>>>>>>>
