Can I get this reviewed and merged; gives all hadoop filesystems with bulk delete calls the ability to issue bulk deletes up to their page sizes; off by default. Tested all the way through iceberg to AWS S3 london.
https://github.com/apache/iceberg/pull/10233 On Mon, 17 Mar 2025 at 12:32, Yuya Ebihara <yuya.ebih...@starburstdata.com> wrote: > Hi, can we include https://github.com/apache/iceberg/pull/12264 to fix > the S3-compatible storage issue? > We downgraded the problematic library in > https://github.com/apache/iceberg/pull/12339, but the issue is still > present in the main branch. > I'd go with downgrading the AWS SDK to 2.29. x and then get on the relevant AWS SDK discussion to express your concerns: https://github.com/aws/aws-sdk-java-v2/discussions/5802 The problem here is that there's such a broad set of implementations of the S3 API that it'll take testing to see how even the suggestions from the SDK team work everywhere -and we now have explicit confirmation that the SDK team leave all such testing to downstream users. *The AWS SDKs and CLI are designed for usage with official AWS services.We may introduce and enable new features by default, such as these new default integrity protections,prior to them being supported or otherwise handled by third-party service implementations.* I think ASF projects need to make clear how dangerous this is -that projects will end up shipping releases which don't work, and the "set an env var or a system property" workarounds are not enough. > > On Mon, Mar 17, 2025 at 8:47 PM Ajantha Bhat <ajanthab...@gmail.com> > wrote: > >> We decided to have a fast follow up on the things that missed release >> train of 1.8.0 during community sync. >> >> More details here: >> https://lists.apache.org/thread/wvz5sd7pmh5ww1yqhsxpt1kwf993276j >> >> On Mon, Mar 17, 2025 at 4:53 PM Russell Spitzer < >> russell.spit...@gmail.com> wrote: >> >>> Can you please rehash the plan? I thought we just did a release last >>> month and were aiming for a three month schedule. I may have missed >>> something >>> >>> On Mon, Mar 17, 2025 at 6:00 AM Ajantha Bhat <ajanthab...@gmail.com> >>> wrote: >>> >>>> Hey everyone, >>>> >>>> Following the plan from the 1.8.0 release, I'll be handling the 1.9.0 >>>> release. We have several major updates ready: >>>> >>>> - *Partition stats APIs:* All core APIs for partition statistics >>>> have been merged, unblocking engines like Dremio, Trino, and Hive that >>>> were >>>> waiting for this feature. >>>> - *REST catalog authentication:* The refactoring PR for the REST >>>> catalog authentication manager has been merged, improving authentication >>>> support. >>>> - *Spark 3.3 support removed:* We've officially dropped support for >>>> Spark 3.3. >>>> - *InternalData support in the core module:* The core module now >>>> supports InternalData, leveraging the internal Parquet and Avro readers >>>> added in the previous release. This allows metadata to be written in >>>> Parquet. >>>> - *Bug fixes:* Many important bug fixes have been merged. >>>> >>>> A 1.9.0 milestone has been created with additional "good-to-have" >>>> issues: >>>> https://github.com/apache/iceberg/milestone/53 >>>> >>>> If there's anything urgent that needs to be included for this release, >>>> please let me know or reply to this thread. >>>> I'm aiming to start the release cut by the end of this week. >>>> >>>> >>>> - Ajantha >>>> >>>
