steveloughran commented on PR #15077: URL: https://github.com/apache/iceberg/pull/15077#issuecomment-3781289410
I'm less worried about S3A than S3FileIO & third parties; remember how they broke third-party last year -and iceberg shipped with a broken release #12649. The AWS SDK team *do not test their SDK against third party stores*. This has been publicly stated. It's why the S3A SDK update policy requires that third party testing, with google cloud the easiest for most people, especially as it lacks some features (no bulk delete; list api is v1). https://github.com/steveloughran/engineering-proposals/blob/trunk/qualifying-an-SDK-upgrade.md Last update (https://github.com/apache/hadoop/pull/8059) we found two sdk regressions [6518](https://github.com/aws/aws-sdk-java-v2/issues/6518) and [6459](https://github.com/aws/aws-sdk-java-v2/issues/6459), the latter a performance regression with S3 express. So it's less "s3a is brittle" and more "test suite is brutal enough it'll find bugs in the SDK and even differences between aws stores" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
