moomindani commented on PR #17179: URL: https://github.com/apache/iceberg/pull/17179#issuecomment-5465448841
@singhpk234 first, a correction to what I wrote here on 7 August: I said this PR stacks on #17177 and that it made sense to review that one first. That is not true — this branch comes off `main`, shares no commits with #17177, and stands on its own. I am sorry for parking it behind a dependency that does not exist. On its own terms: `s3.checksum-algorithm` lets users choose the algorithm the SDK uses for upload integrity protection (`CRC32`, `CRC32C`, `CRC64NVME`, `SHA1`, `SHA256`, plus the `MD5` / `SHA512` / `XXHASH3` values S3 added in April 2026), applied per request in `S3OutputStream` for PutObject, CreateMultipartUpload and UploadPart. Part checksums from `UploadPart` are carried into `CompletedPart`; without that, CompleteMultipartUpload fails with `InvalidPart` once an algorithm is set. Verified against MinIO and real S3 with `CRC64NVME` and `SHA256`, single-part and multipart. Fixes #17178, CI green. One open naming question, also in the PR body: `s3.checksum-algorithm`, sitting next to the existing `s3.checksum-enabled` — versus `s3.write.checksum-algorithm`, matching `s3.write.storage-class`, since this only affects uploads. Hadoop S3A calls the equivalent `fs.s3a.create.checksum.algorithm`. No strong preference on my side; I will rename if you have one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
