ctrlaltdilj commented on code in PR #27026:
URL: https://github.com/apache/flink/pull/27026#discussion_r3952125549
##########
flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImpl.java:
##########
@@ -322,14 +323,16 @@ private static class UploadTask implements Runnable {
@Override
public void run() {
try {
- final UploadPartResult result =
+ final UploadPartResponse result =
s3AccessHelper.uploadPart(
objectName,
uploadId,
partNumber,
file.getInputFile(),
file.getPos());
- future.complete(new PartETag(result.getPartNumber(),
result.getETag()));
+ final CompletedPart completedPart =
+
CompletedPart.builder().partNumber(partNumber).eTag(result.eTag()).build();
+ future.complete(completedPart);
Review Comment:
thanks for calling that out, I fixed it by copying every checksum field from
the UploadPartResponse into the CompletedPart (centralized in a new
S3PartChecksum enum so the set can't drift from the SDK). That also required
bumping S3RecoverableSerializer to v2, recoverable persisted at checkpoints
only stored partNumber and eTag, so a commit after recovery would have failed
the same way. v1 state deserializes as before (it predates checksum generation,
so checksum-less is correct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
