Gargi-jais11 opened a new pull request, #11857: URL: https://github.com/apache/ozone/pull/11857
## What changes were proposed in this pull request? When **CompleteMultipartUpload** lists only some of the uploaded parts, OM builds the object from the listed parts but computes its ETag from all uploaded parts. For example, **with parts 1, 2 and 3 uploaded and the upload completed with `[1, 3]`:** - the object correctly contains parts 1 and 3 (10 MiB), - but the ETag is `md5(E1 + E2 + E3)-3` instead of `md5(E1 + E3)-2`, - so HeadObject reports `PartsCount: 3`, while GetObjectAttributes reports `TotalPartsCount: 2`. As a result, completions with different content (for example `[1]` and `[2]`) get the same ETag. AWS S3 builds the object only from the parts in the request, and the multipart ETag and its `-N` suffix describe those parts ([CompleteMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html), [multipart ETag](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html)). **Fix applied:** `S3MultipartUploadCompleteRequest.multipartUploadedKeyHash` iterated over all uploaded parts (`partKeyInfoMap`). It now iterates over the parts in the request, in request order. A completion that lists every uploaded part produces the same ETag as before. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17572 ## How was this patch tested? Added Unit and Integration tests. **Before Fix** ``` bash-5.1$ export ENDPOINT=http://s3g:9878/ A="aws --endpoint-url $ENDPOINT" export BKT=b1-repro-$RANDOM bash-5.1$ $A s3api create-bucket --bucket "$BKT" { "Location": "http://s3g:9878/b1-repro-9415"; } bash-5.1$ cd /tmp bash-5.1$ for n in 1 2 3; do head -c 5242880 /dev/urandom > b1_part$n.bin; done bash-5.1$ new_mpu3() { KEY="b1-$1-$(date +%s)" UPID=$($A s3api create-multipart-upload --bucket "$BKT" --key "$KEY" --query UploadId --output text) for n in 1 2 3; do E=$($A s3api upload-part --bucket "$BKT" --key "$KEY" --upload-id "$UPID" --part-number "$n" \ --body "b1_part$n.bin" --query ETag --output text | tr -d '"') eval "E$n=$E" done echo "KEY=$KEY UPID=$UPID E1=$E1 E2=$E2 E3=$E3" $A s3api list-parts --bucket "$BKT" --key "$KEY" --upload-id "$UPID" --query 'Parts[].[PartNumber,Size]' --output text } # usage: complete_check 1 3 (part numbers to complete with, ascending) complete_check() { } ozone sh key info "/s3v/$BKT/$KEY" | grep -E '"dataSize"|"ETag"' -f1)"/null | md5sum | cut -d' ' -f1)-3" bash-5.1$ complete_check() { PARTS=""; CAT=""; FILES="" for n in "$@"; do eval "e=\$E$n" PARTS="$PARTS{\"PartNumber\":$n,\"ETag\":\"\\\"$e\\\"\"}," CAT="$CAT$e"; FILES="$FILES b1_part$n.bin" done $A s3api complete-multipart-upload --bucket "$BKT" --key "$KEY" --upload-id "$UPID" \ --multipart-upload "{\"Parts\":[${PARTS%,}]}" --query ETag --output text echo "Expected ETag (requested parts) : $(printf '%s' "$CAT" | md5sum | cut -d' ' -f1)-$#" echo "ETag over all uploaded [1,2,3] : $(printf '%s%s%s' "$E1" "$E2" "$E3" | md5sum | cut -d' ' -f1)-3" $A s3api head-object --bucket "$BKT" --key "$KEY" \ --query '{ContentLength:ContentLength,ETag:ETag,PartsCount:PartsCount}' $A s3api get-object --bucket "$BKT" --key "$KEY" /tmp/b1_out.bin >/dev/null } ozone sh key info "/s3v/$BKT/$KEY" | grep -E '"dataSize"|"ETag"' -f1)" # Control: all parts [1,2,3]. Expected and actual ETag match, PartsCount 3, 15 MiB. bash-5.1$ new_mpu3 all complete_check 1 2 3 KEY=b1-all-1793255659 UPID=01a1865d-d7eb-7333-8e5a-cf071b169248-117522802900992000 E1=0abdc2419630510a440832c49c061690 E2=2f69c7ac7eb232e1ba32bb5a23eb2242 E3=d29deee857e9e8ec7429a6697e96bd7f 1 5242880 2 5242880 3 5242880 "5adeba7cc7cc8c7a148d205b62b24399-3" Expected ETag (requested parts) : 5adeba7cc7cc8c7a148d205b62b24399-3 ETag over all uploaded [1,2,3] : 5adeba7cc7cc8c7a148d205b62b24399-3 { "ContentLength": 15728640, "ETag": "\"5adeba7cc7cc8c7a148d205b62b24399-3\"", "PartsCount": 3 } content md5 expected: 415ee7fee86938f2f0a183aeda8b018b content md5 actual : 415ee7fee86938f2f0a183aeda8b018b "dataSize" : 15728640, "ETag" : "5adeba7cc7cc8c7a148d205b62b24399-3" # Case 1 (the report): [1,3]. Content is correct (10 MiB, parts 1 and 3), # but the ETag is md5(E1E2E3)-3 instead of md5(E1E3)-2, and PartsCount is 3 instead of 2. bash-5.1$ new_mpu3 subset13 complete_check 1 3 KEY=b1-subset13-1793255699 UPID=01a1865e-73b7-7a2a-ad85-b522bae7b780-117522805514764292 E1=0abdc2419630510a440832c49c061690 E2=2f69c7ac7eb232e1ba32bb5a23eb2242 E3=d29deee857e9e8ec7429a6697e96bd7f 1 5242880 2 5242880 3 5242880 "5adeba7cc7cc8c7a148d205b62b24399-3" Expected ETag (requested parts) : 3b0fa8f2451023b1c14bb6bf43b040bd-2 ETag over all uploaded [1,2,3] : 5adeba7cc7cc8c7a148d205b62b24399-3 { "ContentLength": 10485760, "ETag": "\"5adeba7cc7cc8c7a148d205b62b24399-3\"", "PartsCount": 3 } content md5 expected: 6a2d69630ce7cd08db4a0fa2f1be88ef content md5 actual : 6a2d69630ce7cd08db4a0fa2f1be88ef "dataSize" : 10485760, "ETag" : "5adeba7cc7cc8c7a148d205b62b24399-3" # Case 2: [1] only. Object is 5 MiB, but the ETag still ends in -3 and PartsCount is 3. bash-5.1$ new_mpu3 only1 complete_check 1 KEY=b1-only1-1793255917 UPID=01a18661-c8e1-7270-be18-7daabf3c93db-117522819828482056 E1=0abdc2419630510a440832c49c061690 E2=2f69c7ac7eb232e1ba32bb5a23eb2242 E3=d29deee857e9e8ec7429a6697e96bd7f 1 5242880 2 5242880 3 5242880 "5adeba7cc7cc8c7a148d205b62b24399-3" Expected ETag (requested parts) : 00aaacc59a1aeb4f4780e430426a0aa3-1 ETag over all uploaded [1,2,3] : 5adeba7cc7cc8c7a148d205b62b24399-3 { "ContentLength": 5242880, "ETag": "\"5adeba7cc7cc8c7a148d205b62b24399-3\"", "PartsCount": 3 } content md5 expected: 0abdc2419630510a440832c49c061690 content md5 actual : 0abdc2419630510a440832c49c061690 "dataSize" : 5242880, "ETag" : "5adeba7cc7cc8c7a148d205b62b24399-3" # Case 3: [2] only. Different content from case 2, but the same kind of ETag (md5 over all three parts, -3). bash-5.1$ new_mpu3 only2 complete_check 2 KEY=b1-only2-1793255929 UPID=01a18661-f703-7b18-a05b-495c16d47dfe-117522820602462220 E1=0abdc2419630510a440832c49c061690 E2=2f69c7ac7eb232e1ba32bb5a23eb2242 E3=d29deee857e9e8ec7429a6697e96bd7f 1 5242880 2 5242880 3 5242880 "5adeba7cc7cc8c7a148d205b62b24399-3" Expected ETag (requested parts) : 75ce83b4e8d3eecbcfe977783b7677c0-1 ETag over all uploaded [1,2,3] : 5adeba7cc7cc8c7a148d205b62b24399-3 { "ContentLength": 5242880, "ETag": "\"5adeba7cc7cc8c7a148d205b62b24399-3\"", "PartsCount": 3 } content md5 expected: 2f69c7ac7eb232e1ba32bb5a23eb2242 content md5 actual : 2f69c7ac7eb232e1ba32bb5a23eb2242 "dataSize" : 5242880, "ETag" : "5adeba7cc7cc8c7a148d205b62b24399-3" # Extra (case 1 object): HeadObject's PartsCount (from the ETag) vs GetObjectAttributes (from the blocks). bash-5.1$ new_mpu3 attrs complete_check 1 3 KEY=b1-attrs-1793255941 UPID=01a18662-27f3-789b-a793-6bb1a4ffa6ff-117522821423497232 E1=0abdc2419630510a440832c49c061690 E2=2f69c7ac7eb232e1ba32bb5a23eb2242 E3=d29deee857e9e8ec7429a6697e96bd7f 1 5242880 2 5242880 3 5242880 "5adeba7cc7cc8c7a148d205b62b24399-3" Expected ETag (requested parts) : 3b0fa8f2451023b1c14bb6bf43b040bd-2 ETag over all uploaded [1,2,3] : 5adeba7cc7cc8c7a148d205b62b24399-3 { "ContentLength": 10485760, "ETag": "\"5adeba7cc7cc8c7a148d205b62b24399-3\"", "PartsCount": 3 } content md5 expected: 6a2d69630ce7cd08db4a0fa2f1be88ef content md5 actual : 6a2d69630ce7cd08db4a0fa2f1be88ef "dataSize" : 10485760, "ETag" : "5adeba7cc7cc8c7a148d205b62b24399-3" bash-5.1$ $A s3api head-object --bucket "$BKT" --key "$KEY" --query PartsCount $A s3api get-object-attributes --bucket "$BKT" --key "$KEY" --object-attributes ObjectParts ObjectSize 3 { "LastModified": "Thu, 29 Oct 2026 06:39:03 GMT", "ObjectParts": { "TotalPartsCount": 2, "MaxParts": 1000, "IsTruncated": false }, "ObjectSize": 10485760 } ``` **After Fix** ``` # Case 1: bash-5.1$ new_mpu3 all complete_check 1 2 3 KEY=b1-all-1793256684 UPID=01a1866d-7b4f-711f-b67a-cbf57cf907b6-117522870066675712 E1=38f4d7731c34f734163c0422fdf265f4 E2=cc0bdba89de20207e582226b6182e992 E3=e5574a9579cf901d83a8538f787c4068 1 5242880 2 5242880 3 5242880 "918e2b92368117f6046e04d778bc3e3b-3" Expected ETag (requested parts) : 918e2b92368117f6046e04d778bc3e3b-3 ETag over all uploaded [1,2,3] : 918e2b92368117f6046e04d778bc3e3b-3 { "ContentLength": 15728640, "ETag": "\"918e2b92368117f6046e04d778bc3e3b-3\"", "PartsCount": 3 } content md5 expected: d78d5411d79a99cbaad6072d0a30cdb2 content md5 actual : d78d5411d79a99cbaad6072d0a30cdb2 "dataSize" : 15728640, "ETag" : "918e2b92368117f6046e04d778bc3e3b-3" # Case 2: bash-5.1$ new_mpu3 subset13 complete_check 1 3 KEY=b1-subset13-1793256695 UPID=01a1866d-a7c6-73d6-98f7-ea76e4ec84b6-117522870812672004 E1=38f4d7731c34f734163c0422fdf265f4 E2=cc0bdba89de20207e582226b6182e992 E3=e5574a9579cf901d83a8538f787c4068 1 5242880 2 5242880 3 5242880 "e300145c552d6b8d32f0aefaa43ab888-2" Expected ETag (requested parts) : e300145c552d6b8d32f0aefaa43ab888-2 ETag over all uploaded [1,2,3] : 918e2b92368117f6046e04d778bc3e3b-3 { "ContentLength": 10485760, "ETag": "\"e300145c552d6b8d32f0aefaa43ab888-2\"", "PartsCount": 2 } content md5 expected: 9a47302713cf649d1f69fad4e414a6f6 content md5 actual : 9a47302713cf649d1f69fad4e414a6f6 "dataSize" : 10485760, "ETag" : "e300145c552d6b8d32f0aefaa43ab888-2" # Case 3: bash-5.1$ new_mpu3 only1 complete_check 1 KEY=b1-only1-1793256731 UPID=01a1866e-32e0-7b51-97ee-16da0d502236-117522873146408968 E1=38f4d7731c34f734163c0422fdf265f4 E2=cc0bdba89de20207e582226b6182e992 E3=e5574a9579cf901d83a8538f787c4068 1 5242880 2 5242880 3 5242880 "6c863c80de6d26deff13076b3b26ffcb-1" Expected ETag (requested parts) : 6c863c80de6d26deff13076b3b26ffcb-1 ETag over all uploaded [1,2,3] : 918e2b92368117f6046e04d778bc3e3b-3 { "ContentLength": 5242880, "ETag": "\"6c863c80de6d26deff13076b3b26ffcb-1\"", "PartsCount": 1 } content md5 expected: 38f4d7731c34f734163c0422fdf265f4 content md5 actual : 38f4d7731c34f734163c0422fdf265f4 "dataSize" : 5242880, "ETag" : "6c863c80de6d26deff13076b3b26ffcb-1" # Case 4: bash-5.1$ new_mpu3 only2 complete_check 2 KEY=b1-only2-1793256739 UPID=01a1866e-5343-72df-86f2-865d02b4a0cb-117522873689767948 E1=38f4d7731c34f734163c0422fdf265f4 E2=cc0bdba89de20207e582226b6182e992 E3=e5574a9579cf901d83a8538f787c4068 1 5242880 2 5242880 3 5242880 "c53855754fda23efb792f04323898450-1" Expected ETag (requested parts) : c53855754fda23efb792f04323898450-1 ETag over all uploaded [1,2,3] : 918e2b92368117f6046e04d778bc3e3b-3 { "ContentLength": 5242880, "ETag": "\"c53855754fda23efb792f04323898450-1\"", "PartsCount": 1 } content md5 expected: cc0bdba89de20207e582226b6182e992 content md5 actual : cc0bdba89de20207e582226b6182e992 "dataSize" : 5242880, "ETag" : "c53855754fda23efb792f04323898450-1" # Case 5: bash-5.1$ new_mpu3 attrs complete_check 1 3 KEY=b1-attrs-1793256749 UPID=01a1866e-7921-7587-9759-1c81c9eed35f-117522874325073936 E1=38f4d7731c34f734163c0422fdf265f4 E2=cc0bdba89de20207e582226b6182e992 E3=e5574a9579cf901d83a8538f787c4068 1 5242880 2 5242880 3 5242880 "e300145c552d6b8d32f0aefaa43ab888-2" Expected ETag (requested parts) : e300145c552d6b8d32f0aefaa43ab888-2 ETag over all uploaded [1,2,3] : 918e2b92368117f6046e04d778bc3e3b-3 { "ContentLength": 10485760, "ETag": "\"e300145c552d6b8d32f0aefaa43ab888-2\"", "PartsCount": 2 } content md5 expected: 9a47302713cf649d1f69fad4e414a6f6 content md5 actual : 9a47302713cf649d1f69fad4e414a6f6 "dataSize" : 10485760, "ETag" : "e300145c552d6b8d32f0aefaa43ab888-2" bash-5.1$ $A s3api head-object --bucket "$BKT" --key "$KEY" --query PartsCount 2 bash-5.1$ $A s3api get-object-attributes --bucket "$BKT" --key "$KEY" --object-attributes ObjectParts ObjectSize { "LastModified": "Thu, 29 Oct 2026 06:52:30 GMT", "ObjectParts": { "TotalPartsCount": 2, "MaxParts": 1000, "IsTruncated": false }, "ObjectSize": 10485760 } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
