Bablu Raul created HDDS-17649:
---------------------------------
Summary: UploadPartCopy Does Not Enforce
x-amz-copy-source-if-match and x-amz-copy-source-if-none-match Preconditions
Key: HDDS-17649
URL: https://issues.apache.org/jira/browse/HDDS-17649
Project: Apache Ozone
Issue Type: Bug
Components: S3
Reporter: Bablu Raul
The Ozone S3 Gateway does not correctly enforce the ETag-based conditional
headers during the UploadPartCopy operation.
When UploadPartCopy is executed with a non-matching ETag using
{{{}x-amz-copy-source-if-match{}}}, the request succeeds and the part is copied
instead of returning HTTP 412 (PreconditionFailed).
Similarly, when {{x-amz-copy-source-if-none-match}} is supplied with an ETag
matching the current source object, the request succeeds instead of returning
HTTP 412 (PreconditionFailed).
This behavior is inconsistent with CopyObject, which correctly enforces the
source-side ETag conditions.
The issue was also reproduced after overwriting the source object.
UploadPartCopy using the old source ETag still succeeds and copies the updated
source object's content.
h2. Steps to Reproduce
# Authenticate using Kerberos
# Configure the S3 endpoint, credentials, and AWS CLI environment.
# Create a bucket and upload a 6 MiB source object:
{{export TS=$(date +%Y%m%d-%H%M%S)
export BKT=cpif-$TS SRC=src-$TS DST=dst-$TS
A s3api create-bucket --bucket "$BKT"
head -c 6291456 /dev/urandom > /tmp/cpif_v1_$TS.bin
A s3api put-object \
--bucket "$BKT" \
--key "$SRC" \
--body /tmp/cpif_v1_$TS.bin
export ETAG1=$(A s3api head-object \
--bucket "$BKT" \
--key "$SRC" \
--query ETag --output text)
export BAD='"00000000000000000000000000000000"'}}
# Initiate a multipart upload:
{{export UPID=$(A s3api create-multipart-upload \
--bucket "$BKT" \
--key "$DST" \
--query UploadId --output text)}}
# Execute UploadPartCopy with a non-matching If-Match ETag:
{{A s3api upload-part-copy \
--bucket "$BKT" \
--key "$DST" \
--upload-id "$UPID" \
--part-number 1 \
--copy-source "$BKT/$SRC" \
--copy-source-if-match "$BAD"}}
The request succeeds and returns CopyPartResult instead of PreconditionFailed.
# Execute UploadPartCopy with If-None-Match using the current source object's
ETag:
{{A s3api upload-part-copy \
--bucket "$BKT" \
--key "$DST" \
--upload-id "$UPID" \
--part-number 2 \
--copy-source "$BKT/$SRC" \
--copy-source-if-none-match "$ETAG1"}}
The request also succeeds, even though the supplied ETag matches the current
source object.
# Verify the control behavior by executing CopyObject with an invalid If-Match
ETag:
{{A s3api copy-object \
--bucket "$BKT" \
--key "ctl-$TS" \
--copy-source "$BKT/$SRC" \
--copy-source-if-match "$BAD"}}
CopyObject correctly returns PreconditionFailed.
# Verify that UploadPartCopy enforces the date-based condition by supplying an
old date:
{{A s3api upload-part-copy \
--bucket "$BKT" \
--key "$DST" \
--upload-id "$UPID" \
--part-number 3 \
--copy-source "$BKT/$SRC" \
--copy-source-if-unmodified-since "2000-01-01T00:00:00Z"}}
The request correctly returns PreconditionFailed.
# Overwrite the source object with new content and retrieve its updated ETag:
{{head -c 6291456 /dev/urandom > /tmp/cpif_v2_$TS.bin
A s3api put-object \
--bucket "$BKT" \
--key "$SRC" \
--body /tmp/cpif_v2_$TS.bin
export ETAG2=$(A s3api head-object \
--bucket "$BKT" \
--key "$SRC" \
--query ETag --output text)
echo "ETAG1=$ETAG1 ETAG2=$ETAG2"}}
# Initiate another multipart upload and execute UploadPartCopy using the old
ETag:
{{export UP2=$(A s3api create-multipart-upload \
--bucket "$BKT" \
--key "stale-$TS" \
--query UploadId --output text)
export PET=$(A s3api upload-part-copy \
--bucket "$BKT" \
--key "stale-$TS" \
--upload-id "$UP2" \
--part-number 1 \
--copy-source "$BKT/$SRC" \
--copy-source-if-match "$ETAG1" \
--query CopyPartResult.ETag \
--output text)}}
The request succeeds even though the source object's ETag has changed.
# Complete the multipart upload and download the resulting object:
{{A s3api complete-multipart-upload \
--bucket "$BKT" \
--key "stale-$TS" \
--upload-id "$UP2" \
--multipart-upload "\{\"Parts\":[{\"PartNumber\":1,\"ETag\":$PET}]}"
A s3api get-object \
--bucket "$BKT" \
--key "stale-$TS" \
/tmp/cpif_out_$TS.bin
md5sum /tmp/cpif_v1_$TS.bin \
/tmp/cpif_v2_$TS.bin \
/tmp/cpif_out_$TS.bin}}
h2. Actual Result
* UploadPartCopy succeeds when {{x-amz-copy-source-if-match}} contains a
non-matching ETag.
* UploadPartCopy succeeds when {{x-amz-copy-source-if-none-match}} contains an
ETag matching the current source object.
* UploadPartCopy succeeds with an old ETag after the source object has been
overwritten.
* The completed destination object contains the updated source object's
content (V2), instead of rejecting the request based on the old ETag.
* CopyObject correctly enforces the If-Match condition.
* UploadPartCopy correctly enforces the If-Unmodified-Since condition.
h2. Expected Result
UploadPartCopy should enforce the source-side ETag conditional headers
consistently with CopyObject.
* If {{x-amz-copy-source-if-match}} does not match the source object's ETag,
the request should fail with HTTP 412 (PreconditionFailed).
* If {{x-amz-copy-source-if-none-match}} matches the source object's ETag, the
request should fail with HTTP 412 (PreconditionFailed).
* If the source object is overwritten after the client retrieves its ETag,
UploadPartCopy using the old ETag should fail with HTTP 412
(PreconditionFailed).
* No part should be created when the source-side precondition fails.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
