Attila Doroszlai created HDDS-10630:
---------------------------------------
Summary: S3A: parent directory not found during CompleteMPU
request in FSO bucket
Key: HDDS-10630
URL: https://issues.apache.org/jira/browse/HDDS-10630
Project: Apache Ozone
Issue Type: Sub-task
Components: S3
Reporter: Saketa Chalamchala
{{ITestS3ACommitterMRJob}} fails on FSO bucket.
S3 Gateway audit log:
{code}
2024-03-22 18:05:34,785 | INFO | OMAudit | user=s3acontract | ip=192.168.192.3
| op=CREATE_DIRECTORY {volume=s3v, bucket=fso-bucket,
key=test/ITestS3ACommitterMRJob-execute-directory/, dataSize=0} | ret=SUCCESS |
2024-03-22 18:05:40,117 | INFO | OMAudit | user=s3acontract | ip=192.168.192.3
| op=DELETE_KEY {volume=s3v, bucket=fso-bucket,
key=test/ITestS3ACommitterMRJob-execute-directory, dataSize=0} | ret=SUCCESS |
2024-03-22 18:05:40,160 | ERROR | OMAudit | user=s3acontract | ip=192.168.192.3
| op=COMPLETE_MULTIPART_UPLOAD {volume=s3v, bucket=fso-bucket,
key=test/ITestS3ACommitterMRJob-execute-directory/part-m-00000, dataSize=0,
multipartList=[partNumber: 1 partName: "f03627c876605452a26379320675064e" eTag:
"f03627c876605452a26379320675064e" ]} | ret=FAILURE | DIRECTORY_NOT_FOUND
org.apache.hadoop.ozone.om.exceptions.OMException: Failed to find parent
directory of test/ITestS3ACommitterMRJob-execute-directory/part-m-00000
{code}
S3A uses multipart upload to commit job results in a batch. The staging
committer's {{replace}} mode deletes the target directory before completing
MPU. The problem is that FSO creates intermediate dirs during "initiate MPU"
request. Thus the "complete MPU" request finds directory to be non-existent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
