adoroszlai opened a new pull request, #6519:
URL: https://github.com/apache/ozone/pull/6519
## What changes were proposed in this pull request?
Fix test failures related to in S3A contract test. The problem is that some
parts of the code use "modification time" instead of content hash as ETag.
```
Tests run: 11, Failures: 0, Errors: 3, Skipped: 1, Time elapsed: 2.998 s <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
testRenamePopulatesFileAncestors2(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename)
Time elapsed: 1.04 s <<< ERROR!
org.apache.hadoop.fs.s3a.RemoteFileChangedException: copy
`s3a://obs-bucket/test/testRenamePopulatesFileAncestors2/src/dir1/dir2/dir3/fileA':
ETag change detected on client during copy. Expected 2024-03-29T18:34:05.086Z
got f41ee8006d26bd7fc73828b0a0b350a8
testRenameWithNonEmptySubDir(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename)
Time elapsed: 0.281 s <<< ERROR!
org.apache.hadoop.fs.s3a.RemoteFileChangedException: copy
`s3a://obs-bucket/test/testRenameWithNonEmptySubDir/src1/source.txt': ETag
change detected on client during copy. Expected 2024-03-29T18:34:05.352Z got
046d1988b7a01316bb47905ccf3b584a
testRenamePopulatesFileAncestors(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename)
Time elapsed: 0.219 s <<< ERROR!
org.apache.hadoop.fs.s3a.RemoteFileChangedException: copy
`s3a://obs-bucket/test/testRenamePopulatesFileAncestors/source/dir1/dir2/dir3/file4':
ETag change detected on client during copy. Expected 2024-03-29T18:34:05.609Z
got 67e95334b751fbfc6570e8228183c074
Tests run: 4, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 1.314 s <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag
testLocatedStatusAlsoHasEtag(org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag)
Time elapsed: 0.117 s <<< FAILURE!
org.junit.ComparisonFailure: [etag of listLocatedStatus
(S3ALocatedFileStatus{path=s3a://obs-bucket/test/testLocatedStatusAlsoHasEtag/src;
isDirectory=false; length=11; replication=1; blocksize=33554432;
modification_time=1711737272164; access_time=0; owner=...; group=...;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true;
isErasureCoded=false}[eTag='2024-03-29T18:34:32.164Z', versionId='']) compared
to HEAD value of
S3AFileStatus{path=s3a://obs-bucket/test/testLocatedStatusAlsoHasEtag/src;
isDirectory=false; length=11; replication=1; blocksize=33554432;
modification_time=1711737272000; access_time=0; owner=...; group=...;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true;
isErasureCoded=false} isEmptyDirectory=FALSE
eTag=26e078b87fdaa3206ab8bf63a6096c07 versionId=null]
expected:<"2[6e078b87fdaa3206ab8bf63a6096c07]"> but
was:<"2[024-03-29T18:34:32.164Z]">
```
https://issues.apache.org/jira/browse/HDDS-10615
## How was this patch tested?
`ITestS3AContractEtag` is enabled as part of the change.
`ITestS3AContractRename` is still failing with another error for FSO buckets
only (HDDS-10665), but the ones related to ETag change are fixed. The test is
passing with OBS bucket:
```
Tests run: 11, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 5.289 s -
in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
```
https://github.com/adoroszlai/ozone/actions/runs/8661440172
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
