diljotgrewal commented on PR #7015:
URL: https://github.com/apache/hadoop/pull/7015#issuecomment-2325536384
Testing against a Regular S3 bucket in `us-east-1`:
auth-keys
```
<configuration>
<property>
<name>test.fs.s3a.name</name>
<value>s3a://<REDACTED>/</value>
</property>
<property>
<name>fs.contract.test.fs.s3a</name>
<value>s3a://<REDACTED>/</value>
</property>
<property>
<name>fs.s3a.access.key</name>
<description>AWS access key ID. Omit for IAM role-based
authentication.</description>
<value><REDACTED></value>
</property>
<property>
<name>fs.s3a.secret.key</name>
<description>AWS secret key. Omit for IAM role-based
authentication.</description>
<value><REDACTED></value>
</property>
<property>
<name>fs.s3a.encryption.key</name>
<value>arn:aws:kms:us-east-1:<REDACTED>:key/<REDACTED></value>
</property>
<property>
<name>fs.s3a.assumed.role.arn</name>
<value>arn:aws:iam::<REDACTED>:role/<REDACTED></value>
</property>
<property>
<name>test.fs.s3a.sts.enabled</name>
<value>false</value>
</property>
</configuration>
```
#### Scale Tests
```
mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale
```
```
[ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 3, Time elapsed:
8.89 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AConfiguration
[ERROR] testRequestTimeout(org.apache.hadoop.fs.s3a.ITestS3AConfiguration)
Time elapsed: 0.029 s <<< ERROR!
java.lang.IllegalArgumentException: Cannot locate field clientConfiguration
on class
software.amazon.awssdk.services.s3.internal.crossregion.S3CrossRegionSyncClient
at org.apache.commons.lang3.Validate.isTrue(Validate.java:158)
at
org.apache.commons.lang3.reflect.FieldUtils.readField(FieldUtils.java:469)
at
org.apache.hadoop.fs.s3a.ITestS3AConfiguration.getField(ITestS3AConfiguration.java:535)
at
org.apache.hadoop.fs.s3a.ITestS3AConfiguration.testRequestTimeout(ITestS3AConfiguration.java:449)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
```
```
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
169.585 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps
[ERROR]
testThreadPoolCoolDown(org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps)
Time elapsed: 143.941 s <<< FAILURE!
java.lang.AssertionError: Failed to find threads in active FS - test is
flawed. Actual: 0
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failEquals(Assert.java:187)
at org.junit.Assert.assertNotEquals(Assert.java:201)
at
org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps.testThreadPoolCoolDown(ITestS3AConcurrentOps.java:206)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
```
```
[ERROR] Failures:
[ERROR]
ITestS3AConcurrentOps.testThreadPoolCoolDown:206->Assert.assertNotEquals:201->Assert.failEquals:187->Assert.fail:89
Failed to find threads in active FS - test is flawed. Actual: 0
[ERROR] Errors:
[ERROR] ITestS3AConfiguration.testRequestTimeout:449->getField:535 ยป
IllegalArgument C...
[INFO]
[ERROR] Tests run: 1327, Failures: 1, Errors: 1, Skipped: 127
```
#### adding access point:
```
<property>
<name>fs.s3a.bucket.sample-bucket.accesspoint.arn</name>
<value> <REDACTED> </value>
<description>Configure S3a traffic to use this AccessPoint</description>
</property>
```
```
mvn clean verify -Dit.test=ITest* -Dtest=none
```
Getting a lot of errors, currently debugging:
several parsing errors
```
grep "Failed to parse b-00" itest_log| sort | uniq -c
132 java.lang.Exception: Failed to parse b-00
```
```
[^[[1;31mERROR^[[m]
testStatistics(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics)
Time elapsed: 2.179 s <<< FAILURE!
java.lang.AssertionError: Mismatch in bytes written expected:<697> but
was:<512>
```
```
[^[[1;31mERROR^[[m]
testClosedInstrumentation(org.apache.hadoop.fs.s3a.ITestS3AClosedFS) Time
elapsed: 0.333 s <<< FAILURE!
org.junit.ComparisonFailure: [S3AInstrumentation.hasMetricSystem()]
expected:<[fals]e> but was:<[tru]e>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
```
few encryption errors:
```
[^[[1;31mERROR^[[m]
testEncryptionOverRename(org.apache.hadoop.fs.s3a.ITestS3AClientSideEncryptionKms)
Time elapsed: 2.502 s <<< FAILURE!
java.lang.AssertionError: Key wrap algo isn't same as expected
expected:<kms+context> but was:<null>
```
```
[^[[1;31mERROR^[[m]
testDirectoryListingFileLengths(org.apache.hadoop.fs.s3a.ITestS3AClientSideEncryptionKms)
Time elapsed: 3.498 s <<< FAILURE!
java.lang.AssertionError:
[File lengths aren't the same as expected from FileStatus dir. listing]
Expecting:
<[0, 1, 239, 4079]>
to contain exactly in any order:
<[0, 1, 255, 4095]>
elements not found:
<[255, 4095]>
and elements not expected:
<[239, 4079]>
```
```
[^[[1;31mERROR^[[m]
testBytesReadWithStream(org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic)
Time elapsed: 1.646 s <<< FAILURE!
java.lang.AssertionError: Mismatch in number of FS bytes read by
InputStreams expected:<2048> but was:<196167>
```
```
[^[[1;31mERROR^[[m]
testCustomSignerAndInitializer[simple-delete](org.apache.hadoop.fs.s3a.auth.ITestCustomSigner)
Time elapsed: 1.138 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSBadRequestException: getFileStatus on
s3a://s3atestingdsgrewal/job-00/test/testCustomSignerAndInitializer[simple-delete]/customsignerpath1:
software.amazon.awssdk.services.s3.model.S3Exception: The authorization
mechanism you have provided is not supported. Please use Signature Version 4.
(Service: S3, Status Code: 400, Request ID: Z68EWFD946XZRPW9, Extended Request
ID:
4oF/6V21FWJZdR5osr6DcNYC6RLxEQZrNxEf3/X8ur0Tm/aeVe31aNtZXT/VU/yrG17ZMUOAZK4=):InvalidRequest:
The authorization mechanism you have provided is not supported. Please use
Signature Version 4. (Service: S3, Status Code: 400, Request ID:
Z68EWFD946XZRPW9, Extended Request ID:
4oF/6V21FWJZdR5osr6DcNYC6RLxEQZrNxEf3/X8ur0Tm/aeVe31aNtZXT/VU/yrG17ZMUOAZK4=)
```
```
[^[[1;31mERROR^[[m]
testComplexDirActions(org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List)
Time elapsed: 16.345 s <<< FAILURE!
java.lang.AssertionError: getLen expected:<512> but was:<496>
```
```
[^[[1;31mERROR^[[m]
testRenamePopulatesFileAncestors2(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename)
Time elapsed: 2.337 s <<< FAILURE!
java.lang.AssertionError: Number of bytes copied: files_copied_bytes
expected:<16384> but was:<16368>
```
```
[^[[1;31mERROR^[[m] ^[[1;31m ITestS3AClosedFS.testClosedInstrumentation:111
[S3AInstrumentation.hasMetricSystem()] expected:<[fals]e> but was:<[tru]e>^[[m
at [Source: (org.apache.hadoop.fs.FSDataInputStream); line: 1, column:
4235] (through reference chain:
org.apache.hadoop.fs.s3a.commit.files.PendingSet["iostatistics"]->org.apache.hadoop.fs.statistics.IOStatisticsSnapshot["meanstatistics"])
```
#### ILoadTest
```mvn verify -Dtest=skip -Dit.test=ILoadTest* -Dscale > loadtest.txt```
```
[INFO] Running
org.apache.hadoop.fs.s3a.scale.ILoadTestS3ABulkDeleteThrottling
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
200.663 s - in org.apache.hadoop.fs.s3a.scale.ILoadTestS3ABulkDeleteThrottling
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ILoadTestSessionCredentials
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
2.692 s - in
org.apache.hadoop.fs.s3a.auth.delegation.ILoadTestSessionCredentials
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ILoadTestRoleCredentials
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
2.984 s - in org.apache.hadoop.fs.s3a.auth.delegation.ILoadTestRoleCredentials
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
