[
https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815506#comment-17815506
]
ASF GitHub Bot commented on HADOOP-19059:
-----------------------------------------
adnanhemani commented on PR #6538:
URL: https://github.com/apache/hadoop/pull/6538#issuecomment-1933326099
Integration and Unit Test results:
```
88665a0afd7e:hadoop-aws ahemani$ mvn verify -Dparallel-tests -Dprefetch
-DtestsThreadCount=8
Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true
[INFO] Scanning for projects...
[INFO]
------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO]
------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 14.2
[INFO] os.detected.version.major: 14
[INFO] os.detected.version.minor: 2
[INFO] os.detected.classifier: osx-x86_64
[INFO]
[INFO] --------------------< org.apache.hadoop:hadoop-aws
>--------------------
[INFO] Building Apache Hadoop Amazon Web Services support 3.5.0-SNAPSHOT
[INFO] --------------------------------[ jar
]---------------------------------
...
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.s3a.commit.TestMagicCommitPaths
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingDirectoryOutputCommitter
[INFO] Running org.apache.hadoop.fs.s3a.commit.staging.TestStagingCommitter
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedTaskCommit
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedJobCommit
[INFO] Running org.apache.hadoop.fs.s3a.commit.staging.TestPaths
[INFO] Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
1.954 s - in org.apache.hadoop.fs.s3a.commit.TestMagicCommitPaths
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
3.744 s - in org.apache.hadoop.fs.s3a.commit.staging.TestPaths
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestAwsClientConfig
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.083
s - in org.apache.hadoop.fs.s3a.impl.TestAwsClientConfig
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestHeaderProcessing
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestCreateFileBuilder
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
3.112 s - in org.apache.hadoop.fs.s3a.impl.TestHeaderProcessing
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.988
s - in org.apache.hadoop.fs.s3a.impl.TestCreateFileBuilder
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestErrorTranslation
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.945
s - in org.apache.hadoop.fs.s3a.impl.TestErrorTranslation
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestS3AMultipartUploaderSupport
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.851
s - in org.apache.hadoop.fs.s3a.impl.TestS3AMultipartUploaderSupport
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestRequestFactory
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
15.461 s - in
org.apache.hadoop.fs.s3a.commit.staging.TestStagingDirectoryOutputCommitter
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestDirectoryMarkerPolicy
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.202
s - in org.apache.hadoop.fs.s3a.impl.TestDirectoryMarkerPolicy
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.877
s - in org.apache.hadoop.fs.s3a.impl.TestRequestFactory
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestOpenFileSupport
[INFO] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
1.266 s - in org.apache.hadoop.fs.s3a.impl.TestOpenFileSupport
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
20.129 s - in
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestSDKStreamDrainer
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestNetworkBinding
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
20.905 s - in
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedJobCommit
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.994 s - in org.apache.hadoop.fs.s3a.impl.TestSDKStreamDrainer
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.806
s - in org.apache.hadoop.fs.s3a.impl.TestNetworkBinding
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestS3ExpressStorage
[INFO] Running org.apache.hadoop.fs.s3a.TestListing
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.617
s - in org.apache.hadoop.fs.s3a.impl.TestS3ExpressStorage
[INFO] Running org.apache.hadoop.fs.s3a.TestInvoker
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AExceptionTranslation
[INFO] Running org.apache.hadoop.fs.s3a.TestSSEConfiguration
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
24.573 s - in
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedTaskCommit
[INFO] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
1.784 s - in org.apache.hadoop.fs.s3a.TestS3AExceptionTranslation
[INFO] Running org.apache.hadoop.fs.s3a.TestWildflyAndOpenSSLBinding
[INFO] Running
org.apache.hadoop.fs.s3a.auth.TestIAMInstanceCredentialsProvider
[INFO] Running org.apache.hadoop.fs.s3a.auth.TestSignerManager
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
4.802 s - in org.apache.hadoop.fs.s3a.TestSSEConfiguration
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.974
s - in org.apache.hadoop.fs.s3a.auth.TestSignerManager
[INFO] Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
6.044 s - in org.apache.hadoop.fs.s3a.TestInvoker
[WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
4.218 s - in org.apache.hadoop.fs.s3a.TestWildflyAndOpenSSLBinding
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.306
s - in org.apache.hadoop.fs.s3a.auth.TestIAMInstanceCredentialsProvider
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.TestS3ADelegationTokenSupport
[INFO] Running org.apache.hadoop.fs.s3a.auth.TestMarshalledCredentials
[INFO] Running org.apache.hadoop.fs.s3a.TestBucketConfiguration
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AProxy
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.165
s - in org.apache.hadoop.fs.s3a.auth.TestMarshalledCredentials
[INFO] Running org.apache.hadoop.fs.s3a.adapter.TestV1CredentialsProvider
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.168
s - in org.apache.hadoop.fs.s3a.auth.delegation.TestS3ADelegationTokenSupport
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.978
s - in org.apache.hadoop.fs.s3a.TestS3AProxy
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.447 s - in org.apache.hadoop.fs.s3a.TestListing
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.904
s - in org.apache.hadoop.fs.s3a.adapter.TestV1CredentialsProvider
[INFO] Running org.apache.hadoop.fs.s3a.TestStreamChangeTracker
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AInputStreamRetry
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
1.003 s - in org.apache.hadoop.fs.s3a.TestStreamChangeTracker
[INFO] Running
org.apache.hadoop.fs.s3a.audit.impl.TestActiveAuditManagerThreadLeakage
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
4.327 s - in org.apache.hadoop.fs.s3a.TestBucketConfiguration
[INFO] Running org.apache.hadoop.fs.s3a.audit.TestAuditSpanLifecycle
[INFO] Running org.apache.hadoop.fs.s3a.audit.TestAuditIntegration
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.49
s - in org.apache.hadoop.fs.s3a.audit.TestAuditSpanLifecycle
[INFO] Running org.apache.hadoop.fs.s3a.audit.TestHttpReferrerAuditHeader
[INFO] Running org.apache.hadoop.fs.s3a.audit.TestLoggingAuditor
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
2.643 s - in org.apache.hadoop.fs.s3a.audit.TestAuditIntegration
[INFO] Running org.apache.hadoop.fs.s3a.TestS3ABlockOutputStream
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.655
s - in org.apache.hadoop.fs.s3a.audit.TestHttpReferrerAuditHeader
[INFO] Running org.apache.hadoop.fs.s3a.prefetch.TestS3ACachingBlockManager
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.187
s - in org.apache.hadoop.fs.s3a.audit.TestLoggingAuditor
[INFO] Running org.apache.hadoop.fs.s3a.prefetch.TestS3ABlockManager
[INFO] Running org.apache.hadoop.fs.s3a.prefetch.TestS3ARemoteObjectReader
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.864
s - in org.apache.hadoop.fs.s3a.prefetch.TestS3ABlockManager
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
5.536 s - in org.apache.hadoop.fs.s3a.prefetch.TestS3ACachingBlockManager
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.526
s - in org.apache.hadoop.fs.s3a.prefetch.TestS3ARemoteObjectReader
[INFO] Running org.apache.hadoop.fs.s3a.prefetch.TestS3ARemoteInputStream
[INFO] Running org.apache.hadoop.fs.s3a.prefetch.TestS3ARemoteObject
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AGetFileStatus
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.019
s - in org.apache.hadoop.fs.s3a.prefetch.TestS3ARemoteObject
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.854
s - in org.apache.hadoop.fs.s3a.prefetch.TestS3ARemoteInputStream
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AUnbuffer
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.TestAuthoritativePath
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.552
s - in org.apache.hadoop.fs.s3a.s3guard.TestAuthoritativePath
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
17.574 s - in org.apache.hadoop.fs.s3a.TestS3ABlockOutputStream
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.TestMetastoreChecking
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.77
s - in org.apache.hadoop.fs.s3a.s3guard.TestMetastoreChecking
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.TestS3GuardCLI
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.22
s - in org.apache.hadoop.fs.s3a.s3guard.TestS3GuardCLI
[INFO] Running org.apache.hadoop.fs.s3a.TestDataBlocks
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.644
s - in org.apache.hadoop.fs.s3a.TestDataBlocks
[INFO] Running org.apache.hadoop.fs.s3a.TestInstrumentationLifecycle
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
14.388 s - in org.apache.hadoop.fs.s3a.TestS3AGetFileStatus
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AAWSCredentialsProvider
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.32
s - in org.apache.hadoop.fs.s3a.TestInstrumentationLifecycle
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.974 s - in org.apache.hadoop.fs.s3a.TestS3AUnbuffer
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AInputPolicies
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.791 s - in org.apache.hadoop.fs.s3a.TestS3AInputPolicies
[INFO] Running org.apache.hadoop.fs.s3a.TestS3ADeleteOnExit
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
32.997 s - in
org.apache.hadoop.fs.s3a.audit.impl.TestActiveAuditManagerThreadLeakage
[INFO] Running org.apache.hadoop.fs.s3a.TestArnResource
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.635
s - in org.apache.hadoop.fs.s3a.TestArnResource
[INFO] Running org.apache.hadoop.fs.s3native.TestS3xLoginHelper
[INFO] Running org.apache.hadoop.mapreduce.filecache.TestS3AResourceScope
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.509 s - in org.apache.hadoop.fs.s3native.TestS3xLoginHelper
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.414
s - in org.apache.hadoop.mapreduce.filecache.TestS3AResourceScope
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.888
s - in org.apache.hadoop.fs.s3a.TestS3ADeleteOnExit
[INFO] Tests run: 63, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
79.551 s - in org.apache.hadoop.fs.s3a.commit.staging.TestStagingCommitter
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
51.766 s - in org.apache.hadoop.fs.s3a.TestS3AInputStreamRetry
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
22.579 s - in org.apache.hadoop.fs.s3a.TestS3AAWSCredentialsProvider
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
124.798 s - in
org.apache.hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 491, Failures: 0, Errors: 0, Skipped: 2
...
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractDistCp
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek
[INFO] Running
org.apache.hadoop.fs.contract.s3a.ITestS3AContractGetFileStatus
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
[INFO] Running
org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
20.944 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer
[WARNING] Tests run: 17, Failures: 0, Errors: 0, Skipped: 17, Time elapsed:
21.65 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractMkdir
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractDelete
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
36.595 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
[INFO] Running
org.apache.hadoop.fs.contract.s3a.ITestS3AContractContentSummary
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
24.948 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractDelete
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.402 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractContentSummary
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractOpen
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.84
s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractMkdir
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
59.734 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractGetFileStatus
[INFO] Running org.apache.hadoop.fs.s3a.ITestLocatedFileStatusFetcher
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials
[WARNING] Tests run: 32, Failures: 0, Errors: 0, Skipped: 8, Time elapsed:
62.63 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
[INFO] Running
org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
12.073 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag
[INFO] Running
org.apache.hadoop.fs.s3a.statistics.ITestAWSStatisticCollection
[INFO] Tests run: 48, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
79.201 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
17.162 s - in org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
[INFO] Running
org.apache.hadoop.fs.s3a.statistics.ITestS3AContractStreamIOStatistics
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestStagingCommitProtocol
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
26.949 s - in org.apache.hadoop.fs.s3a.ITestLocatedFileStatusFetcher
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestDirectoryCommitProtocol
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
17.473 s - in org.apache.hadoop.fs.s3a.statistics.ITestAWSStatisticCollection
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestPartitionedCommitProtocol
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
42.335 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractOpen
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestStagingCommitProtocolFailure
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
18.407 s - in
org.apache.hadoop.fs.s3a.statistics.ITestS3AContractStreamIOStatistics
[INFO] Running org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitProtocol
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
16.186 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestStagingCommitProtocolFailure
[ERROR] Tests run: 14, Failures: 0, Errors: 4, Skipped: 0, Time elapsed:
57.04 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials
[ERROR] testSTS(org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials) Time
elapsed: 3.56 s <<< ERROR!
java.nio.file.AccessDeniedException: /: request session credentials:
software.amazon.awssdk.services.sts.model.StsException: Cannot call
GetSessionToken with session credentials (Service: Sts, Status Code: 403,
Request ID: f3e41642-7be6-4600-9639-56bf204e962f):AccessDenied
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.requestSessionCredentials(STSClientFactory.java:225)
at
org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials.testSTS(ITestS3ATemporaryCredentials.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.sts.model.StsException: Cannot
call GetSessionToken with session credentials (Service: Sts, Status Code: 403,
Request ID: f3e41642-7be6-4600-9639-56bf204e962f)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.sts.DefaultStsClient.getSessionToken(DefaultStsClient.java:1310)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.lambda$requestSessionCredentials$0(STSClientFactory.java:229)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
... 21 more
[INFO] Running
org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitProtocolFailure
[ERROR]
testSessionTokenExpiry(org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials)
Time elapsed: 2.945 s <<< ERROR!
java.nio.file.AccessDeniedException: /: request session credentials:
software.amazon.awssdk.services.sts.model.StsException: The security token
included in the request is invalid. (Service: Sts, Status Code: 403, Request
ID: f4814437-edd3-463a-b4de-f4957aa7c0a9):InvalidClientTokenId
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.requestSessionCredentials(STSClientFactory.java:225)
at
org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding.requestSessionCredentials(MarshalledCredentialBinding.java:214)
at
org.apache.hadoop.fs.s3a.S3ATestUtils.requestSessionCredentials(S3ATestUtils.java:784)
at
org.apache.hadoop.fs.s3a.S3ATestUtils.requestSessionCredentials(S3ATestUtils.java:765)
at
org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials.testSessionTokenExpiry(ITestS3ATemporaryCredentials.java:222)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.sts.model.StsException: The
security token included in the request is invalid. (Service: Sts, Status Code:
403, Request ID: f4814437-edd3-463a-b4de-f4957aa7c0a9)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.sts.DefaultStsClient.getSessionToken(DefaultStsClient.java:1310)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.lambda$requestSessionCredentials$0(STSClientFactory.java:229)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
... 24 more
[ERROR]
testSessionTokenPropagation(org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials)
Time elapsed: 3.259 s <<< ERROR!
java.nio.file.AccessDeniedException: /: request session credentials:
software.amazon.awssdk.services.sts.model.StsException: The security token
included in the request is invalid. (Service: Sts, Status Code: 403, Request
ID: 6b6c1fe3-87ec-4ab5-b4ec-64d2fa75b697):InvalidClientTokenId
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.requestSessionCredentials(STSClientFactory.java:225)
at
org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding.requestSessionCredentials(MarshalledCredentialBinding.java:214)
at
org.apache.hadoop.fs.s3a.S3ATestUtils.requestSessionCredentials(S3ATestUtils.java:784)
at
org.apache.hadoop.fs.s3a.S3ATestUtils.requestSessionCredentials(S3ATestUtils.java:765)
at
org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials.testSessionTokenPropagation(ITestS3ATemporaryCredentials.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.sts.model.StsException: The
security token included in the request is invalid. (Service: Sts, Status Code:
403, Request ID: 6b6c1fe3-87ec-4ab5-b4ec-64d2fa75b697)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.sts.DefaultStsClient.getSessionToken(DefaultStsClient.java:1310)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.lambda$requestSessionCredentials$0(STSClientFactory.java:229)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
... 24 more
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.9
s - in org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitProtocolFailure
[ERROR]
testInvalidSTSBinding(org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials)
Time elapsed: 3.319 s <<< ERROR!
java.nio.file.AccessDeniedException: /: request session credentials:
software.amazon.awssdk.services.sts.model.StsException: The security token
included in the request is invalid. (Service: Sts, Status Code: 403, Request
ID: b23777ae-62aa-4f51-84bc-52b80d08b0ce):InvalidClientTokenId
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.requestSessionCredentials(STSClientFactory.java:225)
at
org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding.requestSessionCredentials(MarshalledCredentialBinding.java:214)
at
org.apache.hadoop.fs.s3a.S3ATestUtils.requestSessionCredentials(S3ATestUtils.java:784)
at
org.apache.hadoop.fs.s3a.S3ATestUtils.requestSessionCredentials(S3ATestUtils.java:765)
at
org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials.testInvalidSTSBinding(ITestS3ATemporaryCredentials.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.sts.model.StsException: The
security token included in the request is invalid. (Service: Sts, Status Code:
403, Request ID: b23777ae-62aa-4f51-84bc-52b80d08b0ce)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
at
software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.sts.DefaultStsClient.getSessionToken(DefaultStsClient.java:1310)
at
org.apache.hadoop.fs.s3a.auth.STSClientFactory$STSClient.lambda$requestSessionCredentials$0(STSClientFactory.java:229)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
... 24 more
[INFO] Running
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob
[INFO] Running org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
16.453 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory
[ERROR]
testEverything(org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory) Time
elapsed: 15.64 s <<< FAILURE!
java.lang.AssertionError: Expected a
org.apache.hadoop.fs.s3a.commit.PathCommitException to be thrown, but got the
result: :
FileOutputCommitter{PathOutputCommitter{context=TaskAttemptContextImpl{JobContextImpl{jobId=job_202402070036_0002};
taskId=attempt_202402070036_0002_m_000000_0, status=''};
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter@281b2572};
outputPath=s3a://ahemani-s3a-testing/fork-0002/test/testEverything,
workPath=s3a://ahemani-s3a-testing/fork-0002/test/testEverything/_temporary/1/_temporary/attempt_202402070036_0002_m_000000_0,
algorithmVersion=1, skipCleanup=false, ignoreCleanupFailures=false}
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:453)
at
org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory.testInvalidFileBinding(ITestS3ACommitterFactory.java:165)
at
org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory.testEverything(ITestS3ACommitterFactory.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[INFO] Running org.apache.hadoop.fs.s3a.commit.ITestCommitOperations
[WARNING] Tests run: 13, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
165.44 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractDistCp
[INFO] Running org.apache.hadoop.fs.s3a.commit.ITestCommitOperationCost
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
34.946 s - in org.apache.hadoop.fs.s3a.commit.ITestCommitOperationCost
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestRenameDeleteRace
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
47.641 s - in org.apache.hadoop.fs.s3a.commit.ITestCommitOperations
[INFO] Running
org.apache.hadoop.fs.s3a.impl.ITestUploadPurgeOnDirectoryOperations
[INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
126.564 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestStagingCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestTreewalkProblems
[WARNING] Tests run: 24, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
124.132 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestPartitionedCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestConnectionTimeouts
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
18.104 s - in org.apache.hadoop.fs.s3a.impl.ITestRenameDeleteRace
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestPartialRenamesDeletes
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
18.675 s - in org.apache.hadoop.fs.s3a.impl.ITestConnectionTimeouts
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
29.294 s - in
org.apache.hadoop.fs.s3a.impl.ITestUploadPurgeOnDirectoryOperations
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestXAttrCost
[INFO] Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI
[INFO] Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
153.933 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestDirectoryCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContext
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.963 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContext
[INFO] Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextUtil
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
34.149 s - in org.apache.hadoop.fs.s3a.impl.ITestXAttrCost
[WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 24, Time elapsed:
277.279 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek
[INFO] Running
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextCreateMkdir
[INFO] Running
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
21.986 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextUtil
[INFO] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
174.066 s - in org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABucketExistence
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSES3
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
78.601 s - in org.apache.hadoop.fs.s3a.impl.ITestTreewalkProblems
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlocksize
[WARNING] Tests run: 17, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
70.87 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
37.195 s - in
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextCreateMkdir
[INFO] Running org.apache.hadoop.fs.s3a.ITestDowngradeSyncable
[INFO] Running org.apache.hadoop.fs.s3a.tools.ITestBucketTool
[WARNING] Tests run: 26, Failures: 0, Errors: 0, Skipped: 26, Time elapsed:
90.357 s - in org.apache.hadoop.fs.s3a.impl.ITestPartialRenamesDeletes
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
22.578 s - in org.apache.hadoop.fs.s3a.ITestS3ABlocksize
[INFO] Running org.apache.hadoop.fs.s3a.tools.ITestMarkerTool
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFailureHandling
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
37.496 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSES3
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteFilesOneByOne
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
47.231 s - in org.apache.hadoop.fs.s3a.ITestS3ABucketExistence
[INFO] Running
org.apache.hadoop.fs.s3a.scale.ITestS3AMultipartUploadSizeLimits
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
36.465 s - in org.apache.hadoop.fs.s3a.tools.ITestBucketTool
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
26.223 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteFilesOneByOne
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
26.053 s - in org.apache.hadoop.fs.s3a.ITestDowngradeSyncable
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ACreatePerformance
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
36.386 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AMultipartUploadSizeLimits
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
50.92 s - in org.apache.hadoop.fs.s3a.ITestS3AFailureHandling
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteManyFiles
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
17.987 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ACreatePerformance
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AContentEncoding
[WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
34.73 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
19.507 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteManyFiles
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ADelayedFNF
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AUrlScheme
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
22.501 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 8, Time elapsed:
39.653 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AUnbuffer
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
21.171 s - in org.apache.hadoop.fs.s3a.ITestS3AContentEncoding
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestJceksIO
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestHttpSigner
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
22.221 s - in org.apache.hadoop.fs.s3a.ITestS3ADelayedFNF
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
22.427 s - in org.apache.hadoop.fs.s3a.ITestS3AUrlScheme
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestCustomSigner
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestRestrictedReadAccess
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
30.643 s - in org.apache.hadoop.fs.s3a.ITestS3AUnbuffer
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
29.808 s - in org.apache.hadoop.fs.s3a.auth.ITestJceksIO
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestAssumeRole
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationTokens
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
34.183 s - in org.apache.hadoop.fs.s3a.auth.ITestHttpSigner
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
17.128 s - in org.apache.hadoop.fs.s3a.auth.ITestRestrictedReadAccess
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationInFilesystem
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFilesystem
[WARNING] Tests run: 73, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
167.179 s - in
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationTokens
[INFO] Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
131.484 s - in org.apache.hadoop.fs.s3a.tools.ITestMarkerTool
[INFO] Running org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
37.906 s - in org.apache.hadoop.fs.s3a.auth.ITestCustomSigner
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
28.312 s - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationTokens
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestAssumedRoleCommitOperations
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AMetrics
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
330.383 s - in
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob
[WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 7, Time elapsed:
26.304 s - in org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationTokens
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSDefaultKey
[ERROR] Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
32.893 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFilesystem
[ERROR]
testDelegatedFileSystem(org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFilesystem)
Time elapsed: 1.577 s <<< ERROR!
java.nio.file.AccessDeniedException: landsat-pds: HEAD on landsat-pds:
software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status
Code: 403, Request ID: TC3KNKGQWAA86TFV, Extended Request ID:
+YkGVi2rupfduOTxwLyTpj2RkaM5cCtSrmdU3k1BF40xpnfh3Mx1j7yIXJK9OPoS0OQF+xp6Cek=):null
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFilesystem.readLandsatMetadata(ITestSessionDelegationInFilesystem.java:614)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFilesystem.testDelegatedFileSystem(ITestSessionDelegationInFilesystem.java:347)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: null
(Service: S3, Status Code: 403, Request ID: TC3KNKGQWAA86TFV, Extended Request
ID:
+YkGVi2rupfduOTxwLyTpj2RkaM5cCtSrmdU3k1BF40xpnfh3Mx1j7yIXJK9OPoS0OQF+xp6Cek=)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:93)
at
software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:279)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.s3.DefaultS3Client.headBucket(DefaultS3Client.java:6104)
at
software.amazon.awssdk.services.s3.DelegatingS3Client.lambda$headBucket$52(DelegatingS3Client.java:4857)
at
software.amazon.awssdk.services.s3.internal.crossregion.S3CrossRegionSyncClient.invokeOperation(S3CrossRegionSyncClient.java:74)
at
software.amazon.awssdk.services.s3.DelegatingS3Client.headBucket(DelegatingS3Client.java:4857)
at
software.amazon.awssdk.services.s3.S3Client.headBucket(S3Client.java:11603)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFilesystem.lambda$readLandsatMetadata$3(ITestSessionDelegationInFilesystem.java:615)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
... 17 more
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 11, Time elapsed:
45.289 s - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationInFilesystem
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
19.987 s - in org.apache.hadoop.fs.s3a.ITestS3AMetrics
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AStorageClass
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AConfiguration
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AClosedFS
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AClientSideEncryptionKms
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
30.225 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSDefaultKey
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ADeleteOnExit
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
32.094 s - in org.apache.hadoop.fs.s3a.ITestS3AStorageClass
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AMultipartUtils
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
23.258 s - in org.apache.hadoop.fs.s3a.ITestS3AConfiguration
[ERROR] Tests run: 6, Failures: 0, Errors: 3, Skipped: 2, Time elapsed:
61.544 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob
[ERROR]
testJobSubmissionCollectsTokens[0](org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob)
Time elapsed: 14.368 s <<< ERROR!
java.nio.file.AccessDeniedException: s3a://landsat-pds/scene_list.gz:
getFileStatus on s3a://landsat-pds/scene_list.gz:
software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status
Code: 403, Request ID: 1CD7PD45HHY30D10, Extended Request ID:
poj66X3skeF55B+SJ/braF79DN5Zy0Arp3WnFBNFi8UyUhnzqDe4qKLMlR6hVJE/REyzksgEy5M=):null
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:154)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4016)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3922)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getFileStatus$26(S3AFileSystem.java:3899)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2707)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2726)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3897)
at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:115)
at org.apache.hadoop.fs.Globber.doGlob(Globber.java:349)
at org.apache.hadoop.fs.Globber.glob(Globber.java:202)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$globStatus$35(S3AFileSystem.java:4929)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2707)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2726)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.globStatus(S3AFileSystem.java:4922)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:313)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:281)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:445)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:311)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:328)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:201)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1677)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1674)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1674)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob.testJobSubmissionCollectsTokens(ITestDelegatedMRJob.java:281)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: null
(Service: S3, Status Code: 403, Request ID: 1CD7PD45HHY30D10, Extended Request
ID:
poj66X3skeF55B+SJ/braF79DN5Zy0Arp3WnFBNFi8UyUhnzqDe4qKLMlR6hVJE/REyzksgEy5M=)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:93)
at
software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:279)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.s3.DefaultS3Client.headObject(DefaultS3Client.java:6319)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$10(S3AFileSystem.java:2892)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2880)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2860)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3994)
... 46 more
[WARNING] Tests run: 25, Failures: 0, Errors: 0, Skipped: 25, Time elapsed:
85.27 s - in org.apache.hadoop.fs.s3a.auth.ITestAssumeRole
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ACopyFromLocalFile
[ERROR]
testCommonCrawlLookup[1](org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob)
Time elapsed: 2.758 s <<< ERROR!
java.nio.file.AccessDeniedException:
s3a://ahemani-s3a-testing/fork-0007/test: getFileStatus on
s3a://ahemani-s3a-testing/fork-0007/test:
software.amazon.awssdk.services.s3.model.S3Exception: The AWS Access Key Id you
provided does not exist in our records. (Service: S3, Status Code: 403, Request
ID: 5N4XX5GW2Z74X8AT, Extended Request ID:
TQdmT2UKsdj4UwxdXj0KoKGCLnU0N3v0/Iez6ZXBVkvYZwcg6rccbbrC/jOlqInJdAQQqxL7stc=):InvalidAccessKeyId
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:154)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4063)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3922)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$MkdirOperationCallbacksImpl.probePathStatus(S3AFileSystem.java:3794)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.probePathStatusOrNull(MkdirOperation.java:173)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.getPathStatusExpectingDir(MkdirOperation.java:194)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:108)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:57)
at
org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation.apply(ExecutingStoreOperation.java:76)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2707)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2726)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:3766)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2494)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:363)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:205)
at
org.apache.hadoop.fs.s3a.AbstractS3ATestBase.setup(AbstractS3ATestBase.java:111)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob.setup(ITestDelegatedMRJob.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The AWS
Access Key Id you provided does not exist in our records. (Service: S3, Status
Code: 403, Request ID: 5N4XX5GW2Z74X8AT, Extended Request ID:
TQdmT2UKsdj4UwxdXj0KoKGCLnU0N3v0/Iez6ZXBVkvYZwcg6rccbbrC/jOlqInJdAQQqxL7stc=)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:93)
at
software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:279)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.s3.DefaultS3Client.listObjectsV2(DefaultS3Client.java:7327)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$listObjects$13(S3AFileSystem.java:2963)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.listObjects(S3AFileSystem.java:2954)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4038)
... 33 more
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
15.421 s - in org.apache.hadoop.fs.s3a.ITestS3ADeleteOnExit
[INFO] Running org.apache.hadoop.fs.s3a.yarn.ITestS3AMiniYarnCluster
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ACannedACLs
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
27.231 s - in org.apache.hadoop.fs.s3a.ITestS3AClientSideEncryptionKms
[ERROR]
testJobSubmissionCollectsTokens[1](org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob)
Time elapsed: 2.567 s <<< ERROR!
java.nio.file.AccessDeniedException:
s3a://ahemani-s3a-testing/fork-0007/test: getFileStatus on
s3a://ahemani-s3a-testing/fork-0007/test:
software.amazon.awssdk.services.s3.model.S3Exception: The AWS Access Key Id you
provided does not exist in our records. (Service: S3, Status Code: 403, Request
ID: 487HZGW94B0Y5AYB, Extended Request ID:
dlhJwmflEt74S0ikow/gKbJNuCwooU/dAy0cuAhLxTX+pv7IcD7i5xeieBAy7gzxBuugZW/IpoU=):InvalidAccessKeyId
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:265)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:154)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4063)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3922)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$MkdirOperationCallbacksImpl.probePathStatus(S3AFileSystem.java:3794)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.probePathStatusOrNull(MkdirOperation.java:173)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.getPathStatusExpectingDir(MkdirOperation.java:194)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:108)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:57)
at
org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation.apply(ExecutingStoreOperation.java:76)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2707)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2726)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:3766)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2494)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:363)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:205)
at
org.apache.hadoop.fs.s3a.AbstractS3ATestBase.setup(AbstractS3ATestBase.java:111)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob.setup(ITestDelegatedMRJob.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The AWS
Access Key Id you provided does not exist in our records. (Service: S3, Status
Code: 403, Request ID: 487HZGW94B0Y5AYB, Extended Request ID:
dlhJwmflEt74S0ikow/gKbJNuCwooU/dAy0cuAhLxTX+pv7IcD7i5xeieBAy7gzxBuugZW/IpoU=)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:93)
at
software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:279)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.s3.DefaultS3Client.listObjectsV2(DefaultS3Client.java:7327)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$listObjects$13(S3AFileSystem.java:2963)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.listObjects(S3AFileSystem.java:2954)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4038)
... 33 more
[INFO] Running org.apache.hadoop.fs.s3a.yarn.ITestS3A
[INFO] Running
org.apache.hadoop.fs.s3a.ITestBlockingThreadPoolExecutorService
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
14.912 s - in org.apache.hadoop.fs.s3a.ITestS3AMultipartUtils
[WARNING] Tests run: 18, Failures: 0, Errors: 0, Skipped: 18, Time elapsed:
63.392 s - in org.apache.hadoop.fs.s3a.auth.ITestAssumedRoleCommitOperations
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
33.037 s - in org.apache.hadoop.fs.s3a.ITestS3AClosedFS
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.782
s - in org.apache.hadoop.fs.s3a.ITestBlockingThreadPoolExecutorService
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3AEncryptionDSSEKMSUserDefinedKey
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputByteBuffer
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ARequesterPays
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
3.539 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionDSSEKMSUserDefinedKey
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
18.973 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3ACannedACLs
[ERROR]
testCreatedObjectsHaveACLs(org.apache.hadoop.fs.s3a.ITestS3ACannedACLs) Time
elapsed: 15.586 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSBadRequestException: PUT 0-byte object on
fork-0004/test: software.amazon.awssdk.services.s3.model.S3Exception: The
bucket does not allow ACLs (Service: S3, Status Code: 400, Request ID:
YQTBSNMD71QBKHKP, Extended Request ID:
/KNs8llNcb8ahf49IC+fTGZPniNjLRy/zqV9QiDym/j3pfCSeyc7lRz7Y9rKDZygtr5AUGNLIFs/IH4GHHampg==):AccessControlListNotSupported:
The bucket does not allow ACLs (Service: S3, Status Code: 400, Request ID:
YQTBSNMD71QBKHKP, Extended Request ID:
/KNs8llNcb8ahf49IC+fTGZPniNjLRy/zqV9QiDym/j3pfCSeyc7lRz7Y9rKDZygtr5AUGNLIFs/IH4GHHampg==)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:259)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.createEmptyObject(S3AFileSystem.java:4764)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.createFakeDirectory(S3AFileSystem.java:4740)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.access$2900(S3AFileSystem.java:281)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$MkdirOperationCallbacksImpl.createFakeDirectory(S3AFileSystem.java:3800)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:159)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:57)
at
org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation.apply(ExecutingStoreOperation.java:76)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2707)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2726)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:3766)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2494)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:363)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:205)
at
org.apache.hadoop.fs.s3a.AbstractS3ATestBase.setup(AbstractS3ATestBase.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The bucket
does not allow ACLs (Service: S3, Status Code: 400, Request ID:
YQTBSNMD71QBKHKP, Extended Request ID:
/KNs8llNcb8ahf49IC+fTGZPniNjLRy/zqV9QiDym/j3pfCSeyc7lRz7Y9rKDZygtr5AUGNLIFs/IH4GHHampg==)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85)
at
software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:93)
at
software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:279)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at
software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:10191)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$putObjectDirect$19(S3AFileSystem.java:3260)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDurationOfSupplier(IOStatisticsBinding.java:651)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.putObjectDirect(S3AFileSystem.java:3256)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$createEmptyObject$34(S3AFileSystem.java:4765)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
... 36 more
[INFO] Running org.apache.hadoop.fs.s3a.audit.ITestAuditAccessChecks
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
18.383 s - in org.apache.hadoop.fs.s3a.yarn.ITestS3A
[INFO] Running org.apache.hadoop.fs.s3a.audit.ITestAuditManager
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
22.948 s - in org.apache.hadoop.fs.s3a.ITestS3ARequesterPays
[INFO] Running org.apache.hadoop.fs.s3a.audit.ITestAuditManagerDisabled
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
14.806 s - in org.apache.hadoop.fs.s3a.audit.ITestAuditManager
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSUserDefinedKey
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
0.488 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSUserDefinedKey
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputDisk
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.46
s - in org.apache.hadoop.fs.s3a.yarn.ITestS3AMiniYarnCluster
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.412 s - in org.apache.hadoop.fs.s3a.audit.ITestAuditManagerDisabled
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputArray
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AMiscOperations
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
33.203 s - in org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
39.398 s - in org.apache.hadoop.fs.s3a.ITestS3ABlockOutputByteBuffer
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
32.392 s - in org.apache.hadoop.fs.s3a.audit.ITestAuditAccessChecks
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ATestUtils
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.396
s - in org.apache.hadoop.fs.s3a.ITestS3ATestUtils
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.128 s - in org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFSMainOperations
[WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
29.8 s - in org.apache.hadoop.fs.s3a.ITestS3ABlockOutputDisk
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
30.795 s - in org.apache.hadoop.fs.s3a.ITestS3ABlockOutputArray
[INFO] Running org.apache.hadoop.fs.s3a.select.ITestSelectUnsupported
[WARNING] Tests run: 16, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
28.168 s - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost
[INFO] Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
88.479 s - in org.apache.hadoop.fs.s3a.ITestS3ACopyFromLocalFile
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestCreateSessionTimeout
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
1.078 s - in org.apache.hadoop.fs.s3a.performance.ITestCreateSessionTimeout
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3AMkdirCost
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
46.329 s - in org.apache.hadoop.fs.s3a.ITestS3AMiscOperations
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.82
s - in org.apache.hadoop.fs.s3a.ITestS3APrefetchingLruEviction
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3ARenameCost
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
17.814 s - in org.apache.hadoop.fs.s3a.select.ITestSelectUnsupported
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3AMiscOperationCost
[INFO] Running
org.apache.hadoop.fs.s3a.performance.ITestDirectoryMarkerListing
[WARNING] Tests run: 43, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
62.407 s - in org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestUnbufferDraining
[ERROR] Tests run: 8, Failures: 2, Errors: 2, Skipped: 1, Time elapsed:
34.507 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost
[ERROR]
testVectorReadPastEOF(org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost)
Time elapsed: 3.226 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream cannot be cast to
org.apache.hadoop.fs.s3a.S3AInputStream
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.assertS3StreamClosed(ITestS3AOpenCost.java:434)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.lambda$testVectorReadPastEOF$14(ITestS3AOpenCost.java:412)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator.exec(OperationCostValidator.java:167)
at
org.apache.hadoop.fs.s3a.performance.AbstractS3ACostTest.verifyMetrics(AbstractS3ACostTest.java:318)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.testVectorReadPastEOF(ITestS3AOpenCost.java:409)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
testPositionedReadableReadPastEOF(org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost)
Time elapsed: 3.378 s <<< FAILURE!
org.junit.ComparisonFailure: expected:<[16]> but was:<[-1]>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.lambda$testPositionedReadableReadPastEOF$12(ITestS3AOpenCost.java:381)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator.exec(OperationCostValidator.java:167)
at
org.apache.hadoop.fs.s3a.performance.AbstractS3ACostTest.verifyMetrics(AbstractS3ACostTest.java:318)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.testPositionedReadableReadPastEOF(ITestS3AOpenCost.java:374)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
testPositionedReadableReadFullyPastEOF(org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost)
Time elapsed: 3.307 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream cannot be cast to
org.apache.hadoop.fs.s3a.S3AInputStream
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.assertS3StreamClosed(ITestS3AOpenCost.java:434)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.lambda$testPositionedReadableReadFullyPastEOF$11(ITestS3AOpenCost.java:355)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator.exec(OperationCostValidator.java:167)
at
org.apache.hadoop.fs.s3a.performance.AbstractS3ACostTest.verifyMetrics(AbstractS3ACostTest.java:318)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.testPositionedReadableReadFullyPastEOF(ITestS3AOpenCost.java:346)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
testReadPastEOF(org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost) Time
elapsed: 3.262 s <<< FAILURE!
org.junit.ComparisonFailure: [read() at 0] expected:<[48]> but was:<[-1]>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost.testReadPastEOF(ITestS3AOpenCost.java:303)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestCreateFileCost
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
37.681 s - in org.apache.hadoop.fs.s3a.performance.ITestS3AMkdirCost
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
15.366 s - in org.apache.hadoop.fs.s3a.performance.ITestUnbufferDraining
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEmptyDirectory
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
36.485 s - in org.apache.hadoop.fs.s3a.performance.ITestS3AMiscOperationCost
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
43.233 s - in org.apache.hadoop.fs.s3a.performance.ITestS3ARenameCost
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.571 s - in org.apache.hadoop.fs.s3a.ITestS3AEmptyDirectory
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionAlgorithmValidation
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
0.036 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionAlgorithmValidation
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3AEncryptionWithDefaultS3Settings
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
82.614 s - in org.apache.hadoop.fs.s3a.ITestS3AFSMainOperations
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
10.97 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionWithDefaultS3Settings
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3ADSSEEncryptionWithDefaultS3Settings
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AIOStatisticsContext
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
44.755 s - in org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
14.307 s - in
org.apache.hadoop.fs.s3a.ITestS3ADSSEEncryptionWithDefaultS3Settings
[WARNING] Tests run: 12, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
36.465 s - in org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion
[INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
166.057 s - in org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
48.406 s - in org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
32.671 s - in org.apache.hadoop.fs.s3a.ITestS3AIOStatisticsContext
[INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
84.316 s - in org.apache.hadoop.fs.s3a.performance.ITestCreateFileCost
[INFO] Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
163.125 s - in org.apache.hadoop.fs.s3a.performance.ITestDirectoryMarkerListing
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestS3ACommitterFactory.testEverything:115->testInvalidFileBinding:165
Expected a org.apache.hadoop.fs.s3a.commit.PathCommitException to be thrown,
but got the result: :
FileOutputCommitter{PathOutputCommitter{context=TaskAttemptContextImpl{JobContextImpl{jobId=job_202402070036_0002};
taskId=attempt_202402070036_0002_m_000000_0, status=''};
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter@281b2572};
outputPath=s3a://ahemani-s3a-testing/fork-0002/test/testEverything,
workPath=s3a://ahemani-s3a-testing/fork-0002/test/testEverything/_temporary/1/_temporary/attempt_202402070036_0002_m_000000_0,
algorithmVersion=1, skipCleanup=false, ignoreCleanupFailures=false}
[ERROR]
ITestS3AOpenCost.testPositionedReadableReadPastEOF:374->AbstractS3ACostTest.verifyMetrics:318->lambda$testPositionedReadableReadPastEOF$12:381
expected:<[16]> but was:<[-1]>
[ERROR] ITestS3AOpenCost.testReadPastEOF:303 [read() at 0] expected:<[48]>
but was:<[-1]>
[ERROR] Errors:
[ERROR]
ITestS3ACannedACLs>AbstractS3ATestBase.setup:111->AbstractFSContractTestBase.setup:205->AbstractFSContractTestBase.mkdirs:363
» AWSBadRequest
[ERROR] ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 »
AccessDenied /: reque...
[ERROR] ITestS3ATemporaryCredentials.testSTS:132 » AccessDenied /: request
session cre...
[ERROR] ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 »
AccessDenied /: requ...
[ERROR] ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 »
AccessDenied /:...
[ERROR]
ITestDelegatedMRJob.setup:182->AbstractS3ATestBase.setup:111->AbstractFSContractTestBase.setup:205->AbstractFSContractTestBase.mkdirs:363
» AccessDenied
[ERROR] ITestDelegatedMRJob.testJobSubmissionCollectsTokens:281 »
AccessDenied s3a://l...
[ERROR]
ITestDelegatedMRJob.setup:182->AbstractS3ATestBase.setup:111->AbstractFSContractTestBase.setup:205->AbstractFSContractTestBase.mkdirs:363
» AccessDenied
[ERROR]
ITestSessionDelegationInFilesystem.testDelegatedFileSystem:347->readLandsatMetadata:614
» AccessDenied
[ERROR]
ITestS3AOpenCost.testPositionedReadableReadFullyPastEOF:346->AbstractS3ACostTest.verifyMetrics:318->lambda$testPositionedReadableReadFullyPastEOF$11:355->assertS3StreamClosed:434
ClassCast
[ERROR]
ITestS3AOpenCost.testVectorReadPastEOF:409->AbstractS3ACostTest.verifyMetrics:318->lambda$testVectorReadPastEOF$14:412->assertS3StreamClosed:434
ClassCast
[INFO]
[ERROR] Tests run: 1202, Failures: 3, Errors: 11, Skipped: 212
[INFO]
[INFO]
[INFO] -
> update AWS SDK to support S3 Access Grants in S3A
> -------------------------------------------------
>
> Key: HADOOP-19059
> URL: https://issues.apache.org/jira/browse/HADOOP-19059
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build, fs/s3
> Affects Versions: 3.4.0
> Reporter: Jason Han
> Assignee: Jason Han
> Priority: Minor
> Labels: pull-request-available
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In order to support S3 Access
> Grants(https://aws.amazon.com/s3/features/access-grants/) in S3A, we need to
> update AWS SDK in hadooop package.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
