ashutoshcipher commented on PR #4539:
URL: https://github.com/apache/hadoop/pull/4539#issuecomment-1179465579
```
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedJobCommit
[INFO] Running org.apache.hadoop.fs.s3a.commit.staging.TestStagingCommitter
[INFO] Running org.apache.hadoop.fs.s3a.commit.staging.TestPaths
[INFO] Running org.apache.hadoop.fs.s3a.commit.TestMagicCommitPaths
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedTaskCommit
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.TestStagingDirectoryOutputCommitter
[INFO] Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.426 s - in org.apache.hadoop.fs.s3a.commit.TestMagicCommitPaths
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
1.288 s - in org.apache.hadoop.fs.s3a.commit.staging.TestPaths
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestHeaderProcessing
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestCreateFileBuilder
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.38
s - in org.apache.hadoop.fs.s3a.impl.TestHeaderProcessing
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.46
s - in org.apache.hadoop.fs.s3a.impl.TestCreateFileBuilder
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestS3AMultipartUploaderSupport
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.367
s - in org.apache.hadoop.fs.s3a.impl.TestS3AMultipartUploaderSupport
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestRequestFactory
[INFO] Running org.apache.hadoop.fs.s3a.impl.TestDirectoryMarkerPolicy
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.775
s - in org.apache.hadoop.fs.s3a.impl.TestRequestFactory
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.602
s - in org.apache.hadoop.fs.s3a.impl.TestDirectoryMarkerPolicy
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.377
s - in
org.apache.hadoop.fs.s3a.commit.staging.TestStagingDirectoryOutputCommitter
88665a380593:hadoop-aws ashugpt$
88665a380593:hadoop-aws ashugpt$
88665a380593:hadoop-aws ashugpt$ tail -f
integration_test_response_hadoop_trunk.out
[INFO] Running org.apache.hadoop.fs.s3a.audit.TestLoggingAuditor
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.039
s - in org.apache.hadoop.fs.s3a.audit.TestHttpReferrerAuditHeader
[INFO] Running org.apache.hadoop.fs.s3a.TestS3ABlockOutputStream
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AGetFileStatus
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AUnbuffer
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.259
s - in org.apache.hadoop.fs.s3a.audit.TestLoggingAuditor
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.TestAuthoritativePath
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.062
s - in org.apache.hadoop.fs.s3a.s3guard.TestAuthoritativePath
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.TestMetastoreChecking
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.494
s - in org.apache.hadoop.fs.s3a.s3guard.TestMetastoreChecking
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.TestS3GuardCLI
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.143
s - in org.apache.hadoop.fs.s3a.s3guard.TestS3GuardCLI
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.172
s - in org.apache.hadoop.fs.s3a.TestS3AGetFileStatus
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.083
s - in org.apache.hadoop.fs.s3a.TestS3AUnbuffer
[INFO] Running org.apache.hadoop.fs.s3a.TestDataBlocks
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.822
s - in org.apache.hadoop.fs.s3a.TestDataBlocks
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AAWSCredentialsProvider
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.95
s - in org.apache.hadoop.fs.s3a.TestS3ABlockOutputStream
[INFO] Running org.apache.hadoop.fs.s3a.TestS3AInputPolicies
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.433 s - in org.apache.hadoop.fs.s3a.TestS3AInputPolicies
[INFO] Running org.apache.hadoop.fs.s3a.TestArnResource
[INFO] Running org.apache.hadoop.fs.s3native.TestS3xLoginHelper
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.114
s - in org.apache.hadoop.fs.s3a.TestArnResource
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.764 s - in org.apache.hadoop.fs.s3native.TestS3xLoginHelper
[INFO] Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
2.404 s - in org.apache.hadoop.fs.s3a.TestS3AAWSCredentialsProvider
[INFO] Running org.apache.hadoop.mapreduce.filecache.TestS3AResourceScope
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.454
s - in org.apache.hadoop.mapreduce.filecache.TestS3AResourceScope
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
25.804 s - in
org.apache.hadoop.fs.s3a.audit.impl.TestActiveAuditManagerThreadLeakage
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
27.458 s - in org.apache.hadoop.fs.s3a.TestS3AInputStreamRetry
[INFO] Tests run: 63, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
57.71 s - in org.apache.hadoop.fs.s3a.commit.staging.TestStagingCommitter
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
80.766 s - in
org.apache.hadoop.fs.s3a.commit.staging.TestDirectoryCommitterScale
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 406, Failures: 0, Errors: 0, Skipped: 4
[INFO]
[INFO]
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-aws ---
[INFO] Building jar:
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/hadoop-aws-3.4.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:copy-dependencies (default) @
hadoop-aws ---
[INFO] Copying hadoop-common-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-common-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-shaded-protobuf_3_7-1.1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar
[INFO] Copying hadoop-annotations-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-annotations-3.4.0-SNAPSHOT.jar
[INFO] Copying tools.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jdk.tools-1.8.jar
[INFO] Copying hadoop-shaded-guava-1.1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-shaded-guava-1.1.1.jar
[INFO] Copying guava-27.0-jre.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/guava-27.0-jre.jar
[INFO] Copying failureaccess-1.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/failureaccess-1.0.jar
[INFO] Copying
listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
[INFO] Copying checker-qual-2.5.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/checker-qual-2.5.2.jar
[INFO] Copying j2objc-annotations-1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/j2objc-annotations-1.1.jar
[INFO] Copying animal-sniffer-annotations-1.17.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/animal-sniffer-annotations-1.17.jar
[INFO] Copying commons-cli-1.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-cli-1.2.jar
[INFO] Copying commons-math3-3.1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-math3-3.1.1.jar
[INFO] Copying httpclient-4.5.13.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/httpclient-4.5.13.jar
[INFO] Copying httpcore-4.4.13.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/httpcore-4.4.13.jar
[INFO] Copying commons-codec-1.15.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-codec-1.15.jar
[INFO] Copying commons-io-2.8.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-io-2.8.0.jar
[INFO] Copying commons-net-3.6.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-net-3.6.jar
[INFO] Copying commons-collections-3.2.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-collections-3.2.2.jar
[INFO] Copying javax.servlet-api-3.1.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax.servlet-api-3.1.0.jar
[INFO] Copying jakarta.activation-api-1.2.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jakarta.activation-api-1.2.1.jar
[INFO] Copying jetty-server-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-server-9.4.44.v20210927.jar
[INFO] Copying jetty-http-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-http-9.4.44.v20210927.jar
[INFO] Copying jetty-io-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-io-9.4.44.v20210927.jar
[INFO] Copying jetty-util-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-util-9.4.44.v20210927.jar
[INFO] Copying jetty-servlet-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-servlet-9.4.44.v20210927.jar
[INFO] Copying jetty-security-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-security-9.4.44.v20210927.jar
[INFO] Copying jetty-webapp-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-webapp-9.4.44.v20210927.jar
[INFO] Copying jetty-xml-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-xml-9.4.44.v20210927.jar
[INFO] Copying jsp-api-2.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jsp-api-2.1.jar
[INFO] Copying jersey-core-1.19.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jersey-core-1.19.4.jar
[INFO] Copying jsr311-api-1.1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jsr311-api-1.1.1.jar
[INFO] Copying jersey-servlet-1.19.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jersey-servlet-1.19.4.jar
[INFO] Copying jersey-json-1.20.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jersey-json-1.20.jar
[INFO] Copying jettison-1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jettison-1.1.jar
[INFO] Copying jaxb-impl-2.2.3-1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jaxb-impl-2.2.3-1.jar
[INFO] Copying jersey-server-1.19.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jersey-server-1.19.4.jar
[INFO] Copying commons-logging-1.1.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-logging-1.1.3.jar
[INFO] Copying log4j-1.2.17.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/log4j-1.2.17.jar
[INFO] Copying commons-beanutils-1.9.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-beanutils-1.9.4.jar
[INFO] Copying commons-configuration2-2.1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-configuration2-2.1.1.jar
[INFO] Copying commons-lang3-3.12.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-lang3-3.12.0.jar
[INFO] Copying commons-text-1.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-text-1.4.jar
[INFO] Copying slf4j-api-1.7.30.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/slf4j-api-1.7.30.jar
[INFO] Copying slf4j-log4j12-1.7.30.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/slf4j-log4j12-1.7.30.jar
[INFO] Copying avro-1.9.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/avro-1.9.2.jar
[INFO] Copying jackson-core-2.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-core-2.13.2.jar
[INFO] Copying re2j-1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/re2j-1.1.jar
[INFO] Copying protobuf-java-2.5.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/protobuf-java-2.5.0.jar
[INFO] Copying gson-2.9.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/gson-2.9.0.jar
[INFO] Copying hadoop-auth-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-auth-3.4.0-SNAPSHOT.jar
[INFO] Copying nimbus-jose-jwt-9.8.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/nimbus-jose-jwt-9.8.1.jar
[INFO] Copying jcip-annotations-1.0-1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jcip-annotations-1.0-1.jar
[INFO] Copying json-smart-2.4.7.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/json-smart-2.4.7.jar
[INFO] Copying accessors-smart-2.4.7.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/accessors-smart-2.4.7.jar
[INFO] Copying asm-5.0.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/asm-5.0.4.jar
[INFO] Copying curator-framework-5.2.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/curator-framework-5.2.0.jar
[INFO] Copying jsch-0.1.55.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jsch-0.1.55.jar
[INFO] Copying curator-client-5.2.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/curator-client-5.2.0.jar
[INFO] Copying curator-recipes-5.2.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/curator-recipes-5.2.0.jar
[INFO] Copying jsr305-3.0.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jsr305-3.0.2.jar
[INFO] Copying zookeeper-3.6.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/zookeeper-3.6.3.jar
[INFO] Copying zookeeper-jute-3.6.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/zookeeper-jute-3.6.3.jar
[INFO] Copying audience-annotations-0.5.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/audience-annotations-0.5.0.jar
[INFO] Copying metrics-core-3.2.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/metrics-core-3.2.4.jar
[INFO] Copying commons-compress-1.21.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-compress-1.21.jar
[INFO] Copying kerb-core-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-core-1.0.1.jar
[INFO] Copying kerby-pkix-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerby-pkix-1.0.1.jar
[INFO] Copying kerby-asn1-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerby-asn1-1.0.1.jar
[INFO] Copying kerby-util-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerby-util-1.0.1.jar
[INFO] Copying jackson-databind-2.13.2.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-databind-2.13.2.2.jar
[INFO] Copying jackson-annotations-2.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-annotations-2.13.2.jar
[INFO] Copying stax2-api-4.2.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/stax2-api-4.2.1.jar
[INFO] Copying woodstox-core-5.3.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/woodstox-core-5.3.0.jar
[INFO] Copying dnsjava-3.4.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/dnsjava-3.4.0.jar
[INFO] Copying snappy-java-1.1.8.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/snappy-java-1.1.8.2.jar
[INFO] Copying hadoop-common-3.4.0-SNAPSHOT-tests.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-common-3.4.0-SNAPSHOT-tests.jar
[INFO] Copying aws-java-sdk-bundle-1.12.132.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/aws-java-sdk-bundle-1.12.132.jar
[INFO] Copying assertj-core-3.12.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/assertj-core-3.12.2.jar
[INFO] Copying wildfly-openssl-2.2.1.Final.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/wildfly-openssl-2.2.1.Final.jar
[INFO] Copying junit-4.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/junit-4.13.2.jar
[INFO] Copying hamcrest-core-1.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hamcrest-core-1.3.jar
[INFO] Copying mockito-core-2.28.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/mockito-core-2.28.2.jar
[INFO] Copying byte-buddy-1.9.10.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/byte-buddy-1.9.10.jar
[INFO] Copying byte-buddy-agent-1.9.10.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/byte-buddy-agent-1.9.10.jar
[INFO] Copying objenesis-2.6.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/objenesis-2.6.jar
[INFO] Copying hadoop-mapreduce-client-core-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-core-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-yarn-client-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-client-3.4.0-SNAPSHOT.jar
[INFO] Copying websocket-client-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/websocket-client-9.4.44.v20210927.jar
[INFO] Copying jetty-client-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-client-9.4.44.v20210927.jar
[INFO] Copying websocket-common-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/websocket-common-9.4.44.v20210927.jar
[INFO] Copying websocket-api-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/websocket-api-9.4.44.v20210927.jar
[INFO] Copying jline-3.9.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jline-3.9.0.jar
[INFO] Copying hadoop-yarn-common-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-common-3.4.0-SNAPSHOT.jar
[INFO] Copying jaxb-api-2.2.11.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jaxb-api-2.2.11.jar
[INFO] Copying jersey-client-1.19.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jersey-client-1.19.4.jar
[INFO] Copying guice-4.2.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/guice-4.2.3.jar
[INFO] Copying javax.inject-1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax.inject-1.jar
[INFO] Copying aopalliance-1.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/aopalliance-1.0.jar
[INFO] Copying jersey-guice-1.19.4.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jersey-guice-1.19.4.jar
[INFO] Copying jackson-module-jaxb-annotations-2.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-module-jaxb-annotations-2.13.2.jar
[INFO] Copying jakarta.xml.bind-api-2.3.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jakarta.xml.bind-api-2.3.3.jar
[INFO] Copying jackson-jaxrs-json-provider-2.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-jaxrs-json-provider-2.13.2.jar
[INFO] Copying jackson-jaxrs-base-2.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-jaxrs-base-2.13.2.jar
[INFO] Copying hadoop-hdfs-client-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-hdfs-client-3.4.0-SNAPSHOT.jar
[INFO] Copying okhttp-4.9.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/okhttp-4.9.3.jar
[INFO] Copying okio-2.8.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/okio-2.8.0.jar
[INFO] Copying kotlin-stdlib-1.4.10.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kotlin-stdlib-1.4.10.jar
[INFO] Copying kotlin-stdlib-common-1.4.10.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kotlin-stdlib-common-1.4.10.jar
[INFO] Copying javax.ws.rs-api-2.1.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax.ws.rs-api-2.1.1.jar
[INFO] Copying guice-servlet-4.2.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/guice-servlet-4.2.3.jar
[INFO] Copying netty-3.10.6.Final.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/netty-3.10.6.Final.jar
[INFO] Copying hadoop-yarn-server-tests-3.4.0-SNAPSHOT-tests.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-tests-3.4.0-SNAPSHOT-tests.jar
[INFO] Copying hadoop-yarn-server-common-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-common-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-registry-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-registry-3.4.0-SNAPSHOT.jar
[INFO] Copying geronimo-jcache_1.0_spec-1.0-alpha-1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar
[INFO] Copying ehcache-3.3.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/ehcache-3.3.1.jar
[INFO] Copying HikariCP-4.0.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/HikariCP-4.0.3.jar
[INFO] Copying mssql-jdbc-6.2.1.jre7.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/mssql-jdbc-6.2.1.jre7.jar
[INFO] Copying hadoop-yarn-server-nodemanager-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-nodemanager-3.4.0-SNAPSHOT.jar
[INFO] Copying javax-websocket-server-impl-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax-websocket-server-impl-9.4.44.v20210927.jar
[INFO] Copying jetty-annotations-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-annotations-9.4.44.v20210927.jar
[INFO] Copying jetty-plus-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-plus-9.4.44.v20210927.jar
[INFO] Copying jetty-jndi-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-jndi-9.4.44.v20210927.jar
[INFO] Copying javax.annotation-api-1.3.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax.annotation-api-1.3.2.jar
[INFO] Copying asm-commons-9.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/asm-commons-9.2.jar
[INFO] Copying asm-tree-9.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/asm-tree-9.2.jar
[INFO] Copying asm-analysis-9.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/asm-analysis-9.2.jar
[INFO] Copying javax-websocket-client-impl-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax-websocket-client-impl-9.4.44.v20210927.jar
[INFO] Copying javax.websocket-client-api-1.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax.websocket-client-api-1.0.jar
[INFO] Copying websocket-server-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/websocket-server-9.4.44.v20210927.jar
[INFO] Copying websocket-servlet-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/websocket-servlet-9.4.44.v20210927.jar
[INFO] Copying javax.websocket-api-1.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/javax.websocket-api-1.0.jar
[INFO] Copying jna-5.2.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jna-5.2.0.jar
[INFO] Copying hadoop-yarn-server-resourcemanager-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-resourcemanager-3.4.0-SNAPSHOT.jar
[INFO] Copying
hadoop-yarn-server-applicationhistoryservice-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-applicationhistoryservice-3.4.0-SNAPSHOT.jar
[INFO] Copying fst-2.50.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/fst-2.50.jar
[INFO] Copying java-util-1.9.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/java-util-1.9.0.jar
[INFO] Copying json-io-2.5.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/json-io-2.5.1.jar
[INFO] Copying jsonschema2pojo-core-1.0.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jsonschema2pojo-core-1.0.2.jar
[INFO] Copying codemodel-2.6.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/codemodel-2.6.jar
[INFO] Copying commons-lang-2.6.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-lang-2.6.jar
[INFO] Copying jackson-dataformat-yaml-2.13.2.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jackson-dataformat-yaml-2.13.2.jar
[INFO] Copying snakeyaml-1.26.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/snakeyaml-1.26.jar
[INFO] Copying hadoop-yarn-server-timelineservice-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-timelineservice-3.4.0-SNAPSHOT.jar
[INFO] Copying commons-csv-1.0.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-csv-1.0.jar
[INFO] Copying hadoop-yarn-api-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-api-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-mapreduce-client-hs-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-hs-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-hdfs-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-hdfs-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-mapreduce-client-common-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-common-3.4.0-SNAPSHOT.jar
[INFO] Copying leveldbjni-all-1.8.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/leveldbjni-all-1.8.jar
[INFO] Copying hadoop-mapreduce-examples-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-examples-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-hdfs-3.4.0-SNAPSHOT-tests.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-hdfs-3.4.0-SNAPSHOT-tests.jar
[INFO] Copying jetty-util-ajax-9.4.44.v20210927.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/jetty-util-ajax-9.4.44.v20210927.jar
[INFO] Copying commons-daemon-1.0.13.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/commons-daemon-1.0.13.jar
[INFO] Copying netty-all-4.1.68.Final.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/netty-all-4.1.68.Final.jar
[INFO] Copying hadoop-distcp-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-distcp-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-distcp-3.4.0-SNAPSHOT-tests.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-distcp-3.4.0-SNAPSHOT-tests.jar
[INFO] Copying hadoop-mapreduce-client-app-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-app-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-yarn-server-web-proxy-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-yarn-server-web-proxy-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-mapreduce-client-shuffle-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-shuffle-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-mapreduce-client-app-3.4.0-SNAPSHOT-tests.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-app-3.4.0-SNAPSHOT-tests.jar
[INFO] Copying hadoop-mapreduce-client-jobclient-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-jobclient-3.4.0-SNAPSHOT.jar
[INFO] Copying hadoop-mapreduce-client-jobclient-3.4.0-SNAPSHOT-tests.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-mapreduce-client-jobclient-3.4.0-SNAPSHOT-tests.jar
[INFO] Copying hadoop-minikdc-3.4.0-SNAPSHOT.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hadoop-minikdc-3.4.0-SNAPSHOT.jar
[INFO] Copying kerb-simplekdc-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-simplekdc-1.0.1.jar
[INFO] Copying kerb-client-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-client-1.0.1.jar
[INFO] Copying kerby-config-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerby-config-1.0.1.jar
[INFO] Copying kerb-common-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-common-1.0.1.jar
[INFO] Copying kerb-crypto-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-crypto-1.0.1.jar
[INFO] Copying kerb-util-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-util-1.0.1.jar
[INFO] Copying token-provider-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/token-provider-1.0.1.jar
[INFO] Copying kerb-admin-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-admin-1.0.1.jar
[INFO] Copying kerb-server-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-server-1.0.1.jar
[INFO] Copying kerb-identity-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerb-identity-1.0.1.jar
[INFO] Copying kerby-xdr-1.0.1.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/kerby-xdr-1.0.1.jar
[INFO] Copying hamcrest-library-1.3.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/hamcrest-library-1.3.jar
[INFO] Copying bcprov-jdk15on-1.68.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/bcprov-jdk15on-1.68.jar
[INFO] Copying bcpkix-jdk15on-1.68.jar to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/lib/bcpkix-jdk15on-1.68.jar
[INFO]
[INFO] --- maven-site-plugin:3.11.0:attach-descriptor (attach-descriptor) @
hadoop-aws ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test
(default-integration-test) @ hadoop-aws ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractDistCp
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer
[INFO] Running
org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek
[INFO] Running
org.apache.hadoop.fs.contract.s3a.ITestS3AContractGetFileStatus
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
40.391 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractMkdir
[WARNING] Tests run: 17, Failures: 0, Errors: 0, Skipped: 17, Time elapsed:
55.794 s - in
org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractDelete
[WARNING] Tests run: 16, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
106.574 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
[INFO] Running
org.apache.hadoop.fs.contract.s3a.ITestS3AContractContentSummary
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
58.799 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractDelete
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
116.207 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractOpen
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
32.144 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractContentSummary
[INFO] Running org.apache.hadoop.fs.s3a.ITestLocatedFileStatusFetcher
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
23.567 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
100.191 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractMkdir
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials
[INFO] Running
org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.746
s - in org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
[INFO] Running
org.apache.hadoop.fs.s3a.statistics.ITestAWSStatisticCollection
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.099 s - in org.apache.hadoop.fs.s3a.statistics.ITestAWSStatisticCollection
[INFO] Running
org.apache.hadoop.fs.s3a.statistics.ITestS3AContractStreamIOStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
41.225 s - in org.apache.hadoop.fs.s3a.ITestLocatedFileStatusFetcher
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestStagingCommitProtocol
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
31.903 s - in
org.apache.hadoop.fs.s3a.statistics.ITestS3AContractStreamIOStatistics
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestDirectoryCommitProtocol
[INFO] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
85.009 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractOpen
[INFO] Running
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestPartitionedCommitProtocol
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
214.479 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractGetFileStatus
[INFO] Running org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitProtocol
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
248.811 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead
[INFO] Running
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob
[WARNING] Tests run: 12, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
335.192 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractDistCp
[INFO] Running org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.47
s - in org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory
[INFO] Running org.apache.hadoop.fs.s3a.commit.ITestCommitOperations
[ERROR] Tests run: 14, Failures: 0, Errors: 1, Skipped: 4, Time elapsed:
213.419 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials
[ERROR] testSTS(org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials) Time
elapsed: 11.626 s <<< ERROR!
com.amazonaws.SdkClientException: Unable to find a region via the region
provider chain. Must provide an explicit region in the builder or setup
environment to supply a region.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:462)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials.testSTS(ITestS3ATemporaryCredentials.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[INFO] Running org.apache.hadoop.fs.s3a.commit.ITestCommitOperationCost
[ERROR] Tests run: 12, Failures: 3, Errors: 0, Skipped: 0, Time elapsed:
124.126 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob
[ERROR]
test_200_execute[directory](org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob)
Time elapsed: 20.406 s <<< FAILURE!
java.lang.AssertionError:
Job job_1657332805778_0001 failed in state FAILED with cause Job setup
failed : java.nio.file.AccessDeniedException:
s3a://ashugpt-emr-dev-blr-itest/fork-0008/test/ITestS3ACommitterMRJob-execute-directory:
org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials
provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider
EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider :
com.amazonaws.SdkClientException: Unable to load AWS credentials from
environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY
(or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:172)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3565)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3468)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getFileStatus$24(S3AFileSystem.java:3445)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2336)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2355)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3443)
at
org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter.setupJob(DirectoryStagingCommitter.java:74)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS
Credentials provided by TemporaryAWSCredentialsProvider
SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider
IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to
load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or
AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:216)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6428)
at
com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6401)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5438)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$11(S3AFileSystem.java:2506)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2494)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2474)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3540)
... 14 more
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
com.amazonaws.auth.EnvironmentVariableCredentialsProvider.getCredentials(EnvironmentVariableCredentialsProvider.java:50)
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:177)
... 35 more
.
Consult logs under
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/test-dir/8/yarn-2022-07-09-03.13.21.65/yarn-13195147
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob.test_200_execute(ITestS3ACommitterMRJob.java:295)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
test_200_execute[partitioned](org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob)
Time elapsed: 16.402 s <<< FAILURE!
java.lang.AssertionError:
Job job_1657332805778_0002 failed in state FAILED with cause Job setup
failed : java.nio.file.AccessDeniedException:
s3a://ashugpt-emr-dev-blr-itest/fork-0008/test/ITestS3ACommitterMRJob-execute-partitioned/_SUCCESS:
org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials
provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider
EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider :
com.amazonaws.SdkClientException: Unable to load AWS credentials from
environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY
(or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:172)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3565)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3468)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:3070)
at
org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.deleteSuccessMarker(CommitOperations.java:455)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.setupJob(AbstractS3ACommitter.java:600)
at
org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter.setupJob(StagingCommitter.java:430)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS
Credentials provided by TemporaryAWSCredentialsProvider
SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider
IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to
load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or
AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:216)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6428)
at
com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6401)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5438)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$11(S3AFileSystem.java:2506)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2494)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2474)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3540)
... 10 more
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
com.amazonaws.auth.EnvironmentVariableCredentialsProvider.getCredentials(EnvironmentVariableCredentialsProvider.java:50)
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:177)
... 31 more
.
Consult logs under
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/test-dir/8/yarn-2022-07-09-03.13.21.65/yarn-13195147
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob.test_200_execute(ITestS3ACommitterMRJob.java:295)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
test_200_execute[magic](org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob)
Time elapsed: 18.506 s <<< FAILURE!
java.lang.AssertionError:
Job job_1657332805778_0003 failed in state FAILED with cause Job setup
failed : java.nio.file.AccessDeniedException:
s3a://ashugpt-emr-dev-blr-itest/fork-0008/test/ITestS3ACommitterMRJob-execute-magic/_SUCCESS:
org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials
provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider
EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider :
com.amazonaws.SdkClientException: Unable to load AWS credentials from
environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY
(or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:172)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3565)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3468)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:3070)
at
org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.deleteSuccessMarker(CommitOperations.java:455)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.setupJob(AbstractS3ACommitter.java:600)
at
org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter.setupJob(MagicS3GuardCommitter.java:104)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS
Credentials provided by TemporaryAWSCredentialsProvider
SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider
IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to
load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or
AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:216)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6428)
at
com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6401)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5438)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$11(S3AFileSystem.java:2506)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2494)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2474)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3540)
... 10 more
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
com.amazonaws.auth.EnvironmentVariableCredentialsProvider.getCredentials(EnvironmentVariableCredentialsProvider.java:50)
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:177)
... 31 more
.
Consult logs under
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/test-dir/8/yarn-2022-07-09-03.13.21.65/yarn-13195147
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.s3a.commit.integration.ITestS3ACommitterMRJob.test_200_execute(ITestS3ACommitterMRJob.java:295)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestRenameDeleteRace
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
29.702 s - in org.apache.hadoop.fs.s3a.commit.ITestCommitOperationCost
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestPartialRenamesDeletes
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
18.838 s - in org.apache.hadoop.fs.s3a.impl.ITestRenameDeleteRace
[INFO] Running org.apache.hadoop.fs.s3a.impl.ITestXAttrCost
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
17.266 s - in org.apache.hadoop.fs.s3a.impl.ITestXAttrCost
[INFO] Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI
[WARNING] Tests run: 26, Failures: 0, Errors: 0, Skipped: 26, Time elapsed:
57.353 s - in org.apache.hadoop.fs.s3a.impl.ITestPartialRenamesDeletes
[INFO] Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContext
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312
s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContext
[INFO] Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextUtil
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
109.384 s - in org.apache.hadoop.fs.s3a.commit.ITestCommitOperations
[INFO] Running
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextCreateMkdir
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
27.199 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextUtil
[INFO] Running
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations
[WARNING] Tests run: 22, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
286.83 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestPartitionedCommitProtocol
[INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
306.805 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestStagingCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABucketExistence
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSES3
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.55
s - in org.apache.hadoop.fs.s3a.ITestS3ABucketExistence
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlocksize
[INFO] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
322.362 s - in
org.apache.hadoop.fs.s3a.commit.staging.integration.ITestDirectoryCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.ITestDowngradeSyncable
[WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 24, Time elapsed:
525.202 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek
[INFO] Running org.apache.hadoop.fs.s3a.tools.ITestMarkerTool
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.726 s - in org.apache.hadoop.fs.s3a.ITestS3ABlocksize
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFailureHandling
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.461 s - in org.apache.hadoop.fs.s3a.ITestDowngradeSyncable
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteFilesOneByOne
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
4.763 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteFilesOneByOne
[INFO] Running
org.apache.hadoop.fs.s3a.scale.ITestS3AMultipartUploadSizeLimits
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
87.936 s - in
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextCreateMkdir
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
59.811 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSES3
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ACreatePerformance
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
17.822 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AMultipartUploadSizeLimits
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
5.534 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ACreatePerformance
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps
[WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed:
22.98 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteManyFiles
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
8.885 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AConcurrentOps
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AContentEncoding
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
5.191 s - in org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteManyFiles
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ADelayedFNF
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.869 s - in org.apache.hadoop.fs.s3a.ITestS3AContentEncoding
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AUnbuffer
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.08
s - in org.apache.hadoop.fs.s3a.ITestS3ADelayedFNF
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestJceksIO
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 8, Time elapsed:
28.149 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
54.733 s - in org.apache.hadoop.fs.s3a.ITestS3AFailureHandling
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestCustomSigner
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestRestrictedReadAccess
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
5.137 s - in org.apache.hadoop.fs.s3a.auth.ITestRestrictedReadAccess
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestAssumeRole
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.639 s - in org.apache.hadoop.fs.s3a.ITestS3AUnbuffer
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationTokens
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.782
s - in org.apache.hadoop.fs.s3a.auth.ITestCustomSigner
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFileystem
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
16.694 s - in org.apache.hadoop.fs.s3a.auth.ITestJceksIO
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationInFileystem
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
17.347 s - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationTokens
[INFO] Running
org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationTokens
[WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 7, Time elapsed:
23.265 s - in org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationTokens
[INFO] Running org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 11, Time elapsed:
36.966 s - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationInFileystem
[INFO] Running org.apache.hadoop.fs.s3a.auth.ITestAssumedRoleCommitOperations
[WARNING] Tests run: 17, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
223.452 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AMetrics
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.769 s - in org.apache.hadoop.fs.s3a.ITestS3AMetrics
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSDefaultKey
[ERROR] Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
56.217 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFileystem
[ERROR]
testDelegationBindingMismatch2(org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFileystem)
Time elapsed: 3.096 s <<< ERROR!
org.apache.hadoop.service.ServiceStateException:
org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException: no
credentials in configuration or environment variables: No AWS credentials
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:204)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:857)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:522)
at
org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationIT.newS3AInstance(AbstractDelegationIT.java:103)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationInFileystem.testDelegationBindingMismatch2(ITestSessionDelegationInFileystem.java:511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by:
org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException: no
credentials in configuration or environment variables: No AWS credentials
at
org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.validate(MarshalledCredentials.java:336)
at
org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding.loadAWSCredentials(FullCredentialsTokenBinding.java:107)
at
org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding.deployUnbonded(FullCredentialsTokenBinding.java:120)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.deployUnbonded(S3ADelegationTokens.java:251)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.bindToAnyDelegationToken(S3ADelegationTokens.java:284)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.serviceStart(S3ADelegationTokens.java:205)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:195)
... 19 more
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AStorageClass
[ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 2, Time elapsed:
28.99 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob
[ERROR]
testCommonCrawlLookup[1](org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob)
Time elapsed: 0.107 s <<< ERROR!
org.apache.hadoop.service.ServiceStateException:
org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException: no
credentials in configuration or environment variables: No AWS credentials
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:204)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:857)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:522)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3563)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:172)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3668)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3615)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:556)
at
org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:189)
at
org.apache.hadoop.fs.s3a.AbstractS3ATestBase.setup(AbstractS3ATestBase.java:100)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob.setup(ITestDelegatedMRJob.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by:
org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException: no
credentials in configuration or environment variables: No AWS credentials
at
org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.validate(MarshalledCredentials.java:336)
at
org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding.loadAWSCredentials(FullCredentialsTokenBinding.java:107)
at
org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding.deployUnbonded(FullCredentialsTokenBinding.java:120)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.deployUnbonded(S3ADelegationTokens.java:251)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.bindToAnyDelegationToken(S3ADelegationTokens.java:284)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.serviceStart(S3ADelegationTokens.java:205)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:195)
... 26 more
[ERROR]
testJobSubmissionCollectsTokens[1](org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob)
Time elapsed: 0.099 s <<< ERROR!
org.apache.hadoop.service.ServiceStateException:
org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException: no
credentials in configuration or environment variables: No AWS credentials
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:204)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:857)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:522)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3563)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:172)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3668)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3615)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:556)
at
org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:189)
at
org.apache.hadoop.fs.s3a.AbstractS3ATestBase.setup(AbstractS3ATestBase.java:100)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob.setup(ITestDelegatedMRJob.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by:
org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException: no
credentials in configuration or environment variables: No AWS credentials
at
org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.validate(MarshalledCredentials.java:336)
at
org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding.loadAWSCredentials(FullCredentialsTokenBinding.java:107)
at
org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding.deployUnbonded(FullCredentialsTokenBinding.java:120)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.deployUnbonded(S3ADelegationTokens.java:251)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.bindToAnyDelegationToken(S3ADelegationTokens.java:284)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.serviceStart(S3ADelegationTokens.java:205)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:195)
... 26 more
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AConfiguration
[WARNING] Tests run: 25, Failures: 0, Errors: 0, Skipped: 25, Time elapsed:
76.937 s - in org.apache.hadoop.fs.s3a.auth.ITestAssumeRole
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AClosedFS
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.447
s - in org.apache.hadoop.fs.s3a.ITestS3AClosedFS
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AClientSideEncryptionKms
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.626 s - in org.apache.hadoop.fs.s3a.ITestS3AConfiguration
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AMultipartUtils
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.658 s - in org.apache.hadoop.fs.s3a.ITestS3AMultipartUtils
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ACopyFromLocalFile
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
16.433 s - in org.apache.hadoop.fs.s3a.ITestS3AClientSideEncryptionKms
[WARNING] Tests run: 18, Failures: 0, Errors: 0, Skipped: 18, Time elapsed:
57.47 s - in org.apache.hadoop.fs.s3a.auth.ITestAssumedRoleCommitOperations
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ACannedACLs
[INFO] Running org.apache.hadoop.fs.s3a.yarn.ITestS3AMiniYarnCluster
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
43.659 s - in org.apache.hadoop.fs.s3a.ITestS3AStorageClass
[INFO] Running org.apache.hadoop.fs.s3a.yarn.ITestS3A
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.127 s - in org.apache.hadoop.fs.s3a.ITestS3ACannedACLs
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.379
s - in org.apache.hadoop.fs.s3a.yarn.ITestS3A
[INFO] Running
org.apache.hadoop.fs.s3a.ITestBlockingThreadPoolExecutorService
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputByteBuffer
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.832
s - in org.apache.hadoop.fs.s3a.ITestBlockingThreadPoolExecutorService
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ARequesterPays
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
61.934 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSDefaultKey
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
21.826 s - in org.apache.hadoop.fs.s3a.yarn.ITestS3AMiniYarnCluster
[INFO] Running org.apache.hadoop.fs.s3a.audit.ITestAuditAccessChecks
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.43
s - in org.apache.hadoop.fs.s3a.ITestS3ARequesterPays
[INFO] Running org.apache.hadoop.fs.s3a.audit.ITestAuditManager
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
199.136 s - in org.apache.hadoop.fs.s3a.tools.ITestMarkerTool
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.782
s - in org.apache.hadoop.fs.s3a.audit.ITestAuditManager
[INFO] Running org.apache.hadoop.fs.s3a.audit.ITestAuditManagerDisabled
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSUserDefinedKey
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
0.541 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSUserDefinedKey
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputDisk
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.733
s - in org.apache.hadoop.fs.s3a.audit.ITestAuditManagerDisabled
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputArray
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
24.126 s - in org.apache.hadoop.fs.s3a.audit.ITestAuditAccessChecks
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AMiscOperations
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
38.025 s - in org.apache.hadoop.fs.s3a.ITestS3ABlockOutputByteBuffer
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract
[WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
37.833 s - in org.apache.hadoop.fs.s3a.ITestS3ABlockOutputDisk
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
38.247 s - in org.apache.hadoop.fs.s3a.ITestS3ABlockOutputArray
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3ATestUtils
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.148
s - in org.apache.hadoop.fs.s3a.ITestS3ATestUtils
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.533
s - in org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AFSMainOperations
[INFO] Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
101.617 s - in org.apache.hadoop.fs.s3a.ITestS3ACopyFromLocalFile
[INFO] Running org.apache.hadoop.fs.s3a.select.ITestS3SelectMRJob
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
68.757 s - in org.apache.hadoop.fs.s3a.ITestS3AMiscOperations
[INFO] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
594.266 s - in org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitProtocol
[INFO] Running org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat
[INFO] Running org.apache.hadoop.fs.s3a.select.ITestS3SelectCLI
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
25.064 s - in org.apache.hadoop.fs.s3a.select.ITestS3SelectMRJob
[INFO] Running org.apache.hadoop.fs.s3a.select.ITestS3Select
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
70.442 s - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
43.857 s - in org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3AMkdirCost
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
16.704 s - in org.apache.hadoop.fs.s3a.performance.ITestS3AOpenCost
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3ARenameCost
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
45.862 s - in org.apache.hadoop.fs.s3a.select.ITestS3SelectCLI
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3AMiscOperationCost
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.54
s - in org.apache.hadoop.fs.s3a.performance.ITestS3AMiscOperationCost
[INFO] Running
org.apache.hadoop.fs.s3a.performance.ITestDirectoryMarkerListing
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.96
s - in org.apache.hadoop.fs.s3a.performance.ITestS3AMkdirCost
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestCreateFileCost
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.41
s - in org.apache.hadoop.fs.s3a.performance.ITestS3ARenameCost
[INFO] Running org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost
[INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
213.801 s - in org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEmptyDirectory
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
20.286 s - in org.apache.hadoop.fs.s3a.ITestS3AEmptyDirectory
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
61.433 s - in org.apache.hadoop.fs.s3a.performance.ITestCreateFileCost
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
18.175 s - in org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionAlgorithmValidation
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
0.026 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionAlgorithmValidation
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3AEncryptionWithDefaultS3Settings
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
81.142 s - in org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
8.87 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionWithDefaultS3Settings
[WARNING] Tests run: 43, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
283.275 s - in org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
211.511 s - in org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List
[WARNING] Tests run: 73, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
703.526 s - in
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
386.416 s - in org.apache.hadoop.fs.s3a.select.ITestS3Select
[INFO] Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
314.899 s - in org.apache.hadoop.fs.s3a.performance.ITestDirectoryMarkerListing
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
475.876 s - in org.apache.hadoop.fs.s3a.ITestS3AFSMainOperations
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestS3ACommitterMRJob.test_200_execute:295->Assert.fail:89 Job
job_1657332805778_0001 failed in state FAILED with cause Job setup failed :
java.nio.file.AccessDeniedException:
s3a://ashugpt-emr-dev-blr-itest/fork-0008/test/ITestS3ACommitterMRJob-execute-directory:
org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials
provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider
EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider :
com.amazonaws.SdkClientException: Unable to load AWS credentials from
environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY
(or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:172)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3565)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3468)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getFileStatus$24(S3AFileSystem.java:3445)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2336)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2355)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3443)
at
org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter.setupJob(DirectoryStagingCommitter.java:74)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS
Credentials provided by TemporaryAWSCredentialsProvider
SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider
IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to
load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or
AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:216)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6428)
at
com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6401)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5438)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$11(S3AFileSystem.java:2506)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2494)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2474)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3540)
... 14 more
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
com.amazonaws.auth.EnvironmentVariableCredentialsProvider.getCredentials(EnvironmentVariableCredentialsProvider.java:50)
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:177)
... 35 more
.
Consult logs under
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/test-dir/8/yarn-2022-07-09-03.13.21.65/yarn-13195147
[ERROR] ITestS3ACommitterMRJob.test_200_execute:295->Assert.fail:89 Job
job_1657332805778_0003 failed in state FAILED with cause Job setup failed :
java.nio.file.AccessDeniedException:
s3a://ashugpt-emr-dev-blr-itest/fork-0008/test/ITestS3ACommitterMRJob-execute-magic/_SUCCESS:
org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials
provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider
EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider :
com.amazonaws.SdkClientException: Unable to load AWS credentials from
environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY
(or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:172)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3565)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3468)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:3070)
at
org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.deleteSuccessMarker(CommitOperations.java:455)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.setupJob(AbstractS3ACommitter.java:600)
at
org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter.setupJob(MagicS3GuardCommitter.java:104)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS
Credentials provided by TemporaryAWSCredentialsProvider
SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider
IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to
load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or
AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:216)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6428)
at
com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6401)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5438)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$11(S3AFileSystem.java:2506)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2494)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2474)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3540)
... 10 more
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
com.amazonaws.auth.EnvironmentVariableCredentialsProvider.getCredentials(EnvironmentVariableCredentialsProvider.java:50)
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:177)
... 31 more
.
Consult logs under
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/test-dir/8/yarn-2022-07-09-03.13.21.65/yarn-13195147
[ERROR] ITestS3ACommitterMRJob.test_200_execute:295->Assert.fail:89 Job
job_1657332805778_0002 failed in state FAILED with cause Job setup failed :
java.nio.file.AccessDeniedException:
s3a://ashugpt-emr-dev-blr-itest/fork-0008/test/ITestS3ACommitterMRJob-execute-partitioned/_SUCCESS:
org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials
provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider
EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider :
com.amazonaws.SdkClientException: Unable to load AWS credentials from
environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY
(or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:172)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3565)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3468)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:3070)
at
org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.deleteSuccessMarker(CommitOperations.java:455)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.setupJob(AbstractS3ACommitter.java:600)
at
org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter.setupJob(StagingCommitter.java:430)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS
Credentials provided by TemporaryAWSCredentialsProvider
SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider
IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to
load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or
AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:216)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1266)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:842)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:792)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6428)
at
com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6401)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5438)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$11(S3AFileSystem.java:2506)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:431)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2494)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2474)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3540)
... 10 more
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
at
com.amazonaws.auth.EnvironmentVariableCredentialsProvider.getCredentials(EnvironmentVariableCredentialsProvider.java:50)
at
org.apache.hadoop.fs.s3a.AWSCredentialProviderList.getCredentials(AWSCredentialProviderList.java:177)
... 31 more
.
Consult logs under
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/test-dir/8/yarn-2022-07-09-03.13.21.65/yarn-13195147
[ERROR] Errors:
[ERROR] ITestS3ATemporaryCredentials.testSTS:130 » SdkClient Unable to
find a region v...
[ERROR]
ITestDelegatedMRJob.setup:182->AbstractS3ATestBase.setup:100->AbstractFSContractTestBase.setup:189
» ServiceState
[ERROR]
ITestDelegatedMRJob.setup:182->AbstractS3ATestBase.setup:100->AbstractFSContractTestBase.setup:189
» ServiceState
[ERROR]
ITestSessionDelegationInFileystem.testDelegationBindingMismatch2:511->AbstractDelegationIT.newS3AInstance:103
» ServiceState
[INFO]
[ERROR] Tests run: 1128, Failures: 3, Errors: 4, Skipped: 190
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test
(sequential-integration-tests) @ hadoop-aws ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractRootDir
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
39.939 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRootDir
[INFO] Running org.apache.hadoop.fs.s3a.statistics.ITestAggregateIOStatistics
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.553
s - in org.apache.hadoop.fs.s3a.statistics.ITestAggregateIOStatistics
[INFO] Running org.apache.hadoop.fs.s3a.commit.magic.ITestS3AHugeMagicCommits
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
16.881 s - in org.apache.hadoop.fs.s3a.commit.magic.ITestS3AHugeMagicCommits
[INFO] Running org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A
[WARNING] Tests run: 14, Failures: 0, Errors: 0, Skipped: 14, Time elapsed:
27.711 s - in org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A
[INFO] Running
org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
16.282 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics
[INFO] Running org.apache.hadoop.fs.s3a.tools.ITestMarkerToolRootOperations
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.563
s - in org.apache.hadoop.fs.s3a.tools.ITestMarkerToolRootOperations
[INFO] Running
org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesByteBufferBlocks
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
16.987 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesByteBufferBlocks
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesArrayBlocks
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
16.918 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesArrayBlocks
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesEncryption
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
0.195 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesEncryption
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesStorageClass
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
16.788 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesStorageClass
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesSSECDiskBlocks
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
16.748 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesSSECDiskBlocks
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesDiskBlocks
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
17.04 s - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesDiskBlocks
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEC
[INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
197.267 s - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEC
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 124, Failures: 0, Errors: 0, Skipped: 84
[INFO]
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-aws ---
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:verify (default-integration-test)
@ hadoop-aws ---
[ERROR] There are test failures.
Please refer to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/failsafe-reports
for the individual test results.
Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump
and [date].dumpstream.
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:verify
(sequential-integration-tests) @ hadoop-aws ---
[ERROR] There are test failures.
Please refer to
/Users/ashugpt/Documents/OSS/hadoop/hadoop-tools/hadoop-aws/target/failsafe-reports
for the individual test results.
Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump
and [date].dumpstream.
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 29:09 min
[INFO] Finished at: 2022-07-09T03:36:38+01:00
[INFO]
------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
