CLevasseur commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1330911612
The exception for
`ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS` is:
```
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 10, Time elapsed:
1.521 s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR]
testCheckAccessForAccountWithoutNS(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess)
Time elapsed: 1.15 s <<< FAILURE!
java.lang.AssertionError: Expecting
org.apache.hadoop.security.AccessControlException with text "This request is
not authorized to perform this operation using this permission.", 403 but got :
"void"
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:529)
at
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS(ITestAzureBlobFileSystemCheckAccess.java:181)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:829)
```
> are you working with a hierarchical store here? as i think those tests are
failures from that detail, not your change (essentially: etags aren't being
preserved across rename).
No, HNS is disabled on Azure's side, and in the account settings as well:
````
cat ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml | grep
-B1 -A2 namespace
<property>
<name>fs.azure.test.namespace.enabled.****.dfs.core.windows.net</name>
<value>false</value>
</property>
<property>
<name>fs.azure.test.namespace.enabled</name>
<value>false</value>
</property>
```
> do the same test suite against trunk and see if the tests still fail, if
so, create a new jira (or maybe there is one already). it shouldn't block this
pr
The result is the same, here are the logs I'm getting:
```
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.976
s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.174
s - in org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
[ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
2.921 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
[ERROR]
testConfigPropNotFound(org.apache.hadoop.fs.azurebfs.TestAccountConfiguration)
Time elapsed: 0.102 s <<< FAILURE!
java.lang.AssertionError: Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384)
at
org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testMissingConfigKey(TestAccountConfiguration.java:399)
at
org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testConfigPropNotFound(TestAccountConfiguration.java:386)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.396
s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestTextFileBasedIdentityHandler
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.043
s - in org.apache.hadoop.fs.azurebfs.services.TestTextFileBasedIdentityHandler
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestQueryParams
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.2 s
- in org.apache.hadoop.fs.azurebfs.services.TestQueryParams
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.088
s - in org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsInputStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestCachedSASToken
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.651
s - in org.apache.hadoop.fs.azurebfs.utils.TestCachedSASToken
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
8.133 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.278
s - in org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy
[INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.713 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.861
s - in org.apache.hadoop.fs.azurebfs.TestAbfsInputStreamStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.668
s - in org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsOutputStreamStatistics
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.598
s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery
[INFO] Running
org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsNetworkStatistics
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.322 s - in
org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
[INFO] Running org.apache.hadoop.fs.azurebfs.TestTracingContext
[INFO] Running
org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.61
s - in org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.534
s - in org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation
[INFO] Running
org.apache.hadoop.fs.azurebfs.extensions.TestCustomOauthTokenProvider
[INFO] Running
org.apache.hadoop.fs.azurebfs.extensions.TestDTManagerLifecycle
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.441
s - in org.apache.hadoop.fs.azurebfs.TestAbfsOutputStreamStatistics
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.849
s - in org.apache.hadoop.fs.azurebfs.extensions.TestCustomOauthTokenProvider
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.786
s - in org.apache.hadoop.fs.azurebfs.TestAbfsNetworkStatistics
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
6.956 s - in org.apache.hadoop.fs.azurebfs.TestTracingContext
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.59
s - in org.apache.hadoop.fs.azurebfs.extensions.TestDTManagerLifecycle
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsCrc64
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.061
s - in org.apache.hadoop.fs.azurebfs.TestAbfsCrc64
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.686
s - in org.apache.hadoop.fs.azurebfs.TestAbfsStatistics
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
36.877 s - in
org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
55.459 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
[INFO]
[ERROR] There are test failures.
Please refer to /hadoop/hadoop-tools/hadoop-azure/target/surefire-reports
for the individual test results.
Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump
and [date].dumpstream.
[INFO]
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-azure ---
[INFO]
[INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-azure ---
[INFO] Building jar:
/hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.4.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-site-plugin:3.11.0:attach-descriptor (attach-descriptor) @
hadoop-azure ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test
(integration-test-abfs-parallel-classesAndMethods) @ hadoop-azure ---
Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.jar
Downloaded from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.jar
(93 kB at 2.4 MB/s)
Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.pom
Downloaded from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.pom
(7.3 kB at 292 kB/s)
Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.pom
Downloaded from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.pom
(2.4 kB at 70 kB/s)
Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.pom
Downloaded from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.pom
(2.6 kB at 146 kB/s)
Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.jar
Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.jar
Downloaded from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.jar
(22 kB at 1.3 MB/s)
Downloaded from central:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.jar
(40 kB at 1.8 MB/s)
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsRestOperationException
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.294
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsRestOperationException
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.288
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.765
s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCLI
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.222
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCLI
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.588
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsPositionedRead
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.635
s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsPositionedRead
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.634 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsUnbuffer
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.496 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsUnbuffer
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.503
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.634
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAuthorization
[WARNING] Tests run: 29, Failures: 0, Errors: 0, Skipped: 29, Time elapsed:
2.638 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAuthorization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestOauthOverAbfsScheme
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
0.277 s - in org.apache.hadoop.fs.azurebfs.ITestOauthOverAbfsScheme
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.317
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsDurationTrackers
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.97
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsDurationTrackers
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
19.973 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsClient
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
19.652 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
[WARNING] Tests run: 66, Failures: 0, Errors: 0, Skipped: 56, Time elapsed:
11.749 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.013
s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
0.099 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
0.549 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsOutputStreamStatistics
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.074
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsOutputStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsIdentityTransformer
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.745
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsIdentityTransformer
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
23.404 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.549
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemStoreListStatusWithRange
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.767
s - in
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemStoreListStatusWithRange
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 10, Time elapsed:
1.521 s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR]
testCheckAccessForAccountWithoutNS(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess)
Time elapsed: 1.15 s <<< FAILURE!
java.lang.AssertionError: Expecting
org.apache.hadoop.security.AccessControlException with text "This request is
not authorized to perform this operation using this permission.", 403 but got :
"void"
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:529)
at
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS(ITestAzureBlobFileSystemCheckAccess.java:181)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:829)
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.827
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsStatistics
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.649
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.393
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMainOperation
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
5.373 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMainOperation
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.362
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
3.196 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
27.622 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsHugeFiles
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.34
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsHugeFiles
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.802
s - in org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.604
s - in org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.775
s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestSharedKeyAuth
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186
s - in org.apache.hadoop.fs.azurebfs.ITestSharedKeyAuth
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.474
s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestCustomerProvidedKey
[WARNING] Tests run: 28, Failures: 0, Errors: 0, Skipped: 28, Time elapsed:
0.882 s - in org.apache.hadoop.fs.azurebfs.ITestCustomerProvidedKey
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAttributes
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
0.402 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAttributes
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelegationSAS
[WARNING] Tests run: 14, Failures: 0, Errors: 0, Skipped: 14, Time elapsed:
0.41 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelegationSAS
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsMsiTokenProvider
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
0.298 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsMsiTokenProvider
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.921
s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestGetNameSpaceEnabled
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
6.37 s - in org.apache.hadoop.fs.azurebfs.ITestGetNameSpaceEnabled
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAbfsListStatusRemoteIterator
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.053 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsListStatusRemoteIterator
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
[WARNING] Tests run: 128, Failures: 0, Errors: 0, Skipped: 128, Time
elapsed: 9.17 s - in
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamSmallFileReads
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
48.622 s - in
org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamSmallFileReads
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStream
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
48.918 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStream
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamReadFooter
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
58.717 s - in
org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamReadFooter
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
75.544 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:181
Expecting org.apache.hadoop.security.AccessControlException with text "This
request is not authorized to perform this operation using this permission.",
403 but got : "void"
[INFO]
[ERROR] Tests run: 567, Failures: 1, Errors: 0, Skipped: 277
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test
(integration-test-abfs-parallel-classes) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsLoadManifestsStage
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCleanupStage
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.918 s - in org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCommitTaskStage
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
16.879 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
19.657 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCleanupStage
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTaskManifestFileIO
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCreateOutputDirectoriesStage
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
24.991 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsLoadManifestsStage
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
28.569 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
16.159 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCommitTaskStage
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort
[INFO] Running
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestStoreOperations
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.197 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTaskManifestFileIO
[ERROR] Tests run: 19, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
32.672 s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter
[ERROR]
test_0420_validateJob(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter)
Time elapsed: 0.74 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0002/test/TestJobThroughManifestCommitter/out%20put/dir-03-00/dir-02-00/dir-01-00/part-00-0033':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0002/test/TestJobThroughManifestCommitter/out
put/_temporary/202211291549220355_0002/01/tasks/attempt_202211291549220355_0002_m_000000_1/dir-03-00/dir-02-00/dir-01-00/part-00-0033
with etag 0x8DAD22153D4F458 and length 2 but found a file with etag
0x8DAD22159994F7E and length 2
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
33.371 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.86
s - in
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCreateOutputDirectoriesStage
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed:
8.559 s - in
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestStoreOperations
[WARNING] Tests run: 48, Failures: 0, Errors: 0, Skipped: 24, Time elapsed:
41.978 s - in org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.96
s - in org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed:
9.624 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
14.207 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsContractUnbuffer
[ERROR] Tests run: 17, Failures: 0, Errors: 7, Skipped: 0, Time elapsed:
25.661 s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol
[ERROR]
testCommitterWithDuplicatedCommit(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 10.143 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitterWithDuplicatedCommit/part-m-00000':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitterWithDuplicatedCommit/_temporary/job_202211291549500881_0004/01/tasks/attempt_202211291549500881_0004_m_000000_0/part-m-00000
with etag 0x8DAD2215D8BA025 and length 40 but found a file with etag
0x8DAD2215F25DA02 and length 40
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR]
testMapFileOutputCommitter(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 1.857 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testMapFileOutputCommitter/part-m-00000/data':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testMapFileOutputCommitter/_temporary/job_202211291550000055_0004/01/tasks/attempt_202211291550000055_0004_m_000000_0/part-m-00000/data
with etag 0x8DAD2215FF58EA2 and length 296 but found a file with etag
0x8DAD221603B8E0D and length 296
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR]
testCommitLifecycle(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 1.462 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitLifecycle/part-m-00000':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitLifecycle/_temporary/job_202211291550020019_0004/01/tasks/attempt_202211291550020019_0004_m_000000_0/part-m-00000
with etag 0x8DAD22160F41493 and length 40 but found a file with etag
0x8DAD22161466E35 and length 40
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR]
testConcurrentCommitTaskWithSubDir(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 1.262 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testConcurrentCommitTaskWithSubDir/SUB_DIR/part-m-00000':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testConcurrentCommitTaskWithSubDir/_temporary/job_202211291550030237_0004/00/tasks/attempt_202211291550030237_0004_m_000000_1/SUB_DIR/part-m-00000
with etag 0x8DAD22161DB6C66 and length 40 but found a file with etag
0x8DAD22162100941 and length 40
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR]
testParallelJobsToAdjacentPaths(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 1.401 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testParallelJobsToAdjacentPaths/part-m-00000':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testParallelJobsToAdjacentPaths/_temporary/job_202211291550080667_0004/01/tasks/attempt_202211291550080667_0004_m_000000_0/part-m-00000
with etag 0x8DAD221646E60AB and length 40 but found a file with etag
0x8DAD22164B4AE25 and length 40
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR]
testOutputFormatIntegration(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 0.925 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testOutputFormatIntegration/part-m-00000':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testOutputFormatIntegration/_temporary/job_202211291550110055_0004/01/tasks/attempt_202211291550110055_0004_m_000000_0/part-m-00000
with etag 0x8DAD221662398A8 and length 4 but found a file with etag
0x8DAD221664D1399 and length 4
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[ERROR]
testTwoTaskAttemptsCommit(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)
Time elapsed: 1.291 s <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException:
`abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testTwoTaskAttemptsCommit/attempt2-m-00000':
Expected the file renamed from
abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testTwoTaskAttemptsCommit/_temporary/job_202211291550130427_0004/01/tasks/attempt_202211291550130427_0004_m_000000_1/attempt2-m-00000
with etag 0x8DAD221676CB12D and length 40 but found a file with etag
0x8DAD22167ACBE08 and length 40
at
org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
at
org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
7.58 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.177 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
[INFO] Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.624 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
24.35 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
10.322 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsContractUnbuffer
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
11.125 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed:
6.888 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
[INFO] Running
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
9.429 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.189
s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
[INFO] Running
org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.352 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
13.206 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
15.014 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.823 s - in org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
21.969 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
49.734 s - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 2, Time elapsed:
100.527 s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort
[ERROR]
test_110_teragen(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort) Time
elapsed: 25.881 s <<< FAILURE!
java.lang.AssertionError: teragen(1000,
abfs://****@****.dfs.core.windows.net/ITestAbfsTerasort/sortin) failed
expected:<0> but was:<1>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort.executeStage(ITestAbfsTerasort.java:211)
at
org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort.test_110_teragen(ITestAbfsTerasort.java:244)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:829)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestAbfsTerasort.test_110_teragen:244->executeStage:211->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
teragen(1000, abfs://****@****.dfs.core.windows.net/ITestAbfsTerasort/sortin)
failed expected:<0> but was:<1>
[ERROR] Errors:
[ERROR] ITestAbfsJobThroughManifestCommitter.test_0420_validateJob ?
OutputValidation ...
[ERROR] ITestAbfsManifestCommitProtocol.testCommitLifecycle ?
OutputValidation `abfs:/...
[ERROR] ITestAbfsManifestCommitProtocol.testCommitterWithDuplicatedCommit
? OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testConcurrentCommitTaskWithSubDir
? OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testMapFileOutputCommitter ?
OutputValidation ...
[ERROR] ITestAbfsManifestCommitProtocol.testOutputFormatIntegration ?
OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testParallelJobsToAdjacentPaths ?
OutputValidation
[ERROR] ITestAbfsManifestCommitProtocol.testTwoTaskAttemptsCommit ?
OutputValidation `...
[INFO]
[ERROR] Tests run: 335, Failures: 1, Errors: 8, Skipped: 46
```
I'll create another JIRA to fix those
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
