[
https://issues.apache.org/jira/browse/HADOOP-18960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17888291#comment-17888291
]
ASF GitHub Bot commented on HADOOP-18960:
-----------------------------------------
steveloughran commented on code in PR #7104:
URL: https://github.com/apache/hadoop/pull/7104#discussion_r1795416489
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java:
##########
@@ -65,8 +67,8 @@ public void setup() throws Exception {
TestConfigurationKeys.FS_AZURE_TEST_NAMESPACE_ENABLED_ACCOUNT, false);
Assume.assumeTrue(isHNSEnabled);
loadConfiguredFileSystem();
-
this.getConfiguration().set(ConfigurationKeys.FS_AZURE_SAS_TOKEN_PROVIDER_TYPE,
TEST_AUTHZ_CLASS);
-
this.getConfiguration().set(ConfigurationKeys.FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
"SAS");
+ this.getConfiguration().set(FS_AZURE_SAS_TOKEN_PROVIDER_TYPE,
TEST_AUTHZ_CLASS);
Review Comment:
cut the `this.`
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsRestOperationException.java:
##########
@@ -201,4 +183,20 @@ public void testAuthFailException() throws Exception {
.describedAs("Incorrect error message: " + errorDesc)
.contains("Auth failure: ");
}
+
+ private Configuration getEntries(final int numOfRetries) {
+ Configuration config = new Configuration(this.getRawConfiguration());
+ String accountName = config.get(FS_AZURE_ABFS_ACCOUNT_NAME);
+ // Setup to configure custom token provider.
+ config.set(accountProperty(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
accountName), "Custom");
+
config.set(accountProperty(FS_AZURE_ACCOUNT_TOKEN_PROVIDER_TYPE_PROPERTY_NAME,
accountName),
+ RETRY_TEST_TOKEN_PROVIDER);
+ config.set(AZURE_CUSTOM_TOKEN_FETCH_RETRY_COUNT, Integer.toString(
+ numOfRetries));
+ // Stop filesystem creation as it will lead to calls to store.
+ config.set(AZURE_CREATE_REMOTE_FILESYSTEM_DURING_INITIALIZATION, "false");
Review Comment:
setInteger
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsRestOperationException.java:
##########
@@ -201,4 +183,20 @@ public void testAuthFailException() throws Exception {
.describedAs("Incorrect error message: " + errorDesc)
.contains("Auth failure: ");
}
+
+ private Configuration getEntries(final int numOfRetries) {
+ Configuration config = new Configuration(this.getRawConfiguration());
+ String accountName = config.get(FS_AZURE_ABFS_ACCOUNT_NAME);
+ // Setup to configure custom token provider.
+ config.set(accountProperty(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
accountName), "Custom");
+
config.set(accountProperty(FS_AZURE_ACCOUNT_TOKEN_PROVIDER_TYPE_PROPERTY_NAME,
accountName),
+ RETRY_TEST_TOKEN_PROVIDER);
+ config.set(AZURE_CUSTOM_TOKEN_FETCH_RETRY_COUNT, Integer.toString(
+ numOfRetries));
+ // Stop filesystem creation as it will lead to calls to store.
+ config.set(AZURE_CREATE_REMOTE_FILESYSTEM_DURING_INITIALIZATION, "false");
+ config.set(FS_AZURE_ACCOUNT_IS_HNS_ENABLED, config.getBoolean(
Review Comment:
setBoolean
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSeek.java:
##########
@@ -68,6 +68,9 @@ protected Configuration createConfiguration() {
protected AbstractFSContract createContract(final Configuration conf) {
conf.setInt(AZURE_READ_AHEAD_RANGE, MIN_BUFFER_SIZE);
conf.setInt(AZURE_READ_BUFFER_SIZE, MIN_BUFFER_SIZE);
+ // Disabling cache to make sure new configs are picked up.
Review Comment:
pull out into an azure test utils class, just as
`S3ATestUtils.disableFilesystemCaching()does
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsRestOperationException.java:
##########
@@ -201,4 +183,20 @@ public void testAuthFailException() throws Exception {
.describedAs("Incorrect error message: " + errorDesc)
.contains("Auth failure: ");
}
+
+ private Configuration getEntries(final int numOfRetries) {
Review Comment:
nit: javadoc
> ABFS contract-tests with Hadoop-Commons intermittently failing
> --------------------------------------------------------------
>
> Key: HADOOP-18960
> URL: https://issues.apache.org/jira/browse/HADOOP-18960
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Reporter: Pranav Saxena
> Assignee: Anuj Modi
> Priority: Minor
> Labels: pull-request-available
>
> In the merged pr [HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs
> on root path by anujmodi2021 · Pull Request #6003 · apache/hadoop
> (github.com)|https://github.com/apache/hadoop/pull/6003], a config was
> switched-on: `fs.contract.test.root-tests-enabled`. This enables the root
> manipulation tests for the filesystem contract.
> Now, the execution of contract-tests in abfs works as per executionId
> integration-test-abfs-parallel-classes of the pom. The tests would work in
> different jvms, and at a given instance multiple such jvms could be there,
> depending on ${testsThreadCount}. The problem is that all the test jvms for
> contract-test use the same container for test runs which is defined by
> `fs.contract.test.fs.abfs`. Due to this, one jvm root-contract-runs can
> influence other jvm's root-contract-runs. This leads to CI failures for
> hadoop-azure package.
> Solution is to run these tests sequentially and separate from other
> commit/manifest tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
