Thomas Marquardt created HADOOP-15663:
-----------------------------------------
Summary: ABFS: Simplify configuration
Key: HADOOP-15663
URL: https://issues.apache.org/jira/browse/HADOOP-15663
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/azure
Reporter: Thomas Marquardt
Configuration for WASB and ABFS is too complex. The current approach is to use
four files for test configuration.
Both WASB and ABFS have basic test configuration which is committed to the repo
(azure-test.xml and azure-bfs-test.xml). Currently these contain the
fs.AbstractFileSystem.[scheme].impl configuration, but otherwise are empty
except for an include reference to a file containing the endpoint credentials.
Both WASB and ABFS have endpoint credential configuration files
(azure-auth-keys.xml and azure-bfs-auth-keys.xml). These have been added to
.gitignore to prevent them from accidentally being submitted in a patch, which
would leak the developers storage account credentials. These files contain
account names, storage account keys, and service endpoints.
There is some overlap of the configuration for WASB and ABFS, where they use
the same property name but use different values.
1) Let's reduce the number of test configuration files to one, if possible.
2) Let's simplify the account name, key, and endpoint configuration for WASB
and ABFS if possible, but still support the legacy way of doing it, which is
very error prone.
3) Let's improve error handling, so that typos or misconfiguration are not so
difficult to troubleshoot.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
