[
https://issues.apache.org/jira/browse/DRILL-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582369#comment-16582369
]
ASF GitHub Bot commented on DRILL-6662:
---------------------------------------
KazydubB commented on a change in pull request #1419: DRILL-6662: Access AWS
access key ID and secret access key using Cred…
URL: https://github.com/apache/drill/pull/1419#discussion_r210559211
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
##########
@@ -104,6 +109,33 @@ public FileSystemPlugin(FileSystemConfig config,
DrillbitContext context, String
}
}
+ private boolean isS3Connection(Configuration conf) {
+ URI uri = FileSystem.getDefaultUri(conf);
+ return uri.getScheme().equals("s3a");
+ }
+
+ /**
+ * Retrieve secret and access keys from configured (with
+ * {@link
org.apache.hadoop.security.alias.CredentialProviderFactory#CREDENTIAL_PROVIDER_PATH}
property)
+ * credential providers and set it into {@code conf}. If provider path is
not configured or credential
+ * is absent in providers, it will conditionally fallback to configuration
setting. The fallback will occur unless
+ * {@link
org.apache.hadoop.security.alias.CredentialProvider#CLEAR_TEXT_FALLBACK} is set
to {@code false}.
+ *
+ * @param conf {@code Configuration} which will be updated with credentials
from provider
+ * @throws IOException thrown if a credential cannot be retrieved from
provider
+ */
+ private void handleS3Credentials(Configuration conf) throws IOException {
+ String[] credentialKeys = {"fs.s3a.secret.key", "fs.s3a.access.key"};
Review comment:
I am aware of the Constants, but artifact (hadoop-aws), containing this
class is not among the module's dependencies (however it is present in
distribution's (compile-scope) and drill-root's (test-scope) dependencies). Is
there a need to add the dependency?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Access AWS access key ID and secret access key using Credential Provider API
> for S3 storage plugin
> --------------------------------------------------------------------------------------------------
>
> Key: DRILL-6662
> URL: https://issues.apache.org/jira/browse/DRILL-6662
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Bohdan Kazydub
> Assignee: Bohdan Kazydub
> Priority: Major
> Labels: doc-impacting
> Fix For: 1.15.0
>
>
> Hadoop provides [CredentialProvider
> API|[https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html]]
> which allows passwords and other sensitive secrets to be stored in an
> external provider rather than in configuration files in plaintext.
> Currently S3 storage plugin is accessing passwords, namely
> 'fs.s3a.access.key' and 'fs.s3a.secret.key', stored in clear text in
> Configuration with get() method. To give users an ability to remove clear
> text passwords for S3 from configuration files Configuration.getPassword()
> method should be used, given they configure
> 'hadoop.security.credential.provider.path' property which points to a file
> containing encrypted passwords instead of configuring two aforementioned
> properties.
> By using this approach, credential providers will be checked first and if the
> secret is not provided or providers are not configured there will be a
> fallback to secrets configured in clear text (unless
> 'hadoop.security.credential.clear-text-fallback' is configured to be
> "false"), thus making new change backwards-compatible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
