[
https://issues.apache.org/jira/browse/HADOOP-17922?focusedWorklogId=657906&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-657906
]
ASF GitHub Bot logged work on HADOOP-17922:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 30/Sep/21 00:36
Start Date: 30/Sep/21 00:36
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#3466:
URL: https://github.com/apache/hadoop/pull/3466#discussion_r718337521
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AEncryptionMethods.java
##########
@@ -34,22 +34,26 @@
*/
public enum S3AEncryptionMethods {
Review comment:
yes, you are right. will fix
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java
##########
@@ -1597,14 +1678,50 @@ public static String getS3EncryptionKey(String bucket,
* @param conf configuration to scan
* @return the encryption mechanism (which will be {@code NONE} unless
* one is set.
- * @throws IOException on any validation problem.
+ * @throws IOException on JCKES lookup or invalid method/key configuration.
*/
public static S3AEncryptionMethods getEncryptionAlgorithm(String bucket,
Configuration conf) throws IOException {
- S3AEncryptionMethods encryptionMethod = S3AEncryptionMethods.getMethod(
- lookupPassword(bucket, conf,
- Constants.S3_ENCRYPTION_ALGORITHM));
- String encryptionKey = getS3EncryptionKey(bucket, conf);
+ return buildEncryptionSecrets(bucket, conf).getEncryptionMethod();
+ }
+
+ /**
+ * Get the server-side encryption or client side encryption algorithm.
+ * This includes validation of the configuration, checking the state of
+ * the encryption key given the chosen algorithm.
+ *
+ * @param bucket bucket to query for
+ * @param conf configuration to scan
+ * @return the encryption mechanism (which will be {@code NONE} unless
+ * one is set and secrets.
+ * @throws IOException on JCKES lookup or invalid method/key configuration.
+ */
+ @SuppressWarnings("deprecation")
+ public static EncryptionSecrets buildEncryptionSecrets(String bucket,
+ Configuration conf) throws IOException {
+
+ // new key, per-bucket
+ // this will include fixup of the old key in config XML entries
+ String algorithm = lookupBucketSecret(bucket, conf,
S3_ENCRYPTION_ALGORITHM);
+ if (algorithm == null) {
+ // try the old key, per-bucket setting, which will find JCEKS values
+ algorithm = lookupBucketSecret(bucket, conf,
SERVER_SIDE_ENCRYPTION_ALGORITHM);
+ }
+ if (algorithm == null) {
+ // new key, global setting
+ // this will include fixup of the old key in config XML entries
+ algorithm = lookupPassword(null, conf, S3_ENCRYPTION_ALGORITHM);
+ }
+ if (algorithm == null) {
+ // old key, global setting, for JCEKS entries.
+ algorithm = lookupPassword(null, conf, SERVER_SIDE_ENCRYPTION_ALGORITHM);
+ }
Review comment:
no
##########
File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
##########
@@ -1426,32 +1426,47 @@ Finally, the public `s3a://landsat-pds/` bucket can be
accessed anonymously:
</property>
```
-### Customizing S3A secrets held in credential files
+#### per-bucket configuration and deprecated configuration options
+Excluding secrets held in JCEKS files, a per-bucket declaration of a
+deprecated property will take priority over a global option.
-Secrets in JCEKS files or provided by other Hadoop credential providers
-can also be configured on a per bucket basis. The S3A client will
-look for the per-bucket secrets be
+This means that when setting encryption options in XML files,
Review comment:
let me review this one more time
##########
File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
##########
@@ -1426,32 +1426,47 @@ Finally, the public `s3a://landsat-pds/` bucket can be
accessed anonymously:
</property>
```
-### Customizing S3A secrets held in credential files
+#### per-bucket configuration and deprecated configuration options
+Excluding secrets held in JCEKS files, a per-bucket declaration of a
+deprecated property will take priority over a global option.
-Secrets in JCEKS files or provided by other Hadoop credential providers
-can also be configured on a per bucket basis. The S3A client will
-look for the per-bucket secrets be
+This means that when setting encryption options in XML files,
Review comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 657906)
Time Spent: 6h 10m (was: 6h)
> Lookup old S3 encryption configs for JCEKS
> ------------------------------------------
>
> Key: HADOOP-17922
> URL: https://issues.apache.org/jira/browse/HADOOP-17922
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Mehakmeet Singh
> Assignee: Mehakmeet Singh
> Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> HADOOP-17871 introduces new set of S3 encryption configs which are replaced
> by old property names during look-up. We need to look-up for both the
> properties since either could be set in a JCEKS file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
