PradhanPrerak commented on code in PR #6550:
URL: https://github.com/apache/hadoop/pull/6550#discussion_r1717669552
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java:
##########
@@ -407,6 +414,36 @@ private static void initSigner(Configuration conf,
}
}
+ /**
+ * Initialize custom request headers for AWS clients.
+ * @param conf hadoop configuration
+ * @param clientConfig client configuration to update
+ * @param awsServiceIdentifier service name
+ */
+ private static void initRequestHeaders(Configuration conf,
+ ClientOverrideConfiguration.Builder clientConfig, String
awsServiceIdentifier) {
+ String configKey = null;
+ switch (awsServiceIdentifier) {
+ case AWS_SERVICE_IDENTIFIER_S3:
+ configKey = CUSTOM_HEADERS_S3;
+ break;
+ case AWS_SERVICE_IDENTIFIER_STS:
+ configKey = CUSTOM_HEADERS_STS;
+ break;
+ default:
+ // No known service.
+ }
+ if (configKey != null) {
+ Map<String, String> awsClientCustomHeadersMap =
+ S3AUtils.getTrimmedStringCollectionSplitByEquals(conf,
configKey);
+ awsClientCustomHeadersMap.forEach((header, valueString) -> {
+ List<String> headerValues = Arrays.asList(valueString.split(";"));
Review Comment:
This is the largest change in this revision. Having ":" as value separator
is not a good idea. Because,
1/ Some header names in aws start with `aws:<header_name>`
2/ Some values such as arns allow ":" in it,
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html#arns-paths
Hence changed delimiter to ";" which is not so common -
https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines
let me know if you have better ideas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
