steveloughran commented on code in PR #6550:
URL: https://github.com/apache/hadoop/pull/6550#discussion_r1812736795
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java:
##########
@@ -407,6 +414,36 @@ private static void initSigner(Configuration conf,
}
}
+ /**
+ * Initialize custom request headers for AWS clients.
+ * @param conf hadoop configuration
+ * @param clientConfig client configuration to update
+ * @param awsServiceIdentifier service name
+ */
+ private static void initRequestHeaders(Configuration conf,
+ ClientOverrideConfiguration.Builder clientConfig, String
awsServiceIdentifier) {
+ String configKey = null;
+ switch (awsServiceIdentifier) {
+ case AWS_SERVICE_IDENTIFIER_S3:
+ configKey = CUSTOM_HEADERS_S3;
+ break;
+ case AWS_SERVICE_IDENTIFIER_STS:
+ configKey = CUSTOM_HEADERS_STS;
+ break;
+ default:
+ // No known service.
+ }
+ if (configKey != null) {
+ Map<String, String> awsClientCustomHeadersMap =
+ S3AUtils.getTrimmedStringCollectionSplitByEquals(conf,
configKey);
+ awsClientCustomHeadersMap.forEach((header, valueString) -> {
+ List<String> headerValues = Arrays.asList(valueString.split(";"));
Review Comment:
Seems good.
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -1252,6 +1253,33 @@ private Constants() {
public static final String AWS_SERVICE_IDENTIFIER_DDB = "DDB";
public static final String AWS_SERVICE_IDENTIFIER_STS = "STS";
+ /** Prefix for S3A client-specific properties.
+ * value: {@value}
+ */
+ public static final String FS_S3A_CLIENT_PREFIX = "fs.s3a.client.";
+
+ /** Custom headers postfix.
+ * value: {@value}
+ */
+ public static final String CUSTOM_HEADERS_POSTFIX = ".custom.headers";
+
+ /**
+ * List of custom headers to be set on the service client.
+ * Multiple parameters can be used to specify custom headers.
Review Comment:
ok. I'm going to propose here doing all the next lines
in a single <pre> as there's enough formatting it wouldn't render right.
you can then cut the {@literal} bits.
The goal here is generally "looks good on an IDE rendered view"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
