[
https://issues.apache.org/jira/browse/HADOOP-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161424#comment-17161424
]
Hemanth Boyina commented on HADOOP-17140:
-----------------------------------------
thanks [~agrams] for the report , can you provide a patch with a UT
> KMSClientProvider Sends HTTP GET with null "Content-Type" Header
> ----------------------------------------------------------------
>
> Key: HADOOP-17140
> URL: https://issues.apache.org/jira/browse/HADOOP-17140
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
> Affects Versions: 2.7.3
> Reporter: Axton Grams
> Priority: Major
>
> Hive Server uses 'org.apache.hadoop.crypto.key.kms.KMSClientProvider' when
> interacting with HDFS TDE zones. This triggers a call to the KMS server. If
> the request method is a GET, the HTTP Header Content-Type is sent with a null
> value.
> When using Ranger KMS, the embedded Tomcat server returns a HTTP 400 error
> with the following error message:
> {quote}HTTP Status 400 - Bad Content-Type header value: ''
> The request sent by the client was syntactically incorrect.
> {quote}
> This only occurs with HTTP GET method calls.
> This is a captured HTTP request:
>
> {code:java}
> GET /kms/v1/key/xxx/_metadata?doAs=yyy&doAs=yyy HTTP/1.1
> Cookie:
> hadoop.auth="u=hive&p=hive/[email protected]&t=kerberos-dt&e=123789456&s=xxx="
> Content-Type:
> Cache-Control: no-cache
> Pragma: no-cache
> User-Agent: Java/1.8.0_241
> Host: kms.domain.com:9292
> Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
> Connection: keep-alive{code}
>
> Note the empty 'Content-Type' header.
> And the corresponding response:
>
> {code:java}
> HTTP/1.1 400 Bad Request
> Server: Apache-Coyote/1.1
> Content-Type: text/html;charset=utf-8
> Content-Language: en
> Content-Length: 1034
> Date: Thu, 16 Jul 2020 04:23:18 GMT
> Connection: close{code}
>
> This is the stack trace from the Hive server:
>
> {code:java}
> Caused by: java.io.IOException: HTTP status [400], message [Bad Request]
> at
> org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:169)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:608)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:597)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:566)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.getMetadata(KMSClientProvider.java:861)
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.compareKeyStrength(Hadoop23Shims.java:1506)
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.comparePathKeyStrength(Hadoop23Shims.java:1442)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.comparePathKeyStrength(SemanticAnalyzer.java:1990)
> ... 38 more{code}
>
> This looks to occur in
> [https://github.com/hortonworks/hadoop-release/blob/HDP-2.6.5.165-3-tag/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java#L591-L599]
> {code:java}
> if (authRetryCount > 0) {
> String contentType = conn.getRequestProperty(CONTENT_TYPE);
> String requestMethod = conn.getRequestMethod();
> URL url = conn.getURL();
> conn = createConnection(url, requestMethod);
> conn.setRequestProperty(CONTENT_TYPE, contentType);
> return call(conn, jsonOutput, expectedResponse, klass,
> authRetryCount - 1);
> }{code}
> I think when a GET method is received, the Content-Type header is not
> defined, then in line 592:
> {code:java}
> String contentType = conn.getRequestProperty(CONTENT_TYPE);
> {code}
> The code attempts to retrieve the CONTENT_TYPE Request Property, which
> returns null.
> Then in line 596:
> {code:java}
> conn.setRequestProperty(CONTENT_TYPE, contentType);
> {code}
> The null content type is used to construct the HTTP call to the KMS server.
> A null Content-Type header is not allowed/considered malformed by the
> receiving KMS server.
> I propose this code be updated to inspect the value returned by
> conn.getRequestProperty(CONTENT_TYPE), and not use a null value to construct
> the new KMS connection.
> Proposed pseudo-patch:
> {code:java}
> ---
> a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
> +++
> b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
> @@ -593,7 +593,9 @@ public HttpURLConnection run() throws Exception {
> String requestMethod = conn.getRequestMethod();
> URL url = conn.getURL();
> conn = createConnection(url, requestMethod);
> - conn.setRequestProperty(CONTENT_TYPE, contentType);
> + if (conn.getRequestProperty(CONTENT_TYPE) != null) {
> + conn.setRequestProperty(CONTENT_TYPE, contentType);
> + }
> return call(conn, jsonOutput, expectedResponse, klass,
> authRetryCount - 1);
> }{code}
> This should not impact any other use of this class and should only address
> cases where a null is returned for Content-Type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
