[
https://issues.apache.org/jira/browse/HADOOP-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222866#comment-17222866
]
Attila Magyar commented on HADOOP-16881:
----------------------------------------
Hey @[[email protected]], do you have any insights on this? There seems to be
a socket leak in KerberosAuthentication/PseudoAuthenticator which causes lots
of CLOSE_WAIT sockets to pile up and they're never cleared up. Adding explicit
disconnect calls to the http client solves the issue. However it might prevent
the connection pooling from reusing sockets.
{code:java}
diff --git
a/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java
b/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java
index 3bfa349880c..c035dd44ce0 100644
---
a/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java
+++
b/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java
@@ -183,8 +183,9 @@ public void authenticate(URL url, AuthenticatedURL.Token
token)
if (!token.isSet()) {
this.url = url;
base64 = new Base64(0);
+ HttpURLConnection conn = null;
try {
- HttpURLConnection conn = token.openConnection(url, connConfigurator);
+ conn = token.openConnection(url, connConfigurator);
conn.setRequestMethod(AUTH_HTTP_METHOD);
conn.connect();
@@ -218,6 +219,10 @@ public void authenticate(URL url, AuthenticatedURL.Token
token)
} catch (AuthenticationException ex){
throw wrapExceptionWithMessage(ex,
"Error while authenticating with endpoint: " + url);
+ } finally {
+ if (conn != null) {
+ conn.disconnect();
+ }
}
}
}
diff --git
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java
index 4e2ee4fdbea..8546a76c1af 100644
---
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java
+++
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java
@@ -312,8 +312,9 @@ private Map doDelegationTokenOperation(URL url,
dt = ((DelegationTokenAuthenticatedURL.Token)
token).getDelegationToken();
((DelegationTokenAuthenticatedURL.Token) token).setDelegationToken(null);
}
+ HttpURLConnection conn = null;
try {
- HttpURLConnection conn = aUrl.openConnection(url, token);
+ conn = aUrl.openConnection(url, token);
conn.setRequestMethod(operation.getHttpMethod());
HttpExceptionUtils.validateResponse(conn, HttpURLConnection.HTTP_OK);
if (hasResponse) {
@@ -339,6 +340,9 @@ private Map doDelegationTokenOperation(URL url,
if (dt != null) {
((DelegationTokenAuthenticatedURL.Token) token).setDelegationToken(dt);
}
+ if (conn != null) {
+ conn.disconnect();
+ }
}
return ret;
}
{code}
> PseudoAuthenticator does not disconnect HttpURLConnection leading to
> CLOSE_WAIT cnxns
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-16881
> URL: https://issues.apache.org/jira/browse/HADOOP-16881
> Project: Hadoop Common
> Issue Type: Bug
> Components: auth, security
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
>
> PseudoAuthenticator and KerberosAuthentication does not disconnect
> HttpURLConnection leading to lot of CLOSE_WAIT connections. YARN-8414 issue
> is observed due to this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
