[
https://issues.apache.org/jira/browse/HADOOP-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238368#comment-17238368
]
Wei-Chiu Chuang commented on HADOOP-16881:
------------------------------------------
Attila's fix could work, but here's a problem that I see. Most applications use
AuthenticatedURL.openConnection() to trigger the authentication and connection,
which calls KerberosAuthenticator.authenticate. Therefore, there are two
connections involved. It would be the best if the authentication (the first
one) connection can be reused.
{code}
/**
* Returns an authenticated {@link HttpURLConnection}.
*
* @param url the URL to connect to. Only HTTP/S URLs are supported.
* @param token the authentication token being used for the user.
*
* @return an authenticated {@link HttpURLConnection}.
*
* @throws IOException if an IO error occurred.
* @throws AuthenticationException if an authentication exception occurred.
*/
public HttpURLConnection openConnection(URL url, Token token) throws
IOException, AuthenticationException {
if (url == null) {
throw new IllegalArgumentException("url cannot be NULL");
}
if (!url.getProtocol().equalsIgnoreCase("http") &&
!url.getProtocol().equalsIgnoreCase("https")) {
throw new IllegalArgumentException("url must be for a HTTP or HTTPS
resource");
}
if (token == null) {
throw new IllegalArgumentException("token cannot be NULL");
}
authenticator.authenticate(url, token);
// allow the token to create the connection with a cookie handler for
// managing session cookies.
return token.openConnection(url, connConfigurator);
}
{code}
Additionally, another problem is inside the applications themselves, because a
proper Hadoop application should use delegation tokens instead of kerberos for
authentication. If DT is used, there shouldn't be a lot traffic going through
KerberosAuthenticator. I guess this is why I didn't see this problem happening
a lot before.
> PseudoAuthenticator does not disconnect HttpURLConnection leading to
> CLOSE_WAIT cnxns
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-16881
> URL: https://issues.apache.org/jira/browse/HADOOP-16881
> Project: Hadoop Common
> Issue Type: Bug
> Components: auth, security
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
>
> PseudoAuthenticator and KerberosAuthentication does not disconnect
> HttpURLConnection leading to lot of CLOSE_WAIT connections. YARN-8414 issue
> is observed due to this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
