[
https://issues.apache.org/jira/browse/YARN-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762354#comment-16762354
]
Rohith Sharma K S commented on YARN-9224:
-----------------------------------------
[~tarunparimi] thanks for the patch! Skimming through patch, approach looks
good but it appears concurrency issue exist. Did you verify this patch in real
cluster?
> TimelineDelegationTokenIdentifier.Renewer contacts KDC for every renew/cancel
> token operation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-9224
> URL: https://issues.apache.org/jira/browse/YARN-9224
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.6.0, 2.7.3
> Reporter: Tarun Parimi
> Priority: Major
> Attachments: YARN-9224.001.patch, YARN-9224.002.patch
>
>
> In a production cluster, we have observed the active RM principal making
> excessive requests to the KDC server. Being a service principal, this
> shouldn't be the case normally.
> On capturing tcpdump for the connections between RM and KDC, we saw that
> these excessive requests were for the SPNEGO service
> HTTP/ats-host.example.com .
> The requests were also matching in frequency with the below log entry in RM.
> {code:java}
> 2019-01-09T03:41:56.048-0500 INFO
> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl: Timeline service
> address: http://ats-host.example.com:8188/ws/v1/timeline/
> {code}
> On looking at the code in TimelineDelegationTokenIdentifier.java, it seems
> this kdc request for SPNEGO is done as we are creating a new timeline client
> instance every time.
> {code:java}
> @SuppressWarnings("unchecked")
> @Override
> public long renew(Token<?> token, Configuration conf) throws IOException,
> InterruptedException {
> TimelineClient client = TimelineClient.createTimelineClient();
> try {
> client.init(conf);
> client.start();
> return client.renewDelegationToken(
> (Token<TimelineDelegationTokenIdentifier>) token);
> } catch (YarnException e) {
> throw new IOException(e);
> } finally {
> client.stop();
> }
> }
> @SuppressWarnings("unchecked")
> @Override
> public void cancel(Token<?> token, Configuration conf) throws IOException,
> InterruptedException {
> TimelineClient client = TimelineClient.createTimelineClient();
> try {
> client.init(conf);
> client.start();
> client.cancelDelegationToken(
> (Token<TimelineDelegationTokenIdentifier>) token);
> } catch (YarnException e) {
> throw new IOException(e);
> } finally {
> client.stop();
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
