[
https://issues.apache.org/jira/browse/TEZ-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16994692#comment-16994692
]
László Bodor edited comment on TEZ-4102 at 12/12/19 1:56 PM:
-------------------------------------------------------------
thanks for the comments [~jeagles]
in [^TEZ-4102.03.patch] I tried another, safer approach
the idea is to let merge session credentials to be merged first and then merge
amConfig credentials (note that mergeAll instead of addAll), this way if
session credentials already have some tokens which are used while submitting
the [dag
plan|https://github.com/apache/tez/blob/7a3e378b59dad2afe1a068669620846b87d6e732/tez-api/src/main/java/org/apache/tez/client/TezClientUtils.java#L622],
they will be present while creating app, so yarn will extend those tokens,
what do you think about the approach?
in this case, I'll have to rename this jira from
"Merge AM credentials into session credentials while creating appContext"
to something like
"Let session credentials be merged before merging am launch credentials"
was (Author: abstractdog):
thanks for the comments [~jeagles]
in [^TEZ-4102.03.patch] I tried another, safer approach
the idea is to let merge session credentials to be merged first and then merge
amConfig credentials (note that mergeAll instead of addAll), this way if
session credentials already have some tokens which are used while submitting
the [dag
plan|https://github.com/apache/tez/blob/7a3e378b59dad2afe1a068669620846b87d6e732/tez-api/src/main/java/org/apache/tez/client/TezClientUtils.java#L622],
they will be present while creating app, so yarn will extend those tokens,
what do you think about the approach?
in this case I'll have to rename this jira from
"Merge AM credentials into session credentials while creating appContext"
to
"Let session credentials be merged before merging am launch credentials"
> Merge AM credentials into session credentials while creating appContext
> -----------------------------------------------------------------------
>
> Key: TEZ-4102
> URL: https://issues.apache.org/jira/browse/TEZ-4102
> Project: Apache Tez
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: TEZ-4102.01.patch, TEZ-4102.02.patch, TEZ-4102.03.patch
>
>
> Given the following scenario: kerberos + long running session + dags keep
> submitted to the session
> After 24h the queries can fail, because tasks don't have the correct
> HDFS_DELEGATION_TOKEN, because there is chance that am credentials has been
> previously filled with tokens and it cannot be overridden by session
> credentials
> [here|https://github.com/apache/tez/blob/master/tez-api/src/main/java/org/apache/tez/client/TezClientUtils.java#L485]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
