[ https://issues.apache.org/jira/browse/SPARK-25681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-25681: ------------------------------------ Assignee: Apache Spark > Delegation Tokens fetched twice upon HadoopFSDelegationTokenProvider creation > ----------------------------------------------------------------------------- > > Key: SPARK-25681 > URL: https://issues.apache.org/jira/browse/SPARK-25681 > Project: Spark > Issue Type: Improvement > Components: Kubernetes, Mesos, YARN > Affects Versions: 2.5.0 > Reporter: Ilan Filonenko > Assignee: Apache Spark > Priority: Major > Labels: Hadoop, Kerberos > > Looking for a refactor to {{HadoopFSDelegationTokenProvider.}} Within the > function {{obtainDelegationTokens()}}: > This code-block: > {code:java} > val fetchCreds = fetchDelegationTokens(getTokenRenewer(hadoopConf),...) > // Get the token renewal interval if it is not set. It will only be > called once. > if (tokenRenewalInterval == null) { > tokenRenewalInterval = getTokenRenewalInterval(...) > }{code} > calls {{fetchDelegationTokens()}} twice since the {{tokenRenewalInterval}} > will always be null upon creation of the {{TokenManager}} which I think is > unnecessary in the case of Kubernetes (as you are creating 2 DTs when only > one is needed.) Could this possibly be refactored to only call > {{fetchDelegationTokens()}} once upon startup or to have a param to specify > {{tokenRenewalInterval}} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org