Saisai Shao created SPARK-19021:
-----------------------------------
Summary: Generailize HDFSCredentialProvider to support non HDFS
security FS
Key: SPARK-19021
URL: https://issues.apache.org/jira/browse/SPARK-19021
Project: Spark
Issue Type: Improvement
Components: YARN
Affects Versions: 2.1.0
Reporter: Saisai Shao
Currently Spark can only get token renewal interval from security HDFS
(hdfs://), if Spark runs with other security file systems like webHDFS
(webhdfs://), wasb (wasb://), ADLS, it will ignore these tokens and not get
token renewal intervals from these tokens. These will make Spark unable to work
with these security clusters. So instead of only checking HDFS token, we should
generalize to support different {{DelegationTokenIdentifier}}.
This is a follow-up work of SPARK-18840.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
