Saisai Shao created SPARK-19021: ----------------------------------- Summary: Generailize HDFSCredentialProvider to support non HDFS security FS Key: SPARK-19021 URL: https://issues.apache.org/jira/browse/SPARK-19021 Project: Spark Issue Type: Improvement Components: YARN Affects Versions: 2.1.0 Reporter: Saisai Shao
Currently Spark can only get token renewal interval from security HDFS (hdfs://), if Spark runs with other security file systems like webHDFS (webhdfs://), wasb (wasb://), ADLS, it will ignore these tokens and not get token renewal intervals from these tokens. These will make Spark unable to work with these security clusters. So instead of only checking HDFS token, we should generalize to support different {{DelegationTokenIdentifier}}. This is a follow-up work of SPARK-18840. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org