[ https://issues.apache.org/jira/browse/SPARK-19021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15784772#comment-15784772 ]
Apache Spark commented on SPARK-19021: -------------------------------------- User 'jerryshao' has created a pull request for this issue: https://github.com/apache/spark/pull/16432 > Generailize HDFSCredentialProvider to support non HDFS security FS > ------------------------------------------------------------------ > > Key: SPARK-19021 > URL: https://issues.apache.org/jira/browse/SPARK-19021 > Project: Spark > Issue Type: Improvement > Components: YARN > Affects Versions: 2.1.0 > Reporter: Saisai Shao > Priority: Minor > > Currently Spark can only get token renewal interval from security HDFS > (hdfs://), if Spark runs with other security file systems like webHDFS > (webhdfs://), wasb (wasb://), ADLS, it will ignore these tokens and not get > token renewal intervals from these tokens. These will make Spark unable to > work with these security clusters. So instead of only checking HDFS token, we > should generalize to support different {{DelegationTokenIdentifier}}. > This is a follow-up work of SPARK-18840. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org