Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19103 >Oozie client gets the necessary tokens the application needs before launching. It passes those tokens along to the oozie launcher job (MR job) which will then actually call the Spark client to launch the spark app and pass the tokens along. >The oozie launcher job cannot get anymore tokens because all it has is tokens ( you can't get tokens with tokens, you need tgt or keytab). >The error here is because the launcher job runs the Spark Client to submit the spark job but the spark client doesn't see that it already has the hdfs tokens so it tries to get more, which ends with the exception. So the problem is that Oozie will get tokens for Spark instead of letting Spark do itself, and in Oozie launcher we should not let Spark `Yarn#client` to get tokens itself since there might not have tgt available in Oozie launcher. From my understanding of your issue, this seems like a more general issue regarding Oozie launcher and Spark token manage stuff. With the patch, looks like it only address the HDFS issue, how do we handle hive/hbase, looks like still have issues.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org