[
https://issues.apache.org/jira/browse/SPARK-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-1384.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.9.2
> spark-shell on yarn on spark 0.9 branch doesn't always work with secure hdfs
> ----------------------------------------------------------------------------
>
> Key: SPARK-1384
> URL: https://issues.apache.org/jira/browse/SPARK-1384
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 0.9.0, 0.9.1
> Reporter: Thomas Graves
> Assignee: Thomas Graves
> Fix For: 0.9.2
>
>
> I've found an issue with the spark-shell in yarn-client mode in the 0.9.1
> rc3 release. It doesn't work with secure HDFS unless you
> export SPARK_YARN_MODE=true before starting the shell, or if you happen to do
> something immediately with HDFS. If you wait for the connection to the
> namenode to timeout it will fail.
>
> The fix actually went in to master branch with the authentication changes I
> made in master but I never realized that change needed to apply to 0.9.
> https://github.com/apache/spark/commit/7edbea41b43e0dc11a2de156be220db8b7952d01#diff-0ae5b834ce90ec37c19af35aa7a5e1a0
> See the SparkILoop diff.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
