No, you should read:
if spark.local.dir is specified, spark.local.dir will be ignored.
This has been reworded (hopefully for the best) in 1.3.1:
https://spark.apache.org/docs/1.3.1/running-on-yarn.html
Christophe.
On 17/04/2015 18:18, shenyanls wrote:
According to the documentation:
The local directories used by Spark executors will be the local directories
configured for YARN (Hadoop YARN config yarn.nodemanager.local-dirs). If the
user specifies spark.local.dir, it will be ignored.
(https://spark.apache.org/docs/1.2.1/running-on-yarn.html)
If spark.local.dir is specified, the yarn local directory will be ignored,
right? It's a little ambiguous to me.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/local-directories-for-spark-running-on-yarn-tp22543.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce
message, merci de le détruire et d'en avertir l'expéditeur.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
