Thomas Graves created SPARK-1719:
------------------------------------
Summary: spark.executor.extraLibraryPath isn't applied on yarn
Key: SPARK-1719
URL: https://issues.apache.org/jira/browse/SPARK-1719
Project: Spark
Issue Type: Bug
Components: YARN
Affects Versions: 1.0.0
Reporter: Thomas Graves
Looking through the code for spark on yarn I don't see that
spark.executor.extraLibraryPath is being properly applied when it launches
executors. It is using the spark.driver.libraryPath in the ClientBase.
Note I didn't actually test it so its possible I missed something.
I also think better to use LD_LIBRARY_PATH rather then -Djava.library.path.
once java.library.path is set, it doesn't search LD_LIBRARY_PATH. In Hadoop
we switched to use LD_LIBRARY_PATH instead of java.library.path. See
https://issues.apache.org/jira/browse/MAPREDUCE-4072. I'll split this into
separate jira.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
