hey guys,
I'm hoping someone could provide some assistance. I'm having issues
(UnsatisfiedLinkError) calling some native libraries when submitting the
application in "cluster" mode. When running the application in local mode,
the application runs fine. Here's what my setup looks like.
The .so files are stored in the same location across all cluster nodes. The
node that submits the application has its LD_LIBRARY_PATH pointing to the
native libraries. Within the application i set the environment variables
like so:
SparkContext(new SparkConf()
.setAppName("SparkDriver")
.setExecutorEnv("LD_LIBRARY_PATH", sys.env("LD_LIBRARY_PATH"))
I thought the .setExecutorEnv would give each node the correct path to the
native libraries, but I guess not.
I'm not really sure what I'm missing in my setup, any help is greatly
appreciated.
Thanks!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Cannot-find-Native-Library-in-cluster-deploy-mode-tp28072.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
