GitHub user coderplay opened a pull request:
https://github.com/apache/spark/pull/7091
[SPARK-8675] Executors created by LocalBackend won't get the same classpath
as other executor backends
AFAIK, some spark application always use LocalBackend to do some local
initiatives, spark sql is an example. Starting a LocalPoint won't add user
classpath into executor.
```java
override def start() {
localEndpoint = SparkEnv.get.rpcEnv.setupEndpoint(
"LocalBackendEndpoint", new LocalEndpoint(SparkEnv.get.rpcEnv,
scheduler, this, totalCores))
}
```
Thus will cause local executor fail with these scenarios, loading hadoop
built-in native libraries, loading other user defined native libraries, loading
user jars, reading s3 config from a site.xml file, etc
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/coderplay/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7091.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7091
----
commit 45bf62c8a01de4456b0f8c9f1e97af289343db8d
Author: Min Zhou <[email protected]>
Date: 2015-06-29T19:29:12Z
SPARK-8675 Executors created by LocalBackend won't get the same classpath
as other executor backends
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
