GitHub user scwf opened a pull request:
https://github.com/apache/spark/pull/1986
[SPARK-2608] fix executor backend launch commond over mesos mode
https://issues.apache.org/jira/browse/SPARK-2608
This is the updated version of https://github.com/apache/spark/pull/1513
Mesos scheduler backend use spark-class/spark-executor to launch executor
backend, this will lead to issues:
1 when set spark.executor.extraJavaOptions CoarseMesosSchedulerBackend will
throw errors because of the launch command "./bin/spark-class
org.apache.spark.executor.CoarseGrainedExecutorBackend %s %s %s %s
%d").format(basename, extraOpts, driverUrl,
offer.getSlaveId.getValue,offer.getHostname, numCores))
2 spark.executor.extraJavaOptions and spark.executor.extraLibraryPath set
in sparkconf will not be valid
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/scwf/spark mesosfix1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1986.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1986
----
commit 5b150a481a1ddb3dee4e94e14e737f2d20313f5c
Author: scwf <wangfei1.huawei.com>
Date: 2014-07-21T15:23:59Z
fix executor backend launch commond over mesos mode
commit fdc3cb1efa931780f652dcb4d2cda0304ca50710
Author: scwf <wangfei1.huawei.com>
Date: 2014-07-21T15:32:09Z
fix code format
commit 4290c71d17354de0cd345c98da3b328b40a1f4a7
Author: wangfei <[email protected]>
Date: 2014-08-06T10:20:07Z
fix compile error
commit bc92bdb0530247b71e8bcd50f5c12e9586770d99
Author: wangfei <[email protected]>
Date: 2014-08-06T10:20:09Z
fix compile error
commit bb5ada53b9b5e42a5783f719054a246c8e480186
Author: wangfei <[email protected]>
Date: 2014-08-06T10:22:16Z
delete empty lines
commit a954c5c6470ed40dfd8434c70299ef47bdcbde92
Author: wangfei <[email protected]>
Date: 2014-08-06T10:22:47Z
delete empty lines
commit 14578b2f2a52789448329569452898520699dafe
Author: wangfei <[email protected]>
Date: 2014-08-07T00:54:08Z
fix import
commit 7192f6f03f21a5efd9077e839c7f5f06c6fb5185
Author: wangfei <[email protected]>
Date: 2014-08-07T00:54:16Z
fix import
commit d80e45559db90227ee5d0fdd18eb1a33ce2ff30b
Author: wangfei <[email protected]>
Date: 2014-08-07T00:56:41Z
fix import
commit 4f701b59a4b5a3ec04cc3df92d5b13b538d19710
Author: wangfei <[email protected]>
Date: 2014-08-07T01:01:20Z
delete no use import
commit e515fc8029e4f152e0e8fda1f909ca7ccb77fb1e
Author: wangfei <[email protected]>
Date: 2014-08-07T01:20:12Z
Update MesosSchedulerBackend.scala
commit 80d99b02c30f7dfe6a22960243a662d8a72bb051
Author: wangfei <[email protected]>
Date: 2014-08-07T01:28:14Z
Update CoarseMesosSchedulerBackend.scala
commit 4f694c02be49c4962de4b96d847cb19ee6bf7c68
Author: wangfei <[email protected]>
Date: 2014-08-07T01:28:32Z
Update MesosSchedulerBackend.scala
commit 0e6508300d5c9934ed899415b71f47be7eb3b127
Author: scwf <[email protected]>
Date: 2014-08-16T03:24:10Z
Merge branch 'master' into mesosfix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
