GitHub user mgummelt opened a pull request:
https://github.com/apache/spark/pull/10993
[SPARK-5095] [Mesos] Support launching multiple mesos executors in coarse
grained mesos mode.
This is the next iteration of @tnachen's previous PR:
https://github.com/apache/spark/pull/4027
In that PR, we resolved with @andrewor14 and @pwendell to implement the
Mesos scheduler's support of `spark.executor.cores` to be consistent with YARN
and Standalone. This PR implements that resolution.
This PR implements two high-level features. These two features are
co-dependent, so they're implemented both here:
- Mesos support for spark.executor.cores
- Multiple executors per slave
We at Mesosphere have been working with Typesafe on a Spark/Mesos
integration test suite:
https://github.com/typesafehub/mesos-spark-integration-tests, which passes for
this PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mesosphere/spark executor_sizing
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10993.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10993
----
commit f4211331a1e13e2ff9fa3bcf7efb896d6aad729e
Author: Michael Gummelt <[email protected]>
Date: 2016-01-19T22:24:58Z
Support multiple executors per node on Mesos.
Support spark.executor.cores on Mesos.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
