GitHub user ankurcha opened a pull request:
https://github.com/apache/spark/pull/5563
[SPARK-6707] - Mesos Scheduler should allow the user to specify constraints
based on slave attributes
Currently, the mesos scheduler only looks at the 'cpu' and 'mem' resources
when trying to determine the usablility of a resource offer from a mesos slave
node. It may be preferable for the user to be able to ensure that the spark
jobs are only started on a certain set of nodes (based on attributes).
For example, If the user sets a property, let's say
`spark.mesos.constraints` is set to `tachyon=true;us-east-1=false`, then the
resource offers will be checked to see if they meet both these constraints and
only then will be accepted to start new executors.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ankurcha/spark mesos_attribs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5563.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5563
----
commit 608005f8172d95834eff726646d2dc734bcb152d
Author: Ankur Chauhan <[email protected]>
Date: 2015-04-17T19:27:48Z
First pass at add attributes filtering of offers to mesos scheduler
commit 8895eabd71b876930ecccd92ed7b76e36fb2d3c0
Author: Ankur Chauhan <[email protected]>
Date: 2015-04-17T19:38:15Z
code style fixes
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
