Herman Schistad created SPARK-9775:
--------------------------------------
Summary: Query Mesos for number of CPUs to set default parallelism
Key: SPARK-9775
URL: https://issues.apache.org/jira/browse/SPARK-9775
Project: Spark
Issue Type: Improvement
Components: Mesos
Affects Versions: 1.4.1
Reporter: Herman Schistad
Priority: Minor
As highlighted in a TODO on line 400 of MesosSchedulerBackend.scala (at least
on 3ca995b7) we should query the Mesos master and set the default parallelism
based on the number of CPUs available in the cluster (and multiply by two or
three?)
See code in question [here
(gitweb)|https://git-wip-us.apache.org/repos/asf?p=spark.git;a=blob;f=core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala;h=3f63ec1c;hb=HEAD#l400].
This task should also update the documentation [here
(gitweb)|https://git-wip-us.apache.org/repos/asf?p=spark.git;a=blob;f=docs/configuration.md;h=c60dd1;hb=HEAD#l789]
to highlight the fact.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
