Github user maasg commented on a diff in the pull request:
https://github.com/apache/spark/pull/756#discussion_r12578801
--- Diff: docs/running-on-mesos.md ---
@@ -25,31 +115,52 @@ val conf = new SparkConf()
val sc = new SparkContext(conf)
{% endhighlight %}
+When running a shell the `spark.executor.uri` parameter is inherited from
`SPARK_EXECUTOR_URI`, so
+it does not need to be redundantly passed in as a system property.
+
+{% highlight shell %}
+./bin/spark-shell --master mesos://host:5050
+{% endhighlight %}
+
+
# Mesos Run Modes
-Spark can run over Mesos in two modes: "fine-grained" and
"coarse-grained". In fine-grained mode, which is the default,
-each Spark task runs as a separate Mesos task. This allows multiple
instances of Spark (and other frameworks) to share
-machines at a very fine granularity, where each application gets more or
fewer machines as it ramps up, but it comes with an
-additional overhead in launching each task, which may be inappropriate for
low-latency applications (e.g. interactive queries or serving web requests).
The coarse-grained mode will instead
-launch only *one* long-running Spark task on each Mesos machine, and
dynamically schedule its own "mini-tasks" within
-it. The benefit is much lower startup overhead, but at the cost of
reserving the Mesos resources for the complete duration
-of the application.
+Spark can run over Mesos in two modes: "fine-grained" (default) and
"coarse-grained".
+
+In "fine-grained" mode (default), each Spark task runs as a separate Mesos
task. This allows
+multiple instances of Spark (and other frameworks) to share machines at a
very fine granularity,
+where each application gets more or fewer machines as it ramps up and
down, but it comes with an
+additional overhead in launching each task. This mode may be inappropriate
for low-latency
+requirements like interactive queries or serving web requests.
+
+The "coarse-grained" mode will instead launch only *one* long-running
Spark task on each Mesos
+machine, and dynamically schedule its own "mini-tasks" within it. The
benefit is much lower startup
+overhead, but at the cost of reserving the Mesos resources for the
complete duration of the
+application.
-To run in coarse-grained mode, set the `spark.mesos.coarse` property in
your [SparkConf](configuration.html#spark-properties):
+To run in coarse-grained mode, set the `spark.mesos.coarse` property in
your
+[SparkConf](configuration.html#spark-properties):
{% highlight scala %}
conf.set("spark.mesos.coarse", "true")
{% endhighlight %}
-In addition, for coarse-grained mode, you can control the maximum number
of resources Spark will acquire. By default,
-it will acquire *all* cores in the cluster (that get offered by Mesos),
which only makes sense if you run just one
-application at a time. You can cap the maximum number of cores using
`conf.set("spark.cores.max", "10")` (for example).
+In addition, for coarse-grained mode, you can control the maximum number
of resources Spark will
+acquire. By default, it will acquire *all* cores in the cluster (that get
offered by Mesos), which
+only makes sense if you run just one application at a time. You can cap
the maximum number of cores
+using `conf.set("spark.cores.max", "10")` (for example).
--- End diff --
# Troubleshooting and debugging
- Accessing the tasks on the mesos console
- Location of the logs on the Mesos executor
- Usual pitfalls
-- Spark assembly is not reachable/accessible
-- Slaves cannot reach resources (HDFS/...)
-- Tasks cannot reach driver (when e.g. your try to submit from a local
machine to a remote cluster)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
