Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/86#discussion_r10995127
--- Diff: docs/running-on-yarn.md ---
@@ -48,10 +48,12 @@ System Properties:
Ensure that HADOOP_CONF_DIR or YARN_CONF_DIR points to the directory which
contains the (client side) configuration files for the Hadoop cluster.
These configs are used to connect to the cluster, write to the dfs, and
connect to the YARN ResourceManager.
-There are two scheduler modes that can be used to launch Spark
applications on YARN. In yarn-cluster mode, the Spark driver runs inside an
application master process which is managed by YARN on the cluster, and the
client can go away after initiating the application. In yarn-client mode, the
driver runs in the client process, and the application master is only used for
requesting resources from YARN.
+There are two deploy modes that can be used to launch Spark applications
on YARN. In yarn-cluster mode, the Spark driver runs inside an application
master process which is managed by YARN on the cluster, and the client can go
away after initiating the application. In yarn-client mode, the driver runs in
the client process, and the application master is only used for requesting
resources from YARN.
Unlike in Spark standalone and Mesos mode, in which the master's address
is specified in the "master" parameter, in YARN mode the ResourceManager's
address is picked up from the Hadoop configuration. Thus, the master parameter
is simply "yarn-client" or "yarn-cluster".
+The spark-submit script described in the [cluster mode
overview](cluster-overview.html) provides the most straightforward way to
submit a compiled Spark application to YARN in either deploy mode. For info on
the lower-level invocations it uses, read ahead. For running spark-shell
against YARN, skip down to the yarn-client section.
+
## Launching a Spark application with yarn-cluster mode.
--- End diff --
It might be useful to give an example of actual usage of running one of the
examples
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
