Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5165#discussion_r27016084
--- Diff: docs/running-on-yarn.md ---
@@ -274,6 +274,6 @@ If you need a reference to the proper location to put
log files in the YARN so t
# Important notes
- Whether core requests are honored in scheduling decisions depends on
which scheduler is in use and how it is configured.
-- The local directories used by Spark executors will be the local
directories configured for YARN (Hadoop YARN config
`yarn.nodemanager.local-dirs`). If the user specifies `spark.local.dir`, it
will be ignored.
+- In yarn-cluster mode, the local directories used by the Spark executors
and the Spark driver will be the local directories configured for YARN (Hadoop
YARN config `yarn.nodemanager.local-dirs`). If the user specifies
`spark.local.dir`, it will be ignored. In yarn-client mode, the Spark executors
will use the local directories configured for YARN while the Spark driver will
use those defined in `spark.local.dir` (this is because the Spark driver does
not run on the YARN cluster in yarn-client mode, only the Spark executors do).
--- End diff --
Nits: you can back-tick-quote code-like values like `yarn-cluster`. The
last parenthetical can just be a sentence. Yeah it's normal to show all the
commits you made to your branch. They will be squashed on merge. You can also
squash and force-push your branch if desired, though it's not that important.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
