Repository: spark Updated Branches: refs/heads/branch-2.1 e23c8cfc8 -> 39759ff00
[DOCS][MINOR] Update location of Spark YARN shuffle jar Looking at the distributions provided on spark.apache.org, I see that the Spark YARN shuffle jar is under `yarn/` and not `lib/`. This change is so minor I'm not sure it needs a JIRA. But let me know if so and I'll create one. Author: Nicholas Chammas <[email protected]> Closes #16130 from nchammas/yarn-doc-fix. (cherry picked from commit 5a92dc76ab431d73275a2bdfbc2c0a8ceb0d75d1) Signed-off-by: Marcelo Vanzin <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/39759ff0 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/39759ff0 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/39759ff0 Branch: refs/heads/branch-2.1 Commit: 39759ff00ba4313a82834387eea53b1af7b7daaf Parents: e23c8cf Author: Nicholas Chammas <[email protected]> Authored: Mon Dec 5 12:57:41 2016 -0800 Committer: Marcelo Vanzin <[email protected]> Committed: Mon Dec 5 12:58:21 2016 -0800 ---------------------------------------------------------------------- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/39759ff0/docs/running-on-yarn.md ---------------------------------------------------------------------- diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 4d1fafc..d4144c8 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -518,7 +518,7 @@ instructions: pre-packaged distribution. 1. Locate the `spark-<version>-yarn-shuffle.jar`. This should be under `$SPARK_HOME/common/network-yarn/target/scala-<version>` if you are building Spark yourself, and under -`lib` if you are using a distribution. +`yarn` if you are using a distribution. 1. Add this jar to the classpath of all `NodeManager`s in your cluster. 1. In the `yarn-site.xml` on each node, add `spark_shuffle` to `yarn.nodemanager.aux-services`, then set `yarn.nodemanager.aux-services.spark_shuffle.class` to --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
