Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22615#discussion_r222069855
--- Diff: docs/building-spark.md ---
@@ -49,25 +49,20 @@ To create a Spark distribution like those distributed
by the
to be runnable, use `./dev/make-distribution.sh` in the project root
directory. It can be configured
with Maven profile settings and so on like the direct Maven build. Example:
- ./dev/make-distribution.sh --name custom-spark --pip --r --tgz
-Psparkr -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes
+ ./dev/make-distribution.sh --name custom-spark --pip --r --tgz
-Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes
This will build Spark distribution along with Python pip and R packages.
For more information on usage, run `./dev/make-distribution.sh --help`
## Specifying the Hadoop Version and Enabling YARN
You can specify the exact version of Hadoop to compile against through the
`hadoop.version` property.
-If unset, Spark will build against Hadoop 2.6.X by default.
You can enable the `yarn` profile and optionally set the `yarn.version`
property if it is different
from `hadoop.version`.
-Examples:
+Example:
- # Apache Hadoop 2.6.X
- ./build/mvn -Pyarn -DskipTests clean package
-
- # Apache Hadoop 2.7.X and later
- ./build/mvn -Pyarn -Phadoop-2.7 -Dhadoop.version=2.7.3 -DskipTests
clean package
+ ./build/mvn -Pyarn -Dhadoop.version=2.7.7 -DskipTests clean package
--- End diff --
SPARK-25015 is reverted by SPARK-25330. Is it safe to give an example with
`2.7.7` without some warnings?
```
b0ada7dce0 [SPARK-25330][BUILD][BRANCH-2.3] Revert Hadoop 2.7 to 2.7.3
5f9633dc97 [SPARK-25015][BUILD] Update Hadoop 2.7 to 2.7.7
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
