Repository: spark Updated Branches: refs/heads/master 264626536 -> 488863d87
[SPARK-13382][DOCS][PYSPARK] Update pyspark testing notes in build docs ## What changes were proposed in this pull request? The current build documents don't specify that for PySpark tests we need to include Hive in the assembly otherwise the ORC tests fail. ## How was the this patch tested? Manually built the docs locally. Ran the provided build command follow by the PySpark SQL tests.  Author: Holden Karau <hol...@us.ibm.com> Closes #11278 from holdenk/SPARK-13382-update-pyspark-testing-notes-r2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/488863d8 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/488863d8 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/488863d8 Branch: refs/heads/master Commit: 488863d87340e0fb68e81ba0be6df7d7c9723992 Parents: 2646265 Author: Holden Karau <hol...@us.ibm.com> Authored: Tue May 10 10:29:38 2016 -0700 Committer: Marcelo Vanzin <van...@cloudera.com> Committed: Tue May 10 10:29:38 2016 -0700 ---------------------------------------------------------------------- docs/building-spark.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/488863d8/docs/building-spark.md ---------------------------------------------------------------------- diff --git a/docs/building-spark.md b/docs/building-spark.md index 13c95e4..63532c7 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -123,6 +123,21 @@ To produce a Spark package compiled with Scala 2.10, use the `-Dscala-2.10` prop ./dev/change-scala-version.sh 2.10 mvn -Pyarn -Phadoop-2.4 -Dscala-2.10 -DskipTests clean package +# PySpark Tests with Maven + +If you are building PySpark and wish to run the PySpark tests you will need to build Spark with hive support. + +{% highlight bash %} +build/mvn -DskipTests clean package -Phive +./python/run-tests +{% endhighlight %} + +The run-tests script also can be limited to a specific Python version or a specific module + + ./python/run-tests --python-executables=python --modules=pyspark-sql + +**Note:** You can also run Python tests with an sbt build, provided you build Spark with hive support. + # Spark Tests in Maven Tests are run by default via the [ScalaTest Maven plugin](http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin). --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
