Github user markgrover commented on a diff in the pull request:
https://github.com/apache/spark/pull/11278#discussion_r60290250
--- Diff: docs/building-spark.md ---
@@ -123,6 +123,21 @@ To produce a Spark package compiled with Scala 2.10,
use the `-Dscala-2.10` prop
./dev/change-scala-version.sh 2.10
mvn -Pyarn -Phadoop-2.4 -Dscala-2.10 -DskipTests clean package
+# PySpark Tests with Maven
+
+If you are building PySpark and wish to run the PySpark tests you will
need to build an assembly JAR and also ensure you have built with hive support.
--- End diff --
Yeah, I agree with @JoshRosen since assembly jars are now gone. I think the
hive support part still applies but we should take out the part to build an
assembly jar.
The ```hadoop-2.4``` profile sets the ```hadoop.version``` so I don't think
we need to specify the latter explicitly. I am also not sure, at the top of my
head, if we need the ```hive-thriftserver``` profile for running python tests
so it may make sense to see if just specifying the hive profile would suffice.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
