Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/78#issuecomment-36814000
Sure, we use that target for integrating Spark into the Apache Bigtop
distribution. In particular, we build a .tar.gz using the maven-assembly-plugin
and it contains all the spark jars and dependencies:
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/spark/do-component-build#L35
We then take this tarball distribution and use it in building our rpm/deb
packages:
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/spark/install_spark.sh#L134
The goal of that target is to not include any hadoop jars though since
those are provided by Bigtop. The bug this PR fixes is that the distribution
wasn't including the right assembly jar, so it makes it include the correct
assembly uber jar (which in itself contains hadoop jars but that's what
SPARK-1064 will fix).
I know of make-distribution.sh but haven't looked much into it since it
uses sbt. I can't speak for the whole Bigtop project here but I think most of
us integrators are more experienced with maven and therefore prefer it due to
the knowledge base and community. Hope that helps and thank you reviewing!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
