Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/3239#issuecomment-62869875
I had some additional conversation with @pwendell and we agreed that
SPARK-4376 (putting external modules behind maven profiles) is worthwhile, so
this PR implements both that and SPARK-4375. It's not 100% polished - it needs
documentation and I'm still hitting random errors with scala-2.11 under certain
conditions.
To build assembly and not build any of the external modules or examples:
mvn package
To build assembly and one of the external modules:
mvn package -Pflume
To build assembly and all external modules and examples:
mvn package -Pexamples -Pkafka
To build assembly with 2.11:
mvn package -Pscala-2.11
To build assembly and one of the external modules with 2.11:
mvn package -Pscala-2.11 -Pflume
To build assembly and all external modules and examples with 2.11 (except
for Kafka, which doesn't work with 2.11):
mvn package -Pscala-2.11 -Pexamples
This patch removes the Algebird examples entirely, because they're fairly
obscure, show off more Algebird functionality than Spark functionality, don't
work with Scala 2.11, and would require extra profiles to support.
To get the Scala-2.11 build to work, I needed to set all the artifact IDs
to include the scala.binary.version instead of hardcode 2.10, which seems right
to me unless I'm missing something? Not sure how this was working before.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
