Patrick Wendell created SPARK-1843:
--------------------------------------
Summary: Provide a simpler alternative to assemble-deps
Key: SPARK-1843
URL: https://issues.apache.org/jira/browse/SPARK-1843
Project: Spark
Issue Type: Improvement
Components: Build
Reporter: Patrick Wendell
Assignee: Prashant Sharma
Fix For: 1.1.0
Right now we have the assemble-deps tool for speeding up local development.
I was thinking about a simpler solution to this problem where, instead of
creating a fancy assembly jar, we just add an environment variable:
USE_COMPILED_SPARK and, if that variable is present, we simply add the Spark
classes on the classpath before the assembly jar. Since the compiled classes
are on the classpath first, they will take precedence.
This would allow us to remove the entire assemble-deps build and associated
logic in the bash scripts. We'd need to make sure it's propagated correctly
during tests (like SPARK_TESTING) but other than that I think it should work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
