There's a proposal / discussion of the assembly-less distributions at https://github.com/vanzin/spark/pull/2/files / https://issues.apache.org/jira/browse/SPARK-11157.
On Tue, Nov 10, 2015 at 3:53 PM, Reynold Xin <r...@databricks.com> wrote: > > On Tue, Nov 10, 2015 at 3:35 PM, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> >> > 3. Assembly-free distribution of Spark: don’t require building an >> enormous assembly jar in order to run Spark. >> >> Could you elaborate a bit on this? I'm not sure what an assembly-free >> distribution means. >> >> > Right now we ship Spark using a single assembly jar, which causes a few > different problems: > > - total number of classes are limited on some configurations > > - dependency swapping is harder > > > The proposal is to just avoid a single fat jar. > > >
