Marcelo: by my count, there are 19 maven modules in the codebase. I am
typically only concerned with "core" (and therefore its two dependencies as
well, `network/{shuffle,common}`).The `mvn package` workflow (and its sbt equivalent) that most people apparently use involves (for me) compiling+packaging 16 other modules that I don't care about; I pay this cost whenever I rebase off of master or encounter the sbt-compiler-crash bug, among other possible scenarios. Compiling one module (after building/installing its dependencies) seems like the sort of thing that should be possible, and I don't see why my previously-documented attempt is failing. re: Marcelo's comment about "missing the 'spark-parent' project", I saw that error message too and tried to ascertain what it could mean. Why would `network/shuffle` need something from the parent project? AFAICT `network/common` has the same references to the parent project as `network/shuffle` (namely just a <parent> block in its POM), and yet I can `mvn install -pl` the former but not the latter. Why would this be? One difference is that `network/shuffle` has a dependency on another module, while `network/common` does not. Does Maven not let you build modules that depend on *any* other modules without building *all* modules, or is there a way to do this that we've not found yet? Patrick: per my response to Marcelo above, I am trying to avoid having to compile and package a bunch of stuff I am not using, which both `mvn package` and `mvn install` on the parent project do. On Tue Dec 02 2014 at 3:45:48 PM Marcelo Vanzin <van...@cloudera.com> wrote: > On Tue, Dec 2, 2014 at 2:40 PM, Ryan Williams > <ryan.blake.willi...@gmail.com> wrote: > > Following on Mark's Maven examples, here is another related issue I'm > > having: > > > > I'd like to compile just the `core` module after a `mvn clean`, without > > building an assembly JAR first. Is this possible? > > Out of curiosity, may I ask why? What's the problem with running "mvn > install -DskipTests" first (or "package" instead of "install", > although I generally do the latter)? > > You can probably do what you want if you manually build / install all > the needed dependencies first; you found two, but it seems you're also > missing the "spark-parent" project (which is the top-level pom). That > sounds like a lot of trouble though, for not any gains that I can > see... after the first build you should be able to do what you want > easily. > > -- > Marcelo >
