My thinking is that since local/cluster mode depends on whether the MASTER variable is set in the conf, the local/cluster mode distinction in the build has to do whether or not to bundle dependencies. If so, that's a documentation error. Otherwise, separate explicit profiles are needed.
On Mon, Mar 23, 2015 at 12:07 PM, RJ Nowling <[email protected]> wrote: > Thanks, Nirav. > > It seems to me that there should be explicit profiles for local vs cluster > mode separate from the Spark/Hadoop profiles. For example, what if I want > to set the version of Spark used in the local mode? > > > On Mon, Mar 23, 2015 at 11:54 AM, Nirav Mehta <[email protected]> > wrote: > >> 1. Yes, they trigger the build processes accordingly. I just built it >> with spark 1.3.0 and Hadoop 2.6.0 >> 2. Local will build Zeppelin with Spark to trigger a local context with >> local threads. With cluster mode, it will expect the master, hadoop conf >> and spark assembly configurations to be specified in the zeppelin-env.sh >> >> I've just started using Zeppelin myself, so experts please validate my >> claim. >> >> On Mon, Mar 23, 2015 at 10:48 AM, RJ Nowling <[email protected]> wrote: >> >>> Hi all, >>> >>> I wanted to clarify how the Maven profiles work, especially with >>> reference to cluster vs local mode. >>> >>> The documentation [1] specifies that local mode can be compiled like so: >>> >>> $ mvn install -DskipTests >>> >>> and cluster mode like so: >>> >>> $ mvn install -DskipTests -Dspark.version=1.1.0 -Dhadoop.version=2.2.0 >>> >>> I may want to specify different Spark or Hadoop versions for local mode, >>> though. In the pom.xml file, I saw that there are different Spark and >>> Hadoop profiles specified. >>> >>> Questions: >>> 1. Are the Maven profiles mutually exclusive? Can I specify -Pspark-1.3 >>> -Phadoop-2.4 ? >>> >>> 2. What differentiates the local vs cluster mode in the build? >>> >>> Thanks, >>> RJ >>> >>> [1] https://zeppelin.incubator.apache.org/docs/install/install.html >>> >>> >> >
