On Mon, Aug 3, 2015 at 1:00 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com> wrote:
> Thanks a lot for all these documents. Appreciate your effort & time. > > On Mon, Aug 3, 2015 at 10:15 AM, Christian Tzolov <ctzo...@pivotal.io> > wrote: > >> ÐΞ€ρ@Ҝ (๏̯͡๏), >> >> I've successfully run Zeppelin with Spark on YARN. I'm using Ambari and >> PivotalHD30. PHD30 is ODP compliant so you should be able to repeat the >> configuration for HDP (e.g. hortonworks). >> >> 1. Before you start with Zeppelin, make sure that your Spark/YARN env. >> works from the command line (e.g run Pi test). If it doesn't work make sure >> that the hdp.version is set correctly or you can hardcode the stack.name >> and stack.version properties as Ambari Custom yarn-site properties (that is >> what i did). >> >> 2. Your Zeppelin should be build with proper Spark and Hadoop versions >> and YARN support enabled. In my case used this build command: >> >> mvn clean package -Pspark-1.4 -Dspark.version=1.4.1 >> -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn -DskipTests -Pbuild-distr >> >> 3. Open the Spark interpreter configuration and set 'master' property to >> 'yarn-client' ( e.g. master=yarn-client). then press Save. >> >> 4. In (conf/zeppelin-env.sh) set HADOOP_CONF_DIR for PHD and HDP it will >> look like this: >> export HADOOP_CONF_DIR=/etc/hadoop/conf >> >> 5. (optional) i've restarted the zeppelin daemon but i don't think this >> is required. >> >> 6. Make sure that HDFS has /user/<zeppelin user> folder exists and has >> HDFS write permissions. Otherwise you can create it like this: >> sudo -u hdfs hdfs dfs -mkdir /user/<zeppelin user> >> sudo -u hdfs hdfs dfs -chown -R <zeppelin user>t:hdfs /user/<zeppelin >> user> >> >> Good to go! >> >> Cheers, >> Christian >> >> On 3 August 2015 at 17:50, Vadla, Karthik <karthik.va...@intel.com> >> wrote: >> >>> Hi Deepak, >>> >>> >>> >>> I have documented everything here. >>> >>> Please check published document. >>> >>> >>> https://software.intel.com/sites/default/files/managed/bb/bf/Apache-Zeppelin.pdf >>> >>> >>> >>> Thanks >>> >>> Karthik Vadla >>> >>> >>> >>> *From:* ÐΞ€ρ@Ҝ (๏̯͡๏) [mailto:deepuj...@gmail.com] >>> *Sent:* Sunday, August 2, 2015 9:25 PM >>> *To:* users@zeppelin.incubator.apache.org >>> *Subject:* Yarn + Spark + Zepplin ? >>> >>> >>> >>> Hello, >>> >>> I would like to try out Zepplin and hence i got a 7 node Hadoop cluster >>> with spark history server setup. I am able to run sample spark applications >>> on my YARN cluster. >>> >>> >>> >>> I have no clue how to get zepplin to connect to this YARN cluster. Under >>> https://zeppelin.incubator.apache.org/docs/install/install.html i see MASTER >>> to point to spark master. I do not have a spark master running. >>> >>> >>> >>> How do i get Zepplin to be able to read data from YARN cluster ? Please >>> share documentation. >>> >>> >>> >>> Regards, >>> >>> Deepak >>> >>> >>> >> >> >> >> -- >> Christian Tzolov <http://www.linkedin.com/in/tzolov> | Solution >> Architect, EMEA Practice Team | Pivotal <http://pivotal.io/> >> ctzo...@pivotal.io|+31610285517 >> > > > > -- > Deepak > >
