Hi Denis, Thank you very much for your good suggestions! If my Spark running on YARN and there are three master_hosts running, should I pick up one randomly? I will give a try to specify the master_host and master_port.
Thank you! Mei On Tue, Jun 7, 2016 at 10:26 AM, Denis Magda <[email protected]> wrote: > Hi, > > I’m not an expert in this area however have you tried to specify a Spark > master like the following documentation says? > > https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell#working-with-spark-shell > > If you did try please share the full logs, someone from the community will > respond. > > — > Denis > > > On Jun 6, 2016, at 5:58 PM, Hongmei Zong <[email protected]> wrote: > > > > Hi there, > > > > I would like to use "Ignite for Spark" to save the states of Spark jobs > in > > memory and those states can be used for later jobs. For Shared > Deployment, > > the document only offer two ways to deploy Ignite cluster. First is the > > standalone deployment, second is MESOS deployment. But Our Spark clusters > > are running on YARN. My question is: is it possible to run Ignite for > Spark > > on YARN deployment??? > > > > I downloaded and installed Ignite on my machine. Next, I referenced the > link > > below for YARN Deployment. > > http://apacheignite.gridgain.org/docs/yarn-deployment > > > > I created the cluster.properties file and ran the application using the > > command: hadoop jar > > /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar > > /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar > > /u/hongmei/apache-ignite/config/cluster.properties > > > > Form the YARN console, The YARN ignite application works ok. It shows > > running, and 16 containers are allocated for the ignite application. > > > > After this step, what should I do in order to run Spark with Ignite on > YARN > > deployment?? > > > > Many Thanks!!! > > > > Mei > > > > > > > > -- > > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-Spark-on-YARN-Deployment-tp5465.html > > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > >
