Hi,

if your Spark driver runs in Eclipse, you should be able to get what you
want. That is, create a main() function that creates a SparkContext with
the correct master URL of your cluster and execute it with your standard
Eclipse method. For example, I am using Mesos and the following works fine:

        val conf = new SparkConf()
          .setMaster("zk://HOSTNAME:2181/mesos")
          .setAppName("My Application")
          .setJars("/path/to/my-application-assembly-1.0.jar" :: Nil)
          .set("spark.executor.uri", "/path/to/spark-1.0.0-bin-hadoop2.tgz")
          .set("spark.mesos.coarse", "true")
        val sc = new SparkContext(conf)

However, if you change something in the cluster node processing code, you
will have to redeploy the my-application-assembly-1.0.jar file.

Tobias


On Thu, Jul 31, 2014 at 9:20 AM, nunarob <[email protected]> wrote:

> Hi all,
>
> I'm a new Spark user and I'm interested in deploying java-based Spark
> applications directly from Eclipse. Specifically, I'd like to write a
> Spark/java application, have it directly submitted to a spark cluster, and
> then see the output of the spark execution directly in the eclipse console.
>
> Following the quick start guide, I'm able to package up a spark application
> using maven from within eclipse and then able to manually submit the jar
> using the spark-submit script. Is there anyway to submit the packaged jar
> to
> the spark cluster directly from within eclipse, and to see the output in
> the
> eclipse console?
>
> Apologies if this is documented somewhere - still learning about the entire
> spark ecosystem.
>
> Thanks!
>
> -Rob
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Deploying-spark-applications-from-within-Eclipse-tp11001.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

Reply via email to