Launching your application in a cluster may be useful in a number of scenarios.
1) In a number of settings in companies, user who want to run jobs do not have ssh access to any of the cluster nodes. So they have to run the Spark driver program on their "local machine" and connect to the Spark standalone cluster's master for launching jobs. That can be pretty slow and can go down if the connection between "local machine" and cluster is severed. Instead, launching applciation inside the cluster allows you to launch the job and forget it. 2) Another additional functionality that launching in the cluster provides is automatic restart of the Spark driver if it fails. This is useful for long running Spark jobs (e.g., Spark Streaming job) where you want to minimize downtime. TD On Mon, Feb 10, 2014 at 6:10 PM, robin_up <robin...@gmail.com> wrote: > Hi > > Could someone kindly explain what the pros/cons are between "Connecting an > Application to a cluster" and "Launching Applications inside a cluster" in > a > standalone cluster? > > I'm referring to the sections in the below doc. I.e. in which situation one > maybe more preferred than the other, or does it matter? > > http://spark.incubator.apache.org/docs/latest/spark-standalone.html > > thanks > Robin > > > > > ----- > -- Robin Li > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Connecting-App-to-cluster-VS-Launching-app-within-cluster-tp1375.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >