Hello,
I'm hoping to understand exactly what happens when a spark compiled app is
submitted to a spark stand-alone cluster master. Say, our master is A, and
workers are W1 and W2. Client machine C is submitting an app to the master
using spark-submit. Here's what I think happens?
* C submits jar (possibly uber jar) to A. A starts execution and sends
partitions to W1 and W2 to carry out work. Results are sent back to A. Results
are stored in output files / tables according to the application. W1 and W2 may
also be reading and writing data to and from sources. The submission from C is
fire and forget, and the final results aren't sent back to C.
Is this correct?
I noticed something about the submitting processes working as the driver
application for Spark stand alone. That would mean the above is wrong. Is there
some information about exactly what happens when I submit an app to the Spark
master in a stand alone cluster?
Thanks,
Ashic.
