Assume I am running a Spark Client Program in client mode and Spark Cluster
in Stand alone mode.

I want some clarification of the following things

1. Build a DAG
2. DAG Scheduler
3. TASK Scheduler

I want to which of the above part is done by SPARK CLIENT and which of the
above parts are done by SPARK MASTER in the stand alone case?

Building a DAG clearly looks like Spark Client Program
DAG Scheduler is also in the Spark Client Program
Task Scheduler is done by the SPARK MASTER.

is this correct? Also, Does Spark Client every instruct Spark Workers
directly on what transformations to run or the communication is just
unidirectional in the sense that Spark Workers communicate to Spark client
only when returning the results ?


Reply via email to