Re: Spark Scheduler - Task and job levels - How it Works?

Hi Miguel, On Sun, Jan 6, 2019 at 11:35 AM Miguel F. S. Vasconcelos < miguel.vasconce...@usp.br> wrote: > When an action is performed onto a RDD, Spark send it as a job to the > DAGScheduler; > The DAGScheduler compute the execution DAG based on the RDD's lineage, and > split the job into stages

Spark Scheduler - Task and job levels - How it Works?

Hello, I'm new to Spark and trying to understand how exactly spark scheduler works. In the article /"Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing"/ in section 5.1 Job Scheduling" its said that: / "Whenever a user runs an action (e.g., count or sav