one is used when exactly one task has finished -- that means you now have free resources on just that one executor, so you only need to look for something to schedule on that one.
the other one is used when you want to schedule everything you can across the entire cluster. For example, you have just submitted a new taskset, so you want to try to use any idle resources across the entire cluster. Or, for delay scheduling, you periodically retry all idle resources, in case they locality delay has expired. you could eliminate the version which takes an executorId, and always make offers across all idle hosts -- it would still be correct. Its a small efficiency improvement to avoid having to go through the list of all resources. On Thu, Jan 26, 2017 at 5:48 AM, Jacek Laskowski <ja...@japila.pl> wrote: > Hi, > > Why are there two (almost) identical makeOffers in > CoarseGrainedSchedulerBackend [1] and [2]? I can't seem to figure out > why they are there and am leaning towards considering one a duplicate. > > WDYT? > > [1] https://github.com/apache/spark/blob/master/core/src/ > main/scala/org/apache/spark/scheduler/cluster/ > CoarseGrainedSchedulerBackend.scala#L211 > > [2] https://github.com/apache/spark/blob/master/core/src/ > main/scala/org/apache/spark/scheduler/cluster/ > CoarseGrainedSchedulerBackend.scala#L229 > > Pozdrawiam, > Jacek Laskowski > ---- > https://medium.com/@jaceklaskowski/ > Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark > Follow me at https://twitter.com/jaceklaskowski > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
