Patrick Wendell created SPARK-2269:
--------------------------------------
Summary: Clean up and add unit tests for resourceOffers in
MesosSchedulerBackend
Key: SPARK-2269
URL: https://issues.apache.org/jira/browse/SPARK-2269
Project: Spark
Issue Type: Bug
Components: Mesos
Reporter: Patrick Wendell
This function could be simplified a bit. We could re-write it without
offerableIndices or creating the mesosTasks array as large as the offer list.
There is a lot of logic around making sure you get the correct index into
mesosTasks and offers, really we should just build mesosTasks directly from the
offers we get back. To associate the tasks we are launching with the offers we
can just create a hashMap from the slaveId to the original offer.
The basic logic of the function is that you take the mesos offers, convert them
to spark offers, then convert the results back.
One thing we should check is whether Mesos guarantees that it won't give two
offers for the same worker. That would make things much more complicated.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
