Hi Devs, I have started looking at the Mesos core code and came upon following observations. I have seen a few dots but haven’t been able to connect them. Please let me know if you can
As we know the DRF (Dominant Resource Framework) algorithm is used in Mesos to allocate the tasks to the slaves so that the dominant share is equalized (or as equal as possible). The example can be seen in the paper (https://www.cs.berkeley.edu/~alig/papers/drf.pdf). Thus, it looks like Mesos master has the control to allocate the tasks to the slaves (by internally maintaining the status of all tasks for various frameworks and slaves and their offers). On the other hand, there are schedulers (am using Netflix Fenzo) whose job is to figure out the slaves most suitable for the tasks and the API exposed through the Mesos Java API used by Fenzo is — MesosSchedulerDriver.launchTasks(Collection<Offer> offerIds, Collection<TaskInfo> taskIds); The code that I have seen through in Mesos is (master, drf/sorter etc - master.cpp, hierarchical.cpp) showing me on various events (methods in master.cpp) like addFramework(), activeFramework(), addSlave() etc, there is DRF method –>calculateShare() being called in sorter.cpp. But when the tasks are allocate to master and launchTask() method is called in master (by passing the map of tasks assigned to slaves), the slaves which have been calculated by the scheduler and present in the map of tasks and offers get called directly. My question is - if the slaves to execute the tasks are decided by the scheduler, how does the DRF get used by the master in balancing the slaves? Does it not get used for choosing the right slaves for the tasks? Relevant pointers to the code are also welcome. Thanks, Ashish
