We aggregate all the offers for a host, such that placement engine can pack multiple tasks that can be launched on this host using aggregated resources. If offers are unused for that host then they will be implicitly declined.
Placement cycle has two components - determine tasks that can be launched on this host and enque those tasks to be launched. - second is to deque and launch them. During this placement cycle, offers can be rescinded and accordingly placement has to be adjusted. For that we maintain second map of offer id -> host name. Benefits of adding host name and agent id in rescind offer callback. - Mutex locks to synchronize both maps, leads to some performance hit. - Managing second map, is more code and prone to bugs. - Little overhead on heap memory and GC. On Wed, May 2, 2018 at 5:08 PM Benjamin Mahler <bmah...@apache.org> wrote: > I'm a -1 on adding redundant information in the message. > > The scheduler can maintain an index of offers by offer id to address this > issue: > > hostname -> offers > offer_id -> offer > > On Wed, May 2, 2018 at 11:39 AM, Vinod Kone <vinodk...@apache.org> wrote: > > > Can I ask why you are indexing the offers by hostname? Is it to better > > handle agent removal / unreachable signal? > > > > Looking at the code > > < > https://github.com/apache/mesos/blob/master/src/master/master.cpp#L11036> > > , > > I think master has the requested information (agent id, hostname) so we > can > > include it in the rescind message! > > > > But there are couple things to discuss. > > > > The extra information to be included in rescind message is technically > > redundant. So we need to figure out a guideline on what information > should > > be included / not included (e.g., should we include agent IP too) in such > > calls. > > > > Second, adding this extra information in v1 scheduler API would be > > relatively easy. But adding this to v0 API would be hard. Which API do > you > > need to be updated? > > > > > > On Wed, May 2, 2018 at 10:31 AM, Varun Gupta <var...@uber.com> wrote: > > > > > Hi, > > > > > > Currently in our implementation we maintain two maps. > > > > > > Hostname -> []Offers > > > > > > offerID -> Hostname > > > > > > Second map is needed because rescind offers callback only provides > > offerid > > > and we need hostname to do performant lookup in first map. > > > > > > Is it feasible to add hostname or agentid in rescind offers? > > > > > > Thanks, > > > Varun > > > > > >
