> On June 12, 2013, 7:06 p.m., Vinod Kone wrote: > > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines > > 444-446 > > <https://reviews.apache.org/r/11118/diff/4/?file=301280#file301280line444> > > > > Do you need a Math.floor if you are casting this to an int?
Java does an implicit floor on double -> int conversion, but I like to be explicit to avoid ambiguity. I can remove it if necessary. > On June 12, 2013, 7:06 p.m., Vinod Kone wrote: > > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java, lines > > 471-488 > > <https://reviews.apache.org/r/11118/diff/4/?file=301280#file301280line471> > > > > How did you come up with the fair allocation and 60% spread policies? > > Is this what the default Hadoop scheduler does? > > > > It would be really awesome, if we can ask the underlying scheduler how > > to distribute the slots between maps and reduces, when we don't have > > sufficient resources. Have you looked into that? I just chose the value arbitrarily. It has shown good results through testing. I have not looked into that, but I'll investigate. Since jobtracker schedulers are pluggable, it might be dependent on particular scheduler implementation. I'm not sure there's a suitable interface for determining how the underlying scheduler wants to allocate the slots. - Brenden ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11118/#review21813 ----------------------------------------------------------- On June 6, 2013, 2:12 a.m., Brenden Matthews wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11118/ > ----------------------------------------------------------- > > (Updated June 6, 2013, 2:12 a.m.) > > > Review request for mesos. > > > Description > ------- > > Be more intelligent about slot allocation. > > When we get a resource offer, we'll try to be clever about how we > allocate slots to the offers. Ideally we want to use as much of the > resources available when we have a lot of pending tasks, and we also > don't want to use more resources than we actually need. > > Review: https://reviews.apache.org/r/11118 > > > Diffs > ----- > > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java > afe401f5265e3d9494af7eace42eec45943184a3 > > Diff: https://reviews.apache.org/r/11118/diff/ > > > Testing > ------- > > Used in production at airbnb. > > > Thanks, > > Brenden Matthews > >
