> On July 17, 2013, 2:58 a.m., Ben Mahler wrote: > > src/sched/sched.cpp, line 78 > > <https://reviews.apache.org/r/12603/diff/2/?file=322317#file322317line78> > > > > CACHED_UPDATES_CAPACITY? > > > > Also, maybe add a note that this only works for smaller clusters. If > > there are more than 100 offers in 10 seconds, this will not eliminate any > > duplicates and will only be adding unnecessary overhead, right? > > > > Caching O(20-30) seconds of updates seems nicer, but we'd have to be > > careful of the memory / performance overhead. Thoughts?
Changed it to "MAX_ACKED_UPDATES". I think thats more descriptive. Regarding the number, it was definitely pulled out of thin air. Thanks to your nudge, here are my thoughts. As you alluded, the critical number here is the "number of unique status updates" / "status update retry interval" generated by the cluster. Currently, the retry interval is set to 10s. Note that if the rate of status unique updates is high, the changes in this patch are irrelevant. The scheduler would still get backed up processing the updates. Now regarding MAX_ACKED_UPDATES value, the higher the number the more likely the driver can bypass the scheduler for duplicates. As far as I can tell, the only concern would be the memory consumed. Back of envelope math suggests it is roughly, 2 * 16 bytes (UUID size) * MAX_ACKED_UPDATES. For MAX_ACKED_UPDATES=1000, this value is ~32KB whereas for 10,000 it is ~320KB. IMO, ~320KB overhead for the driver is reasonable, so I will bump this number to 10,000. Thoughts? - Vinod ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12603/#review23230 ----------------------------------------------------------- On July 17, 2013, 1:34 a.m., Vinod Kone wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12603/ > ----------------------------------------------------------- > > (Updated July 17, 2013, 1:34 a.m.) > > > Review request for mesos, Benjamin Hindman and Ben Mahler. > > > Bugs: MESOS-551 > https://issues.apache.org/jira/browse/MESOS-551 > > > Repository: mesos > > > Description > ------- > > See summary. > > > Diffs > ----- > > src/sched/sched.cpp 7ea82e547c612159c9fa24fb6d62e3d2b5f11982 > src/tests/status_update_manager_tests.cpp > 42395324dfe49659bee2229c6573ffef0874d923 > > Diff: https://reviews.apache.org/r/12603/diff/ > > > Testing > ------- > > make check (OSX and Linux) > > > Thanks, > > Vinod Kone > >
