> On Jan. 12, 2016, 10:33 p.m., Adam B wrote: > > src/master/master.hpp, line 1683 > > <https://reviews.apache.org/r/42053/diff/5/?file=1194904#file1194904line1683> > > > > Could you call this `masterFlags` so it doesn't get confused with > > FrameworkInfo parameters? > > Even better would be to pass only the specific flag you care about. > > > > Or, maybe we should finally separate the caching into a common class so > > you only have to set the sizes for the circular buffer once. We long ago > > discussed refactoring the circular buffer so we could enable > > storage/retrieval to/from an archive of the data that is no longer cached > > in memory. > > > > I'd be ok with any of these, so I'll let you and your Shepherd choose.
Me and Ben (my shepherd) talked about this and decided that passing flags might be better in case we decide to add more flags we want to pass through in the future. Renaming this field makes sense. Done. > On Jan. 12, 2016, 10:33 p.m., Adam B wrote: > > src/master/flags.cpp, lines 464-472 > > <https://reviews.apache.org/r/42053/diff/5/?file=1194903#file1194903line464> > > > > Since you're adding new command-line flags, you should also update > > `docs/configuration.md` so the 0.27 release managers don't have to do it > > for you. Done. Website changes while under review can be viewed here: http://c99.millennium.berkeley.edu:4567/documentation/latest/configuration/ - Kevin ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42053/#review114077 ----------------------------------------------------------- On Jan. 12, 2016, 11:07 p.m., Kevin Klues wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42053/ > ----------------------------------------------------------- > > (Updated Jan. 12, 2016, 11:07 p.m.) > > > Review request for mesos and Ben Mahler. > > > Bugs: MESOS-3307 > https://issues.apache.org/jira/browse/MESOS-3307 > > > Repository: mesos > > > Description > ------- > > The default size of the buffers used to hold the state of completed > tasks/frameworks is very large. However, many frameworks don't care much > about this information when requesting a master's state. Moreover, if a > large number of frameworks request this state simultaneously, the > master can quickly become overwhelmed because the process of generating > this state both blocks the master and takes up a lot of cycles. By > allowing the master to configure the size of the buffers used to hold > this state, we give it the power to significantly reduce the amount of > state it needs to maintain. > > This change allows the master to limit the size of this state via > command line flags. > > This commit is based on a pull request generated by Felix Bechstein at: > https://github.com/apache/mesos/pull/82 > > Review: https://reviews.apache.org/r/42053 > > > Diffs > ----- > > docs/configuration.md cbe7f5a338a0fc350c4b6c0e2f1f48bd0869ac34 > src/master/constants.hpp ebab341e58035d4b579828add752c1ee37efeb95 > src/master/constants.cpp 77dd31430776e4f24e6e074c1470edcb19e58449 > src/master/flags.hpp d923b1b0444d7e9023f1db4cbc4f7d4b84c20ff5 > src/master/flags.cpp 88909590ff421421659e6faac7f3444bdc57b630 > src/master/master.hpp f02d165874fa8023675e545793de699aeecae29b > src/master/master.cpp 5268408fc63a28afabc27cba96d3ecb360608a65 > > Diff: https://reviews.apache.org/r/42053/diff/ > > > Testing > ------- > > On Darwin I launched a master with: > > ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos > --max_completed_tasks_per_framework=2 --max_completed_frameworks=1 > > and a slave with: > > ./bin/mesos-slave.sh --master=127.0.0.1:5050 > > and then ran a bunch of instances of: > > ./src/test-framework --master=127.0.0.1:5050 > > each of which runs 5 tasks to completion > > I then ran: > > curl http://localhost:5050/tasks > > and verified that only 1 framework and 2 of its completed tasks were given > back to me in the json that was returned. I repeated this for a number of > other configurations with max_completed_frameworks=0..2 and > max_completed_tasks_per_framework=0..5 and verified visually that the proper > number of tasks/frameworks were being returned by the /tasks endpoint. > > There is alos now a unit test for this as reviewed here: > https://reviews.apache.org/r/42053/ > > > Thanks, > > Kevin Klues > >
