On Tue, Nov 24, 2015 at 10:45 AM, Adam Avilla <[email protected]> wrote: > Is there any way to support HUP reloading of config with this design > paradigm?
Can you please elaborate what do you mean by "support HUP reloading"? I don't suppose anyone really runs Mesos in production inside a terminal? At any rate, being a facade, this will support exactly what Mesos supports (and, currently, no, it does not support dynamic configuration reload). > Is HUPing mesos even possible / helpful? From an end user > perspective it sounds useful, but I am not sure if even possible for mesos > to accomplish safely. > If you are running Mesos in multi-Master configuration (using, eg, Zookeeper) yes, it is safe to re-start the process, so long as you always maintain a --quorum of servers running. In fact, that's the recommended way to do a Mesos cluster upgrade. But, again, I don't really think this is what you meant by "HUPing Mesos"? (I mean, if you `kill -1 <PID>` this seems to just terminate Master - at least on OSX) > On Nov 24, 2015 10:04 AM, "haosdent" <[email protected]> wrote: > > > Use YAML sounds great. Suppose I am a framework developer, use a widely > > used format would be helpful. Because framework maybe written by > different > > programming languages, if use some rare formats, I have to write some > code > > to parse the configuration file when I want to get some fields from it. > > > > On Wed, Nov 25, 2015 at 1:57 AM, Tomek Janiszewski <[email protected]> > > wrote: > > > > > Hi all > > > > > > It looks simple for the first sight. Marco, I can work on it. > > > > > > - > > > Tomek > > > > > > wt., 24 lis 2015, 17:49 Marco Massenzio użytkownik < > [email protected]> > > > napisał: > > > > > > > Thank you for asking :) > > > > > > > > The idea is pretty simple (and not even original, it's the "Facade > > > > pattern"): essentially we would define a YAML syntax for the various > > > > configuration flags that Mesos ({Master, Agent}) define, possibly > > > grouping > > > > them by "topic" and then write a few python classes, that read in the > > > YAML > > > > configuration and emit a set of flags that reflect the caller intent, > > > then > > > > invoke the appropriate ./bin/mesos-{master,slave}.sh script. > > > > > > > > (variation on the theme for installed packages) > > > > > > > > E.g.: > > > > > > > > # Configuration /opt/mesos/config-master.yml > > > > > > > > ip: 192.168.1.101 > > > > > > > > zk: > > > > url: zk://192.168.1.220:2181 > > > > quorum: 3 > > > > > > > > directories: > > > > work: /var/run/mesos > > > > log: /var/log/mesos/master.log > > > > > > > > Execute with: > > > > > > > > ./mesos-wrapper.py --run master --config > /opt/mesos/config-master.yml > > > > --no-ssl \ > > > > [other script-specific options] > > > > > > > > would generate something along the lines of: > > > > > > > > /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \ > > > > --zk=zk://192.168.1.220:2181 --quorum=3 \ > > > > --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log > > > > > > > > (example greatly contrived and probably pointless, but hopefully > gives > > > you > > > > the idea). > > > > > > > > The point is that leveraging Python and the existing YAML libraries, > > this > > > > would be pretty quick to implement, would leave Mesos per se > completely > > > > untouched, and would be easy to maintain (add/remove flags as needed; > > > > enforce deprecation cycles; etc.) > > > > > > > > There's probably a ton of detail I'm missing, but you get the point. > > > > > > > > Cheers, > > > > M. > > > > > > > > On Tuesday, November 24, 2015, tommy xiao <[email protected]> wrote: > > > > > > > > > how to do that? Marco > > > > > > > > > > 2015-11-24 3:54 GMT+08:00 Marco Massenzio <[email protected] > > > > > <javascript:;>>: > > > > > > > > > > > I was thinking along the same lines, with a slightly more > "modern" > > > > > approach > > > > > > :) > > > > > > > > > > > > My idea was to write a thin Python layer that reads a > configuration > > > > YAML > > > > > > file (possibly with "override" flags) and then invokes > > > > > mesos-{master,slave} > > > > > > with the appropriate flags. > > > > > > > > > > > > By using YAML, we would also gain the ability to have more > > intuitive > > > > > > syntax; grouping flags by function; and make it extensible. > > > > > > > > > > > > Any takers who may want to work together on this one? > > > > > > > > > > > > -- > > > > > > *Marco Massenzio* > > > > > > Distributed Systems Engineer > > > > > > http://codetrips.com > > > > > > > > > > > > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone < > [email protected] > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > We had this discussion a long while ago and the argument was > that > > > we > > > > > > > already have a configuration file of sorts. It is a bash script > > > with > > > > > one > > > > > > > flag per line that sets the environment. Is this not > sufficient? > > > > > > > > > > > > > > Example: > > > > > > > > > > > > > > config.sh > > > > > > > ------------ > > > > > > > MESOS_WORK_DIR="/var/run/mesos" > > > > > > > MESOS_QUORUM=2 > > > > > > > > > > > > > > $ source config.sh > > > > > > > $ ./bin/mesos-master.sh > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese < > > > [email protected] > > > > > <javascript:;>> > > > > > > > wrote: > > > > > > > > > > > > > > > Thanks for bringing this topic up Alex. I have been thinking > > > about > > > > > the > > > > > > > same > > > > > > > > and was wondering if we can have subsystem specific flags and > > > some > > > > > > scheme > > > > > > > > where there can be a namespace instead of a flat space. And > as > > > alex > > > > > > > > suggested, a configuration to represent this? > > > > > > > > > > > > > > > > -jojy > > > > > > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <[email protected] > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > > > please file a issue to tracking this proposal > > > > > > > > > > > > > > > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma < > [email protected] > > > > > <javascript:;>>: > > > > > > > > > > > > > > > > > > > +1, that's helpful :). > > > > > > > > > > > > > > > > > > > > For the detail of implementing such as auto re-load, I > > think > > > we > > > > > can > > > > > > > let > > > > > > > > > > owner/shepherd to decide :). > > > > > > > > > > > > > > > > > > > > ---- > > > > > > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer > > > > > > > > > > Platform Symphony/DCOS Development & Support, STG, IBM > GCG > > > > > > > > > > +86-10-8245 4084 | [email protected] > <javascript:;> | > > > > > http://k82.me > > > > > > > > > > > > > > > > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla < > [email protected] > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > > > > > > > +1 I think it would be helpful. > > > > > > > > > > > > > > > > > > > > > > This may be orthogonal / feature creep, but would it be > > > > > possible > > > > > > to > > > > > > > > > have > > > > > > > > > > > the config file be able to be safely reloaded with a > HUP > > or > > > > > > > > appropriate > > > > > > > > > > > signal? > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu < > > > > > [email protected] <javascript:;> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1000, introducing a new configuration file for mesos > > > > master > > > > > > and > > > > > > > > > slave > > > > > > > > > > > can > > > > > > > > > > > > help end user take the configuration file as the > source > > > of > > > > > all > > > > > > > > flags. > > > > > > > > > > > > > > > > > > > > > > > > The OpenStack is also using same way to manage all of > > the > > > > > > flags, > > > > > > > it > > > > > > > > > is > > > > > > > > > > > > putting all flags into a configuration file and the > > > > > > configuration > > > > > > > > > file > > > > > > > > > > > > including all flag examples. Most of the flags are > > > disabled > > > > > by > > > > > > > > > default > > > > > > > > > > > and > > > > > > > > > > > > the end user can just enable those flags based on his > > > > > > > requirement. > > > > > > > > > > > > > > > > > > > > > > > > Also the flags in the configuration file can be > > > classified > > > > to > > > > > > > > > different > > > > > > > > > > > > groups for a better management, and mesos can also > > follow > > > > > this > > > > > > to > > > > > > > > > > > classify > > > > > > > > > > > > those flags to different groups, such as ACL, > Cluster, > > > > > > framework > > > > > > > > etc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas < > > > > > > > > > > > [email protected] <javascript:;>> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen > > > that > > > > we > > > > > > > went > > > > > > > > > > from a > > > > > > > > > > > > > handful of flags to around 42 supported flags in > the > > > > > master. > > > > > > At > > > > > > > > > this > > > > > > > > > > > > point > > > > > > > > > > > > > I’m wondering if perhaps we should support a > > > > configuration > > > > > > file > > > > > > > > in > > > > > > > > > > > > > conjunction (or instead of) with all the command > > flags. > > > > > > > > > > > > > > > > > > > > > > > > > > My intuition is that it will make it easier for > > > operators > > > > > as > > > > > > > well > > > > > > > > > as > > > > > > > > > > > for > > > > > > > > > > > > > debuggers to be able to replicate configurations > > > easier. > > > > > > > > > > > > > > > > > > > > > > > > > > Any comments on this idea? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > /adam > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Deshi Xiao > > > > > > > > > Twitter: xds2000 > > > > > > > > > E-mail: xiaods(AT)gmail.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Deshi Xiao > > > > > Twitter: xds2000 > > > > > E-mail: xiaods(AT)gmail.com > > > > > > > > > > > > > > > > > -- > > > > -- > > > > *Marco Massenzio* > > > > Distributed Systems Engineer > > > > http://codetrips.com > > > > > > > > > > > > > > > -- > > Best Regards, > > Haosdent Huang > > >
