Thank you for asking :)
The idea is pretty simple (and not even original, it's the "Facade
pattern"): essentially we would define a YAML syntax for the various
configuration flags that Mesos ({Master, Agent}) define, possibly grouping
them by "topic" and then write a few python classes, that read in the YAML
configuration and emit a set of flags that reflect the caller intent, then
invoke the appropriate ./bin/mesos-{master,slave}.sh script.
(variation on the theme for installed packages)
E.g.:
# Configuration /opt/mesos/config-master.yml
ip: 192.168.1.101
zk:
url: zk://192.168.1.220:2181
quorum: 3
directories:
work: /var/run/mesos
log: /var/log/mesos/master.log
Execute with:
./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
--no-ssl \
[other script-specific options]
would generate something along the lines of:
/opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
--zk=zk://192.168.1.220:2181 --quorum=3 \
--work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
(example greatly contrived and probably pointless, but hopefully gives you
the idea).
The point is that leveraging Python and the existing YAML libraries, this
would be pretty quick to implement, would leave Mesos per se completely
untouched, and would be easy to maintain (add/remove flags as needed;
enforce deprecation cycles; etc.)
There's probably a ton of detail I'm missing, but you get the point.
Cheers,
M.
On Tuesday, November 24, 2015, tommy xiao <[email protected]> wrote:
> how to do that? Marco
>
> 2015-11-24 3:54 GMT+08:00 Marco Massenzio <[email protected]
> <javascript:;>>:
>
> > I was thinking along the same lines, with a slightly more "modern"
> approach
> > :)
> >
> > My idea was to write a thin Python layer that reads a configuration YAML
> > file (possibly with "override" flags) and then invokes
> mesos-{master,slave}
> > with the appropriate flags.
> >
> > By using YAML, we would also gain the ability to have more intuitive
> > syntax; grouping flags by function; and make it extensible.
> >
> > Any takers who may want to work together on this one?
> >
> > --
> > *Marco Massenzio*
> > Distributed Systems Engineer
> > http://codetrips.com
> >
> > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <[email protected]
> <javascript:;>> wrote:
> >
> > > We had this discussion a long while ago and the argument was that we
> > > already have a configuration file of sorts. It is a bash script with
> one
> > > flag per line that sets the environment. Is this not sufficient?
> > >
> > > Example:
> > >
> > > config.sh
> > > ------------
> > > MESOS_WORK_DIR="/var/run/mesos"
> > > MESOS_QUORUM=2
> > >
> > > $ source config.sh
> > > $ ./bin/mesos-master.sh
> > >
> > >
> > >
> > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <[email protected]
> <javascript:;>>
> > > wrote:
> > >
> > > > Thanks for bringing this topic up Alex. I have been thinking about
> the
> > > same
> > > > and was wondering if we can have subsystem specific flags and some
> > scheme
> > > > where there can be a namespace instead of a flat space. And as alex
> > > > suggested, a configuration to represent this?
> > > >
> > > > -jojy
> > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <[email protected]
> <javascript:;>> wrote:
> > > >
> > > > > please file a issue to tracking this proposal
> > > > >
> > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <[email protected]
> <javascript:;>>:
> > > > >
> > > > > > +1, that's helpful :).
> > > > > >
> > > > > > For the detail of implementing such as auto re-load, I think we
> can
> > > let
> > > > > > owner/shepherd to decide :).
> > > > > >
> > > > > > ----
> > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > > +86-10-8245 4084 | [email protected] <javascript:;> |
> http://k82.me
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <[email protected]
> <javascript:;>> wrote:
> > > > > >
> > > > > > > +1 I think it would be helpful.
> > > > > > >
> > > > > > > This may be orthogonal / feature creep, but would it be
> possible
> > to
> > > > > have
> > > > > > > the config file be able to be safely reloaded with a HUP or
> > > > appropriate
> > > > > > > signal?
> > > > > > >
> > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> [email protected] <javascript:;>
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1000, introducing a new configuration file for mesos master
> > and
> > > > > slave
> > > > > > > can
> > > > > > > > help end user take the configuration file as the source of
> all
> > > > flags.
> > > > > > > >
> > > > > > > > The OpenStack is also using same way to manage all of the
> > flags,
> > > it
> > > > > is
> > > > > > > > putting all flags into a configuration file and the
> > configuration
> > > > > file
> > > > > > > > including all flag examples. Most of the flags are disabled
> by
> > > > > default
> > > > > > > and
> > > > > > > > the end user can just enable those flags based on his
> > > requirement.
> > > > > > > >
> > > > > > > > Also the flags in the configuration file can be classified to
> > > > > different
> > > > > > > > groups for a better management, and mesos can also follow
> this
> > to
> > > > > > > classify
> > > > > > > > those flags to different groups, such as ACL, Cluster,
> > framework
> > > > etc.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > [email protected] <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hey guys,
> > > > > > > > >
> > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen that we
> > > went
> > > > > > from a
> > > > > > > > > handful of flags to around 42 supported flags in the
> master.
> > At
> > > > > this
> > > > > > > > point
> > > > > > > > > I’m wondering if perhaps we should support a configuration
> > file
> > > > in
> > > > > > > > > conjunction (or instead of) with all the command flags.
> > > > > > > > >
> > > > > > > > > My intuition is that it will make it easier for operators
> as
> > > well
> > > > > as
> > > > > > > for
> > > > > > > > > debuggers to be able to replicate configurations easier.
> > > > > > > > >
> > > > > > > > > Any comments on this idea?
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > /adam
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Deshi Xiao
> > > > > Twitter: xds2000
> > > > > E-mail: xiaods(AT)gmail.com
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>
--
--
*Marco Massenzio*
Distributed Systems Engineer
http://codetrips.com
