> On 2012-03-26 23:09:55, Matei Zaharia wrote: > > README, line 141 > > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141> > > > > But the slaves-env.sh file needs to be on all the nodes, right? If the > > intent is to ship the options automatically, that would make it hard for > > the user to configure, say, different numbers of CPUs and memory per host, > > or a different LIBPROCESS_IP, or stuff like that. > > Andy Konwinski wrote: > I chatted with Matei offline about this. The current flow does not ship > the slaves-env.sh file over. If people wanted to set up unique per-machine > slave configurations, they can create a config file somewhere on each such > machine containing the special settings for that machine, and then set their > slaves-env.sh to point to that via the > --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in > those files would override anything passed over via the deploy scripts as env > variables. > > This raises the important issue of what the order of precedence is for > options that are set in multipole of {env variable, configuration file, > command line} (note that this is not a new issue). > https://github.com/mesos/mesos/wiki/Configuration currently says > "Configuration values are searched for first on the command line, then in the > environment, and then in the config file." but we should expand that comment > to say explicitly what happens if a variable is set in more than one place > (maybe even give an example). > > Matei Zaharia wrote: > Just to make it clear, the goal of the deploy scripts was never to ship > the configuration automatically to all the machines. The old deploy-to-slaves > script happened to do that, but its main purpose is to let people developing > Mesos modify and recompile the code locally, then sync it to all the > machines. I think it's fine to tell the user to configure each machine > separately -- in fact they won't have to do a single thing by default because > most users don't set any config settings on slaves. Otherwise, the mechanisms > for "global" and "local" configuration should be consistent (both environment > variables or both files).
By the way, my suggestion with this kind of stuff is that if you don't want to spend a long time thinking about how to do it, find another project that uses a similar configuration process, and copy what it does. Otherwise, you will repeat mistakes that others have found and fixed in the past. This is why the old scripts were based on per-machine config files in a key-value pair format -- because a lot of other cluster software does that. Same thing with the localstatedir / .template file issue. If in doubt go with .template and tell the user to copy it. - Matei ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4491/#review6383 ----------------------------------------------------------- On 2012-03-26 21:38:18, Benjamin Hindman wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/4491/ > ----------------------------------------------------------- > > (Updated 2012-03-26 21:38:18) > > > Review request for mesos, Andy Konwinski and Matei Zaharia. > > > Summary > ------- > > Updated deploy scripts to work with latest version of Mesos. Eliminated > 'deploy-to-slaves' functionality, instead we assume the binaries and > libraries will get installed via other means. Updated the semantics of where > configuration options are gathered from since a single configuration file was > insufficient given conflicting configuration options between binaries (e.g., > mesos-master and mesos-slave). Added a section to the README describing these > scripts. > > > Diffs > ----- > > README 8166b04 > configure.ac 7a0d89f > src/Makefile.am 090c07a > src/conf/deploy-env.sh.template a4565e2 > src/configurator/configurator.hpp ecd1365 > src/deploy/deploy-to-slaves 0e10486 > src/deploy/mesos-common-env.sh.in PRE-CREATION > src/deploy/mesos-daemon b3144c0 > src/deploy/mesos-daemon.sh.in PRE-CREATION > src/deploy/mesos-deploy-env.sh.in PRE-CREATION > src/deploy/mesos-env.sh c3dfda2 > src/deploy/mesos-masters-env.sh.in PRE-CREATION > src/deploy/mesos-slaves-env.sh.in PRE-CREATION > src/deploy/mesos-start-masters.sh.in PRE-CREATION > src/deploy/mesos-start-slaves.sh.in PRE-CREATION > src/deploy/mesos-start.sh.in PRE-CREATION > src/deploy/mesos-stop-masters.sh.in PRE-CREATION > src/deploy/mesos-stop-slaves.sh.in PRE-CREATION > src/deploy/mesos-stop.sh.in PRE-CREATION > src/deploy/start-masters a450ee0 > src/deploy/start-mesos 9e5b922 > src/deploy/start-slaves b3b931f > src/deploy/stop-masters f2aa9e2 > src/deploy/stop-mesos 679a09e > src/deploy/stop-slaves 141acd4 > src/master/main.cpp f31212b > src/tests/master_detector_tests.cpp PRE-CREATION > > Diff: https://reviews.apache.org/r/4491/diff > > > Testing > ------- > > N/A > > > Thanks, > > Benjamin > >
