Re: Running services on all slaves

2015-01-08 Thread Itamar Ostricher
Thanks again! We're not (yet) using docker (or any other form of containerization for that matter). I think in the meantime the approach of sub-advertising resources should work for us. As we're spinning clusters up and down on demand on GCE, I can probably do this pretty easily by adding these

Re: Running services on all slaves

2015-01-08 Thread Tim Chen
Hi Itamar, You can pass the amount of CPU and memory that the slave is advertising to the master for scheduling as part of the resources slave flag. So you can only schedule 12 cpus and leave 4 for your services if you want. That said, there are discussions about launching multiple tasks

Re: Running services on all slaves

2015-01-08 Thread Tom Arnfeld
That's a great point Itamar, and something we discussed quite some time ago here but never implemented. These are the first two options that spring to mind that I can remember... - Are you using docker containers for your tasks? Why not use containers pre-configured on the box for these

Re: Running services on all slaves

2015-01-06 Thread Tom Arnfeld
I completely agree with Charles, though I think I can appreciate what you're trying to do here. Take the log aggregation service as an example, you want that on every slave to aggregate logs, but want to avoid using yet another layer of configuration management to deploy it. I'm of the

Re: Running services on all slaves

2015-01-06 Thread David Greenberg
We've been experimenting with using Marathon to do this. We've found a couple issues in making in smooth, but those bugs are all in the process of being fixed. Within the next few weeks, I think that Marathon will be a solid choice for this use case. On Tue, Jan 6, 2015 at 4:05 AM, Itamar