Re: ensuring a particular task is deployed to "all" Mesos Worker hosts

2017-07-01 Thread Erik Weathers
A. That would be satisfactory yes. The framework is already "smart" (read as "overly complicated") since we handle reassembly of fragmented offers, amongst other intricate coding details. So it does seem possible to add the kind of logic you suggest! Thanks for the idea James! My team curre

Re: ensuring a particular task is deployed to "all" Mesos Worker hosts

2017-07-01 Thread James Peach
> On Jul 1, 2017, at 11:14 AM, Erik Weathers wrote: > > Thanks for the info Kevin. Seems there's no JIRAs nor design docs floating > about yet for "admin tasks" or "daemon sets". > > Just FYI, this is the ticket in Storm for the problem I've been mentioning: > > https://issues.apache.org/jir

Re: ensuring a particular task is deployed to "all" Mesos Worker hosts

2017-07-01 Thread Erik Weathers
Thanks for the info Kevin. Seems there's no JIRAs nor design docs floating about yet for "admin tasks" or "daemon sets". Just FYI, this is the ticket in Storm for the problem I've been mentioning: https://issues.apache.org/jira/browse/STORM-1342 I'll update it with the info you've provided belo

Re: ensuring a particular task is deployed to "all" Mesos Worker hosts

2017-07-01 Thread Kevin Klues
What you are describing is a feature we call 'admin tasks' or 'daemon sets'. Unfortunately, there is no direct support for these yet, but we do have plans in the (relatively) near future to start working on it. One of our use cases is exactly what you describe with the logging service. On DC/OS w

Re: ensuring a particular task is deployed to "all" Mesos Worker hosts

2017-07-01 Thread Erik Weathers
That works for our particular use case, and is effectively what *we* do, but renders storm a "strange bird" amongst mesos frameworks. Is there no trickery that could be played with mesos roles and/or reservations? - Erik On Sat, Jul 1, 2017 at 3:57 AM Dick Davies wrote: > If it _needs_ to be t

Re: ensuring a particular task is deployed to "all" Mesos Worker hosts

2017-07-01 Thread Dick Davies
If it _needs_ to be there always then I'd roll it out with whatever automation you use to deploy the mesos workers ; depending on the scale you're running at launching it as a task is likely to be less reliable due to outages etc. ( I understand the 'maybe all hosts' constraint but if it's 'up to