As a starting point, you might be able to cook up something involving
{{mesos.instance}} as a lookup key to a pystachio list. You do have a
unique integer task number per instance to work with.
cf.
http://aurora.apache.org/documentation/latest/configuration-reference/#template-namespaces
On Mon, Jan 11, 2016 at 8:05 PM Bill Farner <wfar...@apache.org> wrote:
> I agree that this appears necessary when parameters are needed to define
> the runtime environment of the task (in this case, setting up the docker
> container).
>
> What's particularly interesting here is that this would call for the
> scheduler to fill in the parameter values prior to launching each task.
> Using pystachio variables for this is certainly the most natural in the
> DSL, but becomes a paradigm shift since the scheduler is currently ignorant
> of pystachio.
>
> Possibly only worth mentioning for shock value, but in the DSL this starts
> to look like lambdas pretty quickly.
>
> On Mon, Jan 11, 2016 at 7:46 PM, Mauricio Garavaglia <
> mauriciogaravag...@gmail.com> wrote:
>
> > Hi guys,
> >
> > We are using the docker rbd volume plugin
> > <
> https://ceph.com/planet/getting-started-with-the-docker-rbd-volume-plugin>
> > to
> > provide persistent storage to the aurora jobs that runs in the
> containers.
> > Something like:
> >
> > p = [Parameter(name='volume', value='my-ceph-volume:/foo'), ...]
> > jobs = [ Service(..., container = Container(docker = Docker(...,
> parameters
> > = p)))]
> >
> > But in the case of jobs with multiple instances it's required to start
> each
> > container using different volumes, in our case different ceph images.
> This
> > could be achieved by deploying, for example, 10 instances and then update
> > each one independently to use the appropiate volume. Of course this is
> > quite inconvenient, error prone, and adds a lot of logic and state
> outside
> > aurora.
> >
> > We where thinking if it would make sense to have a way to parameterize
> the
> > task instances, in a similar way that is done with portmapping for
> example.
> > In the job definition have something like
> >
> > params = [
> > Parameter( name='volume',
> > value='service-{{instanceParameters.volume}}:/foo' )
> > ]
> > ...
> > jobs = [
> > Service(
> > name = 'logstash',
> > ...
> > instanceParameters = { "volume" : ["foo", "bar", "zaa"]},
> > instances = 3,
> > container = Container(
> > docker = Docker(
> > image = 'image',
> > parameters = params
> > )
> > )
> > )
> > ]
> >
> >
> > Something like that, it would create 3 instances of the tasks, each one
> > running in a container that uses the volumes foo, bar, and zaa.
> >
> > Does it make sense? I'd be glad to work on it but I want to validate the
> > idea with you first and hear comments about the api/implementation.
> >
> > Thanks
> >
> >
> > Mauricio
> >
>
