There's a chicken and egg problem though. That variable will only be filled
in on the executor, when we're already running in the docker environment.
In this case, the parameter is used to *define* the docker environment.
On Mon, Jan 11, 2016 at 9:07 PM, ben...@gmail.com <ben...@gmail.com> wrote:
> As a starting point, you might be able to cook up something involving
> {{mesos.instance}} as a lookup key to a pystachio list. You do have a
> unique integer task number per instance to work with.
>
> cf.
>
> http://aurora.apache.org/documentation/latest/configuration-reference/#template-namespaces
>
> On Mon, Jan 11, 2016 at 8:05 PM Bill Farner <wfar...@apache.org> wrote:
>
> > I agree that this appears necessary when parameters are needed to define
> > the runtime environment of the task (in this case, setting up the docker
> > container).
> >
> > What's particularly interesting here is that this would call for the
> > scheduler to fill in the parameter values prior to launching each task.
> > Using pystachio variables for this is certainly the most natural in the
> > DSL, but becomes a paradigm shift since the scheduler is currently
> ignorant
> > of pystachio.
> >
> > Possibly only worth mentioning for shock value, but in the DSL this
> starts
> > to look like lambdas pretty quickly.
> >
> > On Mon, Jan 11, 2016 at 7:46 PM, Mauricio Garavaglia <
> > mauriciogaravag...@gmail.com> wrote:
> >
> > > Hi guys,
> > >
> > > We are using the docker rbd volume plugin
> > > <
> >
> https://ceph.com/planet/getting-started-with-the-docker-rbd-volume-plugin>
> > > to
> > > provide persistent storage to the aurora jobs that runs in the
> > containers.
> > > Something like:
> > >
> > > p = [Parameter(name='volume', value='my-ceph-volume:/foo'), ...]
> > > jobs = [ Service(..., container = Container(docker = Docker(...,
> > parameters
> > > = p)))]
> > >
> > > But in the case of jobs with multiple instances it's required to start
> > each
> > > container using different volumes, in our case different ceph images.
> > This
> > > could be achieved by deploying, for example, 10 instances and then
> update
> > > each one independently to use the appropiate volume. Of course this is
> > > quite inconvenient, error prone, and adds a lot of logic and state
> > outside
> > > aurora.
> > >
> > > We where thinking if it would make sense to have a way to parameterize
> > the
> > > task instances, in a similar way that is done with portmapping for
> > example.
> > > In the job definition have something like
> > >
> > > params = [
> > > Parameter( name='volume',
> > > value='service-{{instanceParameters.volume}}:/foo' )
> > > ]
> > > ...
> > > jobs = [
> > > Service(
> > > name = 'logstash',
> > > ...
> > > instanceParameters = { "volume" : ["foo", "bar", "zaa"]},
> > > instances = 3,
> > > container = Container(
> > > docker = Docker(
> > > image = 'image',
> > > parameters = params
> > > )
> > > )
> > > )
> > > ]
> > >
> > >
> > > Something like that, it would create 3 instances of the tasks, each one
> > > running in a container that uses the volumes foo, bar, and zaa.
> > >
> > > Does it make sense? I'd be glad to work on it but I want to validate
> the
> > > idea with you first and hear comments about the api/implementation.
> > >
> > > Thanks
> > >
> > >
> > > Mauricio
> > >
> >
>
