Parameterize each Job Instance.

[Mauricio Garavaglia]

Hi guys,

We are using the docker rbd volume plugin
<https://ceph.com/planet/getting-started-with-the-docker-rbd-volume-plugin> to
provide persistent storage to the aurora jobs that runs in the containers.
Something like:

p = [Parameter(name='volume', value='my-ceph-volume:/foo'), ...]
jobs = [ Service(..., container = Container(docker = Docker(..., parameters
= p)))]

But in the case of jobs with multiple instances it's required to start each
container using different volumes, in our case different ceph images. This
could be achieved by deploying, for example, 10 instances and then update
each one independently to use the appropiate volume. Of course this is
quite inconvenient, error prone, and adds a lot of logic and state outside
aurora.

We where thinking if it would make sense to have a way to parameterize the
task instances, in a similar way that is done with portmapping for example.
In the job definition have something like

params = [
  Parameter( name='volume',
value='service-{{instanceParameters.volume}}:/foo' )
]
...
jobs = [
  Service(
    name = 'logstash',
    ...
    instanceParameters = { "volume" : ["foo", "bar", "zaa"]},
    instances = 3,
    container = Container(
      docker = Docker(
        image = 'image',
        parameters = params
      )
    )
  )
]


Something like that, it would create 3 instances of the tasks, each one
running in a container that uses the volumes foo, bar, and zaa.

Does it make sense? I'd be glad to work on it but I want to validate the
idea with you first and hear comments about the api/implementation.

Thanks


Mauricio