Hi Chetan -
As mentioned in the linked DCOS issue, using a marathon "uri" for config files
(fetched before container is started) is an option - is there any reason that
won't work for us?
Files will end up in /mnt/mesos/sandbox in the container, not sure if that path
matters (we can copy the files too, or else change the classpath a bit)
On 8/23/18, 2:20 AM, "Chetan Mehrotra" <chetan.mehro...@gmail.com> wrote:
> How will this impact other deployment tools, like Docker Compose? I'm
aware
that your change keeps the old path working, but do we envision to drop
that at some point?
Docker compose can also make use of config mount to make config file
available. As to dropping support for current transformation support ...
this we can decide sometime in future once we put proposed approach in use
and see if it covers all usecase. For example one possible case can be
where container orchestrator has some way to inject credentials from a
secret store. That would work with simple env variables but tricky to get
it work with stuff embedded in a file
> Would it be valuable to have a writeup in the Wiki outlining how we'd
envision a future configuration method to work across all the deployment
methods that we have? I feel like there's lots of simplification to get
especially for cases like Mesos and/or Kubernetes, if we drop the env-var
rewriting.
Would work on a write up. k8s ConfigMap support is quite useful.
Unfortunately Mesos/Marathon does not have such a feature [2]. So we have
to make it work with cramming stuff in env variables only for now.
As of now env-var rewriting works without much overhead. So should be ok to
continue supporting that. Just that as part of official documentation steps
around any configuration we provide examples only for typesafe config way.
Chetan Mehrotra
[2]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdcosjira.atlassian.net%2Fbrowse%2FDCOS-516&data=02%7C01%7Ctnorris%40adobe.com%7C4271606ed8eb408156d008d608d99e3b%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636706128094622033&sdata=E%2BZa%2BCSP5gYVO05pVFIlaAgYwD9AFJ2cR7twRkGnWkw%3D&reserved=0
On Thu, Aug 23, 2018 at 2:12 PM Markus Thömmes <markusthoem...@apache.org>
wrote:
> Hi Chetan,
>
> Am Do., 23. Aug. 2018 um 10:28 Uhr schrieb Chetan Mehrotra <
> chetan.mehro...@gmail.com>:
>
> > > Is it possible to layer configurations with this? For example: If I
> > create
> > a `database.conf` and a `controller.conf`, is there a way to mount these
> in
> > a way that they are both read and merged by the specific component?
> >
> > This should be possible as of now also. If file mount option is used [1]
> > then you need to
> >
> > 1. Mount configMap directory `/controller/config`
> > 2. Have one `application.conf` in that directory which has includes for
> > `database.conf` and `controller.conf`
> >
>
> Ahhh thanks, I completely forgot about includes!
>
>
> >
> > Then with changes done for #3059 these files would be included in
> classpath
> > and typesafe config logic to read all "application.conf" would get them
> > included.
> >
> > Only issue is order is non deterministic so config override would not
> work.
> > That can be enabled by setting system property
> > -Dconfig.file=/controller/config/application.conf
> > (or env CONFIG_config_file). This would ensure that any later includes
> > would supersede previous includes
> >
>
> Sounds good.
>
> How will this impact other deployment tools, like Docker Compose? I'm
aware
> that your change keeps the old path working, but do we envision to drop
> that at some point?
>
> Would it be valuable to have a writeup in the Wiki outlining how we'd
> envision a future configuration method to work across all the deployment
> methods that we have? I feel like there's lots of simplification to get
> especially for cases like Mesos and/or Kubernetes, if we drop the env-var
> rewriting.
>
>
> >
> >
> > Chetan Mehrotra
> > [1]
> >
> >
>
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkubernetes.io%2Fdocs%2Ftasks%2Fconfigure-pod-container%2Fconfigure-pod-configmap%2F%23populate-a-volume-with-data-stored-in-a-configmap&data=02%7C01%7Ctnorris%40adobe.com%7C4271606ed8eb408156d008d608d99e3b%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636706128094622033&sdata=DK9oQ%2F%2Fi6PDi0XU%2BD7WaDFeaZwlXAMG%2BXdC3Xwjzpyc%3D&reserved=0
> >
> > On Thu, Aug 23, 2018 at 1:32 PM Markus Thömmes <
> markusthoem...@apache.org>
> > wrote:
> >
> > > Hi Chetan,
> > >
> > > good idea!
> > >
> > > A bit of background on why it is how it is: When I implemented the
> > approach
> > > we're having today, the basic thought was to be able to detect on a
> quite
> > > granular level what changes are needed to which components, even when
> > they
> > > share values. For example: In the Kubernetes case, most of the env
> > > variables are read out of a ConfigMap. Updating such a ConfigMap can
> > > trigger a restart of the specific pods to reload the config.
> > >
> > > This is possible with this approach as well, in fact for something
like
> > > Kubernetes it's even easier (I believe you can mount the ConfigMap
> > directly
> > > without needing to pass it via env). What I'd like to make sure
though:
> > >
> > > Is it possible to layer configurations with this? For example: If I
> > create
> > > a `database.conf` and a `controller.conf`, is there a way to mount
> these
> > in
> > > a way that they are both read and merged by the specific component?
> > >
> > > Cheers,
> > > Markus
> > >
> > > Am Do., 23. Aug. 2018 um 08:36 Uhr schrieb Chetan Mehrotra <
> > > chetan.mehro...@gmail.com>:
> > >
> > > > Hi Team,
> > > >
> > > > Currently we rely on env variables to configure OpenWhisk component
> > like
> > > > Controller and Invoker. For example setting
> > > > whisk.loadbalancer.invoker-busy-threshold would need env name like
> > > > "CONFIG_whisk_loadbalancer_invokerBusyThreshold".
> > > >
> > > > This mapping is at times tricky to get right and bit error prone
> > > (specially
> > > > when setting list props). It would be better if we can directly pass
> in
> > > the
> > > > config in typesafe config format [1]. This would simplify managing
> the
> > > > config in version control and also reusing the default config
> directly.
> > > >
> > > > For this purpose I have created a PR #3984. For now this PR only
> enable
> > > use
> > > > to this proposed approach. If it looks good we can modify the
default
> > > > ansible setup to use the new approach in a subsequent PR.
> > > >
> > > > Feedback welcome!
> > > >
> > > > Chetan Mehrotra
> > > > [1]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flightbend%2Fconfig%2Fblob%2Fmaster%2FHOCON.md&data=02%7C01%7Ctnorris%40adobe.com%7C4271606ed8eb408156d008d608d99e3b%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636706128094622033&sdata=geThuwG7pqNl2sExAr%2BU3w0QoJl7Ecfax5LEy3%2FXlYo%3D&reserved=0
> > > > [2]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fpull%2F3984&data=02%7C01%7Ctnorris%40adobe.com%7C4271606ed8eb408156d008d608d99e3b%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636706128094622033&sdata=dgMUHCZH9A9h89G%2BOeansWrT%2FwZuhulNObyNpEPC7ew%3D&reserved=0
> > > >
> > >
> >
>
