Hi Sahan,

I don't think choosing a yaml file over deployment.properties would
complicate anything, it will only simplify the configuration management. If
we are anyway using yaml files for component configuration (carbon.yaml,
identity.yaml etc), then why not use deployment.yaml for user specific
configuration (override defaults). That would make the configuration
management process more consistent. Note that in most of the deployments,
users will be using a configuration management tool (Salt, Puppet, Chef) to
generate these config files. So it would make things easier to keep them
all in one format.

Other thing, I don't think we have got the right yaml configuration
hierarchy. First of all we should have *namespaces* [1] for each component
so that ConfigResolver would read parameters belonging to a particular
namespace. We can keep all params belonging to a particular namespace in a
separate yaml file. In deployment.yaml file we can define params for any
namespace which would override default ones defined in specific yaml files.
Since component specific yaml files may contain config hierarchies, we
should be able to do the same in deployment.yaml. deployment.properties
will have a limitation of a flat structure.

[1] http://yaml.org/spec/1.1/#id861700

Thanks.


On Fri, Oct 14, 2016 at 7:31 PM, Shan Mahanama <sh...@wso2.com> wrote:

> Hi all,
>
> It seems like there is a bit of misunderstanding about how the
> *deployment.properties* file can be used. It's functionality(after
> migrating to C5) is to provide a single place(file) to
> override configurations which are defined in various other configuration
> files in the products, making configuration changes easier. It is basically
> a file which will have commonly used configurations by users by default.
> But many other configurations can be added, which will be used to override
> the values in the corresponding configuration file in-memory at runtime
> after the ConfigResolver is started. It does not change the values in the
> original configuration files. Some example usages of
> *deployment.properties* files are as follows.
>
> 1) carbon.yaml file will have 0 as the port offset by default. Adding this
> configuration to the *deployment.properties* will set the offset value to
> 1 at runtime.
>
> [carbon.yaml]/ports/offset=1
>
> 2) After migrating to C5 and using ConfigResolver, we can easily change
> configuration values in the identity.xml or any other configuration file
> which provides this configuration override support through ConfigResolver.
>
> # Overriding xml element
> [identity.xml]/Server/JDBCPersistenceManager/DataSource/Name=SomeOtherName
> # Overriding xml attribute
> [identity.xml]/Server/OAuth/ClientAuthHandlers/
> ClientAuthHandler/Property/@Name=SomeOtherName
>
> As we can see from the examples, ConfigResolver can be used to override
> any configuration value if the component developer provides this
> support.How the component developers can provide this support is, rather
> directly reading the configuration file, they can use the ConfigResolver to
> read the configuration files. When ConfigResolver is starting, it reads and
> stores the configurations in the *deployment.properties* file. Then when
> the developer reads a configuration file using the ConfigResolver, it will
> identify and replace the corresponding configuration values. This way, we
> can override any configuration value in any configuration file if the
> support is provided.
>
> Advantages of having this deployment.properties files are
> 1) User don't have to browse through  many configuration files to change
> configurations. User can change any configuration from a single file.
> 2) User can identify what configurations are changed by only looking at
> the *deployment.properties* file. Otherwise user will have to go through
> many configuration files to identify changes.
> 3) Configurations directly related to component implementation are not
> exposed to basic users.
> 4) When users update products using WUM, currently they have to apply
> configuration changes manually. If their changed configurations are all in
> a single *deployment.properties* file, it would be much easier to apply
> their configurations to the newly updated pack because all they have to do
> is replace the* deployment.properties *file in the newly created pack
> with their own *deployment.properties *file after making the mentioned
> changes in the update instructions.
>
> As for using a yaml file over a properties file, I don't see any
> significant benefits using a yaml file. IMHO, it will only complicate the
> structure of the file and configuration defining process. Currently, the
> path defining is very similar to XPath so it is much easier to write and
> understand.
>
> Thanks,
> Shan.
>
>
> On Fri, Oct 14, 2016 at 4:56 PM, Nuwan Dias <nuw...@wso2.com> wrote:
>
>> I also would prefer to have a YAML based config file per profile. This
>> will keep the configuration simple and easy to read/understand as well. Of
>> course as Jayanga pointed we only should put something in there if it is
>> likely to be changed by a majority of users.
>>
>> Using a property file won't give a lot of readability. With config files,
>> sometimes it makes sense to write them in a hierarchical structure so that
>> we can group relevant configurations together. YAML will give you that
>> benefit.
>>
>> Thanks,
>> NuwanD.
>>
>> On Fri, Oct 14, 2016 at 2:34 PM, Jayanga Dissanayake <jaya...@wso2.com>
>> wrote:
>>
>>> @Imesh: Thanks for pointing this. I checked the link you shared.
>>> The files you pointed are for configuring a product for a particular
>>> deployment. That is why its only a subset of all files are changed. For a
>>> product to operate there are a lot more configurations that are not needed
>>> to change depending on the environment all the time.
>>> one example is, <EnableGatewayKeyCache>true</EnableGatewayKeyCache>, it
>>> is not present in [1]. The reason is by default it is 'true' and not needed
>>> to change unless there is an extreme case. That is why that configuration
>>> present in the api-manager.xml file but not in gateway-worker.yaml.
>>>
>>> This is just one example there are a lot more configurations like that,
>>> that way we would end up in the requirement of merging all ~30 file, isn't
>>> it?
>>>
>>> [1] https://github.com/wso2/puppet-modules/blob/master/hiera
>>> data/dev/wso2/wso2am/1.10.0/kubernetes/gateway-worker.yaml
>>>
>>> Thanks,
>>> Jayanga.
>>>
>>> *Jayanga Dissanayake*
>>> Associate Technical Lead
>>> WSO2 Inc. - http://wso2.com/
>>> lean . enterprise . middleware
>>> email: jaya...@wso2.com
>>> mobile: +94772207259
>>> <http://wso2.com/signature>
>>>
>>> On Fri, Oct 14, 2016 at 1:46 PM, Imesh Gunaratne <im...@wso2.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Oct 14, 2016 at 1:44 PM, Imesh Gunaratne <im...@wso2.com>
>>>> wrote:
>>>>
>>>>> On Fri, Oct 14, 2016 at 11:14 AM, Jayanga Dissanayake <
>>>>> jaya...@wso2.com> wrote:
>>>>>
>>>>>>
>>>>>> @Imesh/@Azeez: I also believe that merging all the
>>>>>> configurations into one file would complicate the configuration process.
>>>>>>
>>>>>
>>>>> ​Yes, it might be complicated if we were to add configurations of 30
>>>>> files into one. However the reality is bit different, please see below:
>>>>>
>>>>> [Correction]
>>>>
>>>> https://github.com/wso2/puppet-modules/tree/master/hieradata
>>>> /dev/wso2/wso2am/1.10.0/kubernetes
>>>> ​
>>>> Thanks
>>>>
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Jayanga.
>>>>>>
>>>>>> *Jayanga Dissanayake*
>>>>>> Associate Technical Lead
>>>>>> WSO2 Inc. - http://wso2.com/
>>>>>> lean . enterprise . middleware
>>>>>> email: jaya...@wso2.com
>>>>>> mobile: +94772207259
>>>>>> <http://wso2.com/signature>
>>>>>>
>>>>>> On Fri, Oct 14, 2016 at 10:50 AM, Afkham Azeez <az...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I think Imesh's suggestion merges all the config files and
>>>>>>> complicates stuff a lot. With the deployment.properties file we are
>>>>>>> including only the bits that most users will be concerned about and will
>>>>>>> provide a simple way to configure such stuff.
>>>>>>>
>>>>>>> On Fri, Oct 14, 2016 at 9:50 AM, Isuru Perera <isu...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 for using a YAML file instead of a properties file.
>>>>>>>>
>>>>>>>> On Fri, Oct 14, 2016 at 8:45 AM, Imesh Gunaratne <im...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I would like to propose to use a single YAML file for each
>>>>>>>>> distribution (product/profile) to make the configuration process 
>>>>>>>>> easier.
>>>>>>>>>
>>>>>>>>> I understand that we are trying to do something similar using a
>>>>>>>>> properties file (by overriding configurations in separate files), 
>>>>>>>>> however
>>>>>>>>> IMO a properties file might not suite well for this purpose. A YAML 
>>>>>>>>> file or
>>>>>>>>> any other type of a file which is more readable and designed for 
>>>>>>>>> managing
>>>>>>>>> hierarchical data structures would work well. More importantly having 
>>>>>>>>> a
>>>>>>>>> single configuration file would make the configuration process more 
>>>>>>>>> simpler
>>>>>>>>> and clean. WDYT?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thursday, October 13, 2016, Sidath Weerasinghe <sid...@wso2.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Jayanga,
>>>>>>>>>>
>>>>>>>>>> What are the most frequently changing configurations in C5 which
>>>>>>>>>> are going to store in the deployment.properties" file ?
>>>>>>>>>>
>>>>>>>>>> On Thu, Oct 13, 2016 at 5:07 PM, Jayanga Dissanayake <
>>>>>>>>>> jaya...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> With C5, we introduced "ConfigResolver" which enhances the user
>>>>>>>>>>> experience in changing configuration values. With the previous C4x
>>>>>>>>>>> approach, users had to know where the configuration files are and 
>>>>>>>>>>> to,
>>>>>>>>>>> change several configuration files to get the product working in 
>>>>>>>>>>> some
>>>>>>>>>>> scenarios.
>>>>>>>>>>>
>>>>>>>>>>> With "ConfigResolver" it allows us to have more frequently
>>>>>>>>>>> changing configurations in one location "deployment.properties" 
>>>>>>>>>>> file.
>>>>>>>>>>>
>>>>>>>>>>> A product has set of configurations that are needed to be
>>>>>>>>>>> changed in the deployments and there are some other configurations 
>>>>>>>>>>> that we
>>>>>>>>>>> don't change unless there is a complex situation. Hence, ideally,
>>>>>>>>>>> deployment.properties file should contain only the configurations 
>>>>>>>>>>> that are
>>>>>>>>>>> frequently used and can add more entries if a requirement arise.
>>>>>>>>>>>
>>>>>>>>>>> But with the requirements coming in with the "profile" support
>>>>>>>>>>> [1]. we have to rethink the way config resolver handle the 
>>>>>>>>>>> configuration
>>>>>>>>>>> files.
>>>>>>>>>>>
>>>>>>>>>>> eg:
>>>>>>>>>>> 1. We need to enable indexing in API store and publisher, not in
>>>>>>>>>>> other profiles.
>>>>>>>>>>> 2. Enabling certain handlers in particular profiles.
>>>>>>>>>>>
>>>>>>>>>>> At present, there is no configuration to enable/disable these
>>>>>>>>>>> features. We have to rethink the way we define configurations in 
>>>>>>>>>>> features
>>>>>>>>>>> in future. We have to have a way to enable/disable certain features 
>>>>>>>>>>> so that
>>>>>>>>>>> those could be disabled in certain profiles.
>>>>>>>>>>>
>>>>>>>>>>> Any idea/questions/clarifications are highly appreciated as it
>>>>>>>>>>> will help to model the new configurations story in C5.
>>>>>>>>>>>
>>>>>>>>>>> [1] "Multiple profile support for C5 based products."
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> *Jayanga Dissanayake*
>>>>>>>>>>> Associate Technical Lead
>>>>>>>>>>> WSO2 Inc. - http://wso2.com/
>>>>>>>>>>> lean . enterprise . middleware
>>>>>>>>>>> email: jaya...@wso2.com
>>>>>>>>>>> mobile: +94772207259
>>>>>>>>>>> <http://wso2.com/signature>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Architecture mailing list
>>>>>>>>>>> Architecture@wso2.org
>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Thank You,
>>>>>>>>>> Best Regards,
>>>>>>>>>>
>>>>>>>>>> Sidath Weerasinghe
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Intern*
>>>>>>>>>>
>>>>>>>>>> *WSO2, Inc. *
>>>>>>>>>>
>>>>>>>>>> *lean . enterprise . middleware *
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Mobile: +94719802550 <%2B94719802550>*
>>>>>>>>>>
>>>>>>>>>> *Email: *sid...@wso2.com
>>>>>>>>>>
>>>>>>>>>> Blog: https://medium.com/@sidath
>>>>>>>>>>
>>>>>>>>>> Linkedin: https://lk.linkedin.com/in/sidathweerasinghe
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Imesh Gunaratne*
>>>>>>>>> Software Architect
>>>>>>>>> WSO2 Inc: http://wso2.com
>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>>>>> W: https://medium.com/@imesh TW: @imesh
>>>>>>>>> lean. enterprise. middleware
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Architecture mailing list
>>>>>>>>> Architecture@wso2.org
>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Isuru Perera
>>>>>>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>
>>>>>>>> about.me/chrishantha
>>>>>>>> Contact: +IsuruPereraWSO2
>>>>>>>> <https://www.google.com/+IsuruPereraWSO2/about>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Architecture mailing list
>>>>>>>> Architecture@wso2.org
>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Afkham Azeez*
>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>>>>> Member; Apache Software Foundation; http://www.apache.org/
>>>>>>> * <http://www.apache.org/>*
>>>>>>> *email: **az...@wso2.com* <az...@wso2.com>
>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>>>>>> *twitter: **http://twitter.com/afkham_azeez*
>>>>>>> <http://twitter.com/afkham_azeez>
>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>*
>>>>>>>
>>>>>>> *Lean . Enterprise . Middleware*
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Architecture mailing list
>>>>>>> Architecture@wso2.org
>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Architecture mailing list
>>>>>> Architecture@wso2.org
>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Imesh Gunaratne*
>>>>> Software Architect
>>>>> WSO2 Inc: http://wso2.com
>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>> W: https://medium.com/@imesh TW: @imesh
>>>>> lean. enterprise. middleware
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Imesh Gunaratne*
>>>> Software Architect
>>>> WSO2 Inc: http://wso2.com
>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>> W: https://medium.com/@imesh TW: @imesh
>>>> lean. enterprise. middleware
>>>>
>>>>
>>>> _______________________________________________
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> Nuwan Dias
>>
>> Software Architect - WSO2, Inc. http://wso2.com
>> email : nuw...@wso2.com
>> Phone : +94 777 775 729
>>
>> _______________________________________________
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Shan Mahanama
>
> Software Engineer, WSO2 Inc. http://wso2.com
> <http://l.facebook.com/l.php?u=http%3A%2F%2Fwso2.com&h=gAQEswASa>
> Email: sh...@wso2.com
> Mobile: +94712000498
>
>
>
> _______________________________________________
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
Akila Ravihansa Perera
WSO2 Inc.;  http://wso2.com/

Blog: http://ravihansa3000.blogspot.com
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Reply via email to