On Mon, Apr 11, 2016 at 2:06 PM, Chamila De Alwis <chami...@wso2.com> wrote:
> Hi Pubudu,
>
> That's a good idea, however, we can't exactly map `environment` with the
> `platform` concept. How about using a new level on top of the profile
> specific level mapping to `platform`?
>
> :hierarchy:
> - "node/%{::clientcert}"
> * -
> "wso2/%{::product_name}/%{::product_version}/%{::product_profile}-%{::platform}"*
> - "wso2/%{::product_name}/%{::product_version}/%{::product_profile}"
> - "wso2/%{::product_name}/%{::product_version}/default"
> - "osfamily/%{::osfamily}"
> - "vm_type/%{::vm_type}"
> - wso2/common
> - common
>
+1 for introducing a new level in hierarchy to lookup for platform specific
data on top of profile specific data per product.
>
> This will allow us to extract the platform specific data only to one layer
> up. An example would be,
>
> puppet-modules/hieradata/dev/wso2/wso2am/1.10.0/default-kubernetes.yaml
>
It would be better to have spearate folders for platform specific yaml
files inside each product versions like below with changing the above hiera
level to: *-
"wso2/%{::product_name}/%{::product_version}/%{::product_profile}/%{::platform}"*
hieradata/dev/wso2/wso2as/
└── 5.3.0
├── default.yaml
├── <*platform*>
│ ├── default.yaml
│ ├── manager.yaml
│ └── worker.yaml
├── kubernetes
│ ├── default.yaml
│ ├── manager.yaml
│ └── worker.yaml
├── manager.yaml
└── worker.yaml
With the above structure, configs common to all platform will be there in
the products' root level hierafiles. Only the platform specific configs
related to a particular profile will be there in <platform>/<profile>.yaml
(ex: proxy_ports).
> WDYT?
>
>
> Regards,
> Chamila de Alwis
> Committer and PMC Member - Apache Stratos
> Software Engineer | WSO2 | +94772207163
> Blog: code.chamiladealwis.com
>
>
>
> On Mon, Apr 11, 2016 at 1:36 PM, Pubudu Gunatilaka <pubu...@wso2.com>
> wrote:
>
>> Hi Isuru,
>>
>> Without duplicating files and adding new files, I would rather prefer to
>> use the following hierarchy. For each and every different environment we
>> can have product profiles and include the environment specific values. For
>> Kubernetes, we can include the clustering section only in manager.yaml file
>> which resides under environment Kubernetes. And the rest of the other
>> configurations can be included in the generic file folder, i.e "
>> wso2/%{::product_name}/%{::product_version}/%{::product_profile}".
>>
>>
>> :hierarchy:
>> - "node/%{::clientcert}" - - "
>> wso2/%{::product_name}/%{::product_version}/%{::environment}
>> /%{::product_profile}"
>> - "wso2/%{::product_name}/%{::product_version}/%{::product_profile}"
>> - "wso2/%{::product_name}/%{::product_version}/default"
>> - "osfamily/%{::osfamily}"
>> - "vm_type/%{::vm_type}"
>> - wso2/common
>> - common
>>
>>
>> Thank you!
>>
>> On Mon, Apr 11, 2016 at 1:07 PM, Isuru Haththotuwa <isu...@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Apr 11, 2016 at 12:47 PM, Gayan Gunarathne <gay...@wso2.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Apr 11, 2016 at 12:39 PM, Lakmal Warusawithana <lak...@wso2.com
>>>> > wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 11, 2016 at 12:22 PM, Imesh Gunaratne <im...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Chamila,
>>>>>>
>>>>>> On Mon, Apr 11, 2016 at 7:19 AM, Chamila De Alwis <chami...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Isuru, Imesh,
>>>>>>>
>>>>>>> IMO we shouldn't have any platform specific restructuring in
>>>>>>> wso2/puppet-modules. This should be done at the end user's setup.
>>>>>>> Another
>>>>>>> point is that we have now decoupled wso2/puppet-modules from
>>>>>>> wso2/dockerfiles and users are not required to incorporate
>>>>>>> puppet-modules
>>>>>>> in their container setup.
>>>>>>>
>>>>>>
>>>>>> A very good concern! The way I see this is little different. Let's
>>>>>> evaluate the options we have:
>>>>>>
>>>>>> 1. Ship generic product/profile Hiera YAML files and let the
>>>>>> users configure them according to their platform (VM, AWS, Azure,
>>>>>> OpenStack, K8S, Mesos, OpenShift, CF, etc)
>>>>>> 2. Ship product/profile/platform Hiera YAML files and let users
>>>>>> use them OOB with very few changes.
>>>>>>
>>>>>> +1 for 2nd option. Yes, it has some duplication on configurations,
>>>>> but customer POV, it is very easy to look at single place to do the
>>>>> minimum
>>>>> changes. (rather looking for many files in deferent folders to do the
>>>>> changes)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Which one would be the best option? IMO 2nd option would provide a
>>>>>> much better user experience compared to 1 as it provides platform
>>>>>> specific
>>>>>> values such as clustering configuration & port mappings OOB. User will
>>>>>> only
>>>>>> need to provide values such as database hosts, passwords, identity
>>>>>> management, etc which are user specific.
>>>>>>
>>>>>> The whole idea of this effort is to provide a better user experience.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>>
>>>>>>> IMO Docker images will not be able to run OOB on Kubernetes using
>>>>>>> wso2/puppet-modules and wso2/kubernetes-artifacts. There will anyway be
>>>>>>> changes related to the Kubernetes Membership Scheme in
>>>>>>> wso2/puppet-modules
>>>>>>> and in wso2/kubernetes-artifacts where environment dependent changes
>>>>>>> such
>>>>>>> as image names, SecureVault passwords, etc. need to be adjusted.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Chamila de Alwis
>>>>>>> Committer and PMC Member - Apache Stratos
>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 11, 2016 at 1:36 AM, Imesh Gunaratne <im...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Gayan,
>>>>>>>>
>>>>>>>> On Sun, Apr 10, 2016 at 5:02 PM, Gayan Gunarathne <gay...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> IMO this will create maintainability issue. We need to maintain
>>>>>>>>> all the separate hieradata structure for each scenarios.For the one
>>>>>>>>> particular alternation we need to change whole set of files.
>>>>>>>>>
>>>>>>>>
>>>>>>>> In this scenario user experience is much more important than the
>>>>>>>> maintainability of few yaml files. If we do not do this, users will
>>>>>>>> not be
>>>>>>>> able to use puppet modules OOB until they manually update configuration
>>>>>>>> values in above files. The whole idea of this effort is to let users do
>>>>>>>> following:
>>>>>>>>
>>>>>>>> - Setup a K8S cluster
>>>>>>>> - Download puppet modules zip file(s).
>>>>>>>> - Download docker files
>>>>>>>> - Build docker images using puppet for different product
>>>>>>>> profiles
>>>>>>>> - Deploy WSO2 product on K8S using K8S artifacts
>>>>>>>>
>>>>>>>> The above process will allow users to deploy any WSO2 product (with
>>>>>>>> mutlitple deployment patterns) on K8S with zero configurations. This
>>>>>>>> will
>>>>>>>> be true for any VM based platform or any other container cluster
>>>>>>>> management
>>>>>>>> system.
>>>>>>>>
>>>>>>>
>>>> Mainly the target users group of the puppet/hiera files will be system
>>>> administrators/Dev Ops. So those guys will be consider the fact the
>>>> maintainability of puppet/hiera files. So if this is a maintainability
>>>> issue, it will become bad experience for the end user in end of the day.
>>>>
>>>>
>>>>>
>>>>>>>>> Why can't we do this by using defined types in Hiera and lookup
>>>>>>>>> parameters for a given instance? Based on the identify keys we
>>>>>>>>> set for each vm, docker, K8S etc we can select the appropriate data
>>>>>>>>> set from Hiera file.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Will you be able to provide a sample?
>>>>>>>>
>>>>>>>
>>>>
>>>> I think we can make this with with Defined Types[1][2] without creating
>>>> duplicate set of YAML files for each platform. We can do the same as the
>>>> example given in the document.
>>>>
>>> I do not think we need to duplicate the yaml files here. Sorry that the
>>> sample in the first reply sent by me implied so. What we can do is refactor
>>> out the hiera data so that data which is changing according to the platform
>>> (ex.: clustering section) can be moved to a different yaml file(s). For an
>>> example, clustering for kubernetes scenario can be included in
>>> *puppet-modules/hieradata/dev/wso2/kubernetes/clustering.yaml*. Here,
>>> 'kubernetes' is the value of the facter which is used to identify the
>>> environment. Similarly, other such data can be refactored out.
>>>
>>
If we follow this approach, then we can't specify product and profile
specific data (ex: kubernetes service name varies per product and
proxy_ports varies per poduct/profile) in the common platform hiera file:
*puppet-modules/hieradata/dev/wso2/kubernetes/clustering.yaml.*
>>>> [1]
>>>> https://docs.puppet.com/puppet/latest/reference/lang_defined_types.html
>>>> [2]http://puppetlunch.com/puppet/hiera.html
>>>>
>>>>
>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Gayan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Apr 9, 2016 at 8:28 AM, Imesh Gunaratne <im...@wso2.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 8, 2016 at 7:48 PM, Isuru Haththotuwa <
>>>>>>>>>> isu...@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> hieradata
>>>>>>>>>>> |--- dev
>>>>>>>>>>> |--- wso2
>>>>>>>>>>> |---- <product_name>
>>>>>>>>>>> |--- <product_version>
>>>>>>>>>>> |-- *vm*
>>>>>>>>>>> |--
>>>>>>>>>>> default.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> manager.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> worker.yaml
>>>>>>>>>>> |--*
>>>>>>>>>>> docker*
>>>>>>>>>>> |--
>>>>>>>>>>> default.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> manager.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> worker.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> *kubernetes*
>>>>>>>>>>> |--
>>>>>>>>>>> default.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> manager.yaml
>>>>>>>>>>> |--
>>>>>>>>>>> worker.yaml
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> +1 for the suggestion Isuru, will proceed with this. We can add
>>>>>>>>>> other platforms such as OpenShift, Mesos, Cloud Foundry on the same
>>>>>>>>>> level.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Imesh Gunaratne*
>>>>>>>>>>>> Senior Technical Lead
>>>>>>>>>>>> WSO2 Inc: http://wso2.com
>>>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>>>>>>>> W: http://imesh.io
>>>>>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>
>>>>>>>>>>> Isuru H.
>>>>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Imesh Gunaratne*
>>>>>>>>>> Senior Technical Lead
>>>>>>>>>> WSO2 Inc: http://wso2.com
>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>>>>>> W: http://imesh.io
>>>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Dev mailing list
>>>>>>>>>> Dev@wso2.org
>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Gayan Gunarathne
>>>>>>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>> email : gay...@wso2.com | mobile : +94 775030545
>>>>>>>>> <%2B94%20766819985>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Dev mailing list
>>>>>>>>> Dev@wso2.org
>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Imesh Gunaratne*
>>>>>>>> Senior Technical Lead
>>>>>>>> WSO2 Inc: http://wso2.com
>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>>>> W: http://imesh.io
>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Dev mailing list
>>>>>>>> Dev@wso2.org
>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Imesh Gunaratne*
>>>>>> Senior Technical Lead
>>>>>> WSO2 Inc: http://wso2.com
>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>> W: http://imesh.io
>>>>>> Lean . Enterprise . Middleware
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dev mailing list
>>>>>> Dev@wso2.org
>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lakmal Warusawithana
>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>> Mobile : +94714289692
>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> Dev@wso2.org
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Gayan Gunarathne
>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>> Committer & PMC Member, Apache Stratos
>>>> email : gay...@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Dev mailing list
>>>> Dev@wso2.org
>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list
>>> Dev@wso2.org
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>> *Pubudu Gunatilaka*
>> Committer and PMC Member - Apache Stratos
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> mobile : +94774078049 <%2B94772207163>
>>
>>
>> _______________________________________________
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
> _______________________________________________
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
Thanuja Uruththirakodeeswaran
Software Engineer
WSO2 Inc.;http://wso2.com
lean.enterprise.middleware
mobile: +94 774363167
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
