Yes we should have separate deployment.properties files and not duplicate the config files.
On Tue, Oct 11, 2016 at 10:31 AM, Muhammed Shariq <[email protected]> wrote: > The issue with packing all configs file according to a particular profile > is that we'll end up packing many duplicate files, which might not be > optimal. One way to get around this problem would be to use the > ConfigResolver - deployment.properties file introduced in C5 to override > only the required configs, this way we would have a deployment.properties > file per profile. We'll need to make some changes in the ConfigResolver to > pick the profile specific property file. > > Also for the osgi we could use bundles.info file as you suggested, but I > was thinking if its possible to package jars required to particular profile > in sub directories. So inside plugins/ directory we'll have a commons/ dir > for the jars common to all the profiles and sub directories to hold profile > specific jars. IINM mistaken, we'll have to modify the carbon-p2-plugin to > do this. > > WDYT? > > > On Mon, Oct 10, 2016 at 2:42 PM, Kishanthan Thangarajah < > [email protected]> wrote: > >> I like the idea of providing a descriptor about a profile (describes what >> artifacts that should be included) and use a tool or a script to create the >> profile specific runtime pack. But we also need to consider what Sameera >> mentioned. I see the issue can come only with configuration repo because, >> different profiles have different configurations (eg axis2.xml). The base >> distribution should pack all the config files according to different >> profile and from the descriptor, we can point to the correct config >> file(s). Other repositories (artifacts and osgi) do not have such issue and >> for osgi, we could use the bundles.info file as profile specific at >> descriptor, which will then read by the tool here. >> >> WDYT? >> >> Thanks, >> Kishanthan. >> >> On Fri, Oct 7, 2016 at 7:50 AM, Muhammed Shariq <[email protected]> wrote: >> >>> Hi. >>> >>> I had a chat with Azeez and Lakmal to discuss some ideas on improving >>> the profile support by taking the Cloud / container architecture into >>> consideration. >>> >>> We discussed an approach similar to Option-2 in the previous mail where >>> we package all artifacts into one distribution and provide a tool to create >>> the profile specific bear minimum pack. The tool should provide the >>> following functionality; >>> >>> 1. List the available profiles / runtimes for a given distro >>> 2. Once the user selects a profile, the tool should create the minimum >>> pack >>> - The pack should provide the meta info needed to create the pack, >>> we could use a yaml config for example; >>> type: deployment >>> webapp: >>> - admin.war >>> - axc.war >>> jaggeryapps: >>> - storex >>> - abcdx >>> services: >>> - foo.aar >>> - Similarly we'll need some meta descriptors for the jars and >>> configs as well, we could either use the bundles.info to parse the >>> required jars or look for a better option. We might also have to define a >>> way to specify configs in a profile specific manner so we can simplify the >>> process of creating the bear minimum runtime. >>> >>> 3. Provide the option create docker-compose yaml that would take the >>> created archives and deploy it in containers. Ideally we should be able to >>> provide a fully functional docker based distributed deployment (with host >>> entries, databases etc) OOTB. >>> >>> If we are to go with this approach, we need to rethink our packaging >>> stricture to be able to create the profile specific pack. With this >>> approach, I feel we are moving away from P2 based profiles, so maybe we can >>> refer to the minimum packs as a "runtime" of product. >>> >>> Any thoughts, suggestions? >>> >>> >>> On Wed, Oct 5, 2016 at 11:16 AM, Muhammed Shariq <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> If we are to reduce the pack size to bear minimum and pack only the >>>> essential artifacts, we can use one of the following approaches; >>>> >>>> 1. Build a bear minimum distribution (with only the required jars, >>>> config and artifacts) at build time (maven build) >>>> 2. Pack all the artifacts into one distribution (like we do now) and >>>> provide a script / tool to create the bear minimum pack at runtime. >>>> >>>> If we go ahead with Option-1, then we will be creating multiple >>>> distributions for a single product so a product like API-M will have many >>>> different distributions. This I feel will make the simple case too >>>> complicated, specially for a user trying to get started. >>>> >>>> With Option-2, we can still have the default profile as it is for the >>>> simplest case, but provide users the ability to create profile specific >>>> distributions for larger deployments. Users can then use these profile >>>> specific distributions to create their images. >>>> >>>> In both cases, I feel we are moving away from using profiles as we used >>>> to use them since we are creating a pack with only the required jars and >>>> artifacts. >>>> >>>> Considering these factors, should we look to creating only the >>>> container friendly bear minimum distribution (Option-1) or provide the >>>> ability to a create profile specific pack from the default distribution? >>>> >>>> >>>> On Wed, Oct 5, 2016 at 9:03 AM, Lakmal Warusawithana <[email protected]> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Tue, Oct 4, 2016 at 5:39 PM, Afkham Azeez <[email protected]> wrote: >>>>> >>>>>> At runtime, there should be profile specific packs shouldn't have >>>>>> anything extra other than the bear minimum. This is to make it container >>>>>> friendly as well. >>>>>> >>>>> >>>>> Yes, reducing image size is critical to support container native >>>>> architecture. >>>>> >>>>> >>>>>> >>>>>> On Tue, Oct 4, 2016 at 5:27 PM, Muhammed Shariq <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi folks, >>>>>>> >>>>>>> I had a chat with Sameera and Jayanga on how we can improve support >>>>>>> for managing configurations for a particular profile. >>>>>>> >>>>>>> We were discussing the possibility of extending the ConfigResolver >>>>>>> [1] concept to manage profile specific configurations. With >>>>>>> ConfigResolver, >>>>>>> we have the ability to override certain configurations using the >>>>>>> deployment.properties file, this way we only need modify a single file >>>>>>> to >>>>>>> override configurations scattered in different files. >>>>>>> >>>>>>> We are looking into the possibility of using the ConfigResolver >>>>>>> approach to handle configs related to a specific profile as well. The >>>>>>> idea >>>>>>> is to have a profile specific deployment.properties file where we can >>>>>>> specify the configs related to that particular profile. This way we can >>>>>>> avoid duplicating the same config file and parameteriz values using the >>>>>>> properties file. >>>>>>> >>>>>>> If we build a pack specific to a particular profile, this might >>>>>>> create complication for the simplest scenario where a user wants to >>>>>>> simply >>>>>>> extract the distribution and try it out. With profile specific >>>>>>> distributions, users will have to deal with multiple distributions >>>>>>> (configuring offsets etc) even to try out a product which is cumbersome. >>>>>>> >>>>>>> So I feel, having a per profile properties is a feasible mechanism >>>>>>> to define profile specify configs. Shall we go ahead with this approach? >>>>>>> >>>>>>> Any concerns, thoughts? >>>>>>> >>>>>>> [1] - [Architecture] [C5] Adding deployment.properties file to >>>>>>> override configurations in components >>>>>>> [2] - [C5] Less configuration elements with meaningful defaults >>>>>>> >>>>>>> On Mon, Oct 3, 2016 at 5:11 PM, Sameera Jayasoma <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> We can categorize all the files which need to be handled by >>>>>>>> profiles into three groups. >>>>>>>> >>>>>>>> >>>>>>>> *1) OSGi repository * >>>>>>>> >>>>>>>> - *Location:* repository/components >>>>>>>> - *Description:* Contains all the OSGi bundles, >>>>>>>> dropins artifacts and other required files. This repository is >>>>>>>> already >>>>>>>> organized into multiple profiles(if any). >>>>>>>> >>>>>>>> >>>>>>>> *2) Config repository* >>>>>>>> >>>>>>>> - *Location:* repository/conf >>>>>>>> - *Description: Contains configuration files of the products. >>>>>>>> We need to figure out a way handle profile specific configuration >>>>>>>> files.* >>>>>>>> >>>>>>>> >>>>>>>> *3) Artifact repository* >>>>>>>> >>>>>>>> - *Location*: repository/deployment >>>>>>>> - *Description*: Contains deployment artifacts of the product. >>>>>>>> We need to figure out a way to handle profile specific deployment >>>>>>>> artifacts. >>>>>>>> >>>>>>>> >>>>>>>> Shariq from the platform team will work on this. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Sameera. >>>>>>>> >>>>>>>> On Fri, Sep 23, 2016 at 3:18 PM, Nuwan Dias <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 23, 2016 at 2:38 PM, Srinath Perera <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I think this happen with ESB NIO transport and Servelt transport >>>>>>>>>> for webapps. ( Nuwan, is there other examples?). >>>>>>>>>> >>>>>>>>> >>>>>>>>> In the regsitry.xml, we configure the indexers and handlers. These >>>>>>>>> again are only required in certain profiles. There are also some >>>>>>>>> configs in >>>>>>>>> the api-manager.xml which only makes sense to enable in certain >>>>>>>>> profiles. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Sep 23, 2016 at 9:42 AM, Kishanthan Thangarajah < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Current issue is that all bundles and artifacts (conf files, >>>>>>>>>>> webapps) are common to the server which are shared among all the >>>>>>>>>>> profiles. >>>>>>>>>>> We don't have a way to delete and modify them when starting up a >>>>>>>>>>> profile. >>>>>>>>>>> >>>>>>>>>>> One other option is we could pack everything (profile specific >>>>>>>>>>> artifacts) in the base distribution and provide a build script >>>>>>>>>>> (ant) which >>>>>>>>>>> create profile specific runtime a. >>>>>>>>>>> >>>>>>>>>>> We will check for the other alternatives along with this PoC and >>>>>>>>>>> see. >>>>>>>>>>> >>>>>>>>>>> On Thu, Sep 22, 2016 at 12:27 PM, Afkham Azeez <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> We proposed an idea to build a pack based on a profile. That >>>>>>>>>>>> will contain only the essential stuff. So rather than starting up >>>>>>>>>>>> a runtime >>>>>>>>>>>> and then loading a profile, you build a pack that contains the bare >>>>>>>>>>>> minimum stuff required. Perhaps we can have a descriptor which >>>>>>>>>>>> describes >>>>>>>>>>>> what non-OSGi stuff are required for a profile and we can combine >>>>>>>>>>>> that with >>>>>>>>>>>> the OSGi bundles.info to figure out exactly what is needed. >>>>>>>>>>>> Can someone in the kernel team do a quick PoC? >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Sep 22, 2016 at 11:26 AM, Srinath Perera < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Smaeera, are these things we can fix? >>>>>>>>>>>>> >>>>>>>>>>>>> --Srinath >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Sep 22, 2016 at 11:23 AM, Nuwan Dias <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is to raise some concerns over the current server >>>>>>>>>>>>>> profiles. Although we are able to control the bundles which are >>>>>>>>>>>>>> loaded to >>>>>>>>>>>>>> the runtime based on the -Dprofile parameter, we still lack the >>>>>>>>>>>>>> ability of >>>>>>>>>>>>>> removing files and modifying configuration files when the server >>>>>>>>>>>>>> starts on >>>>>>>>>>>>>> a profile. And this is forcing us to start unnecessary bundles >>>>>>>>>>>>>> at startup. >>>>>>>>>>>>>> Let me explain... >>>>>>>>>>>>>> >>>>>>>>>>>>>> API Manager has both webapps and a gateway in its >>>>>>>>>>>>>> distribution. The synapse bundles are only required in the >>>>>>>>>>>>>> Gateway >>>>>>>>>>>>>> profiles. However since the axis2.xml file of API Manager >>>>>>>>>>>>>> defines the http >>>>>>>>>>>>>> transport senders and receivers based on the Synapse passthrough >>>>>>>>>>>>>> senders >>>>>>>>>>>>>> and receivers, the axis2 engine will try to load the synapse >>>>>>>>>>>>>> classes on >>>>>>>>>>>>>> startup. Ideally if we were able to modify the axis2.xml on the >>>>>>>>>>>>>> Publisher, >>>>>>>>>>>>>> Store and Key Manager profiles and replace the passthrough >>>>>>>>>>>>>> senders and >>>>>>>>>>>>>> receivers with our standard http senders and receivers, we could >>>>>>>>>>>>>> avoid >>>>>>>>>>>>>> loading the synapse bundles on the Publisher, Store and Key >>>>>>>>>>>>>> Manager. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The same problem occurs for registry indexers and handlers. >>>>>>>>>>>>>> Since the registry indexers and handlers are configured on the >>>>>>>>>>>>>> registry.xml, even though these are only required in the >>>>>>>>>>>>>> publisher and >>>>>>>>>>>>>> store profiles, these bundles will be activated and running even >>>>>>>>>>>>>> on the >>>>>>>>>>>>>> Gateway, Key Manager and Traffic Manager. So unless we modify the >>>>>>>>>>>>>> registry.xml on those nodes manually, we can't stop those >>>>>>>>>>>>>> bundles from >>>>>>>>>>>>>> running. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Another problem we're facing is the inability to remove >>>>>>>>>>>>>> webapps. Since all webapps in the >>>>>>>>>>>>>> repository/deployment/server/webapps >>>>>>>>>>>>>> and repository/deployment/server/jaggeryapps are deployed >>>>>>>>>>>>>> into the runtime, unless we remove these webapps manually there >>>>>>>>>>>>>> is no other >>>>>>>>>>>>>> way to stop them from being deployed in unrelated profiles. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I heard there is a discussion to bind a profile to a >>>>>>>>>>>>>> container. Which would solve these problems. However it still >>>>>>>>>>>>>> won't help >>>>>>>>>>>>>> the "non-container" deployments. Are there ways to overcome the >>>>>>>>>>>>>> above >>>>>>>>>>>>>> mentioned limitations and enhance the efficiency of our profiles? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> NuwanD. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Nuwan Dias >>>>>>>>>>>>>> >>>>>>>>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>>>>>>>> email : [email protected] >>>>>>>>>>>>>> Phone : +94 777 775 729 >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> ============================ >>>>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>>>> http://people.apache.org/~hemapani/ >>>>>>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>>>> *email: **[email protected]* >>>>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>>>> >>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Kishanthan Thangarajah* >>>>>>>>>>> Technical Lead, >>>>>>>>>>> Platform Technologies Team, >>>>>>>>>>> WSO2, Inc. >>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>> >>>>>>>>>>> Mobile - +94773426635 >>>>>>>>>>> Blog - *http://kishanthan.wordpress.com >>>>>>>>>>> <http://kishanthan.wordpress.com>* >>>>>>>>>>> Twitter - *http://twitter.com/kishanthan >>>>>>>>>>> <http://twitter.com/kishanthan>* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Kishanthan Thangarajah* >>>>>>>>>>> Technical Lead, >>>>>>>>>>> Platform Technologies Team, >>>>>>>>>>> WSO2, Inc. >>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>> >>>>>>>>>>> Mobile - +94773426635 >>>>>>>>>>> Blog - *http://kishanthan.wordpress.com >>>>>>>>>>> <http://kishanthan.wordpress.com>* >>>>>>>>>>> Twitter - *http://twitter.com/kishanthan >>>>>>>>>>> <http://twitter.com/kishanthan>* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> ============================ >>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>> http://people.apache.org/~hemapani/ >>>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Nuwan Dias >>>>>>>>> >>>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>>> email : [email protected] >>>>>>>>> Phone : +94 777 775 729 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Sameera Jayasoma, >>>>>>>> Software Architect, >>>>>>>> >>>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>>> email: [email protected] >>>>>>>> blog: https://medium.com/@sameera.jayasoma >>>>>>>> twitter: https://twitter.com/sameerajayasoma >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks, >>>>>>> Shariq >>>>>>> Associate Technical Lead >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Afkham Azeez* >>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>> * <http://www.apache.org/>* >>>>>> *email: **[email protected]* <[email protected]> >>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>> <http://twitter.com/afkham_azeez> >>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>> >>>>>> *Lean . Enterprise . Middleware* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lakmal Warusawithana >>>>> Director - Cloud Architecture; WSO2 Inc. >>>>> Mobile : +94714289692 >>>>> Blog : http://lakmalsview.blogspot.com/ >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Shariq >>>> Associate Technical Lead >>>> >>> >>> >>> >>> -- >>> Thanks, >>> Shariq >>> Associate Technical Lead >>> >> >> >> >> -- >> *Kishanthan Thangarajah* >> Technical Lead, >> Platform Technologies Team, >> WSO2, Inc. >> lean.enterprise.middleware >> >> Mobile - +94773426635 >> Blog - *http://kishanthan.wordpress.com >> <http://kishanthan.wordpress.com>* >> Twitter - *http://twitter.com/kishanthan <http://twitter.com/kishanthan>* >> > > > > -- > Thanks, > Shariq > Associate Technical Lead > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919blog: **http://blog.afkham.org* <http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez* <http://twitter.com/afkham_azeez> *linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
