The issue with packing all configs file according to a particular profile is that we'll end up packing many duplicate files, which might not be optimal. One way to get around this problem would be to use the ConfigResolver - deployment.properties file introduced in C5 to override only the required configs, this way we would have a deployment.properties file per profile. We'll need to make some changes in the ConfigResolver to pick the profile specific property file.
Also for the osgi we could use bundles.info file as you suggested, but I was thinking if its possible to package jars required to particular profile in sub directories. So inside plugins/ directory we'll have a commons/ dir for the jars common to all the profiles and sub directories to hold profile specific jars. IINM mistaken, we'll have to modify the carbon-p2-plugin to do this. WDYT? On Mon, Oct 10, 2016 at 2:42 PM, Kishanthan Thangarajah <[email protected] > wrote: > I like the idea of providing a descriptor about a profile (describes what > artifacts that should be included) and use a tool or a script to create the > profile specific runtime pack. But we also need to consider what Sameera > mentioned. I see the issue can come only with configuration repo because, > different profiles have different configurations (eg axis2.xml). The base > distribution should pack all the config files according to different > profile and from the descriptor, we can point to the correct config > file(s). Other repositories (artifacts and osgi) do not have such issue and > for osgi, we could use the bundles.info file as profile specific at > descriptor, which will then read by the tool here. > > WDYT? > > Thanks, > Kishanthan. > > On Fri, Oct 7, 2016 at 7:50 AM, Muhammed Shariq <[email protected]> wrote: > >> Hi. >> >> I had a chat with Azeez and Lakmal to discuss some ideas on improving the >> profile support by taking the Cloud / container architecture into >> consideration. >> >> We discussed an approach similar to Option-2 in the previous mail where >> we package all artifacts into one distribution and provide a tool to create >> the profile specific bear minimum pack. The tool should provide the >> following functionality; >> >> 1. List the available profiles / runtimes for a given distro >> 2. Once the user selects a profile, the tool should create the minimum >> pack >> - The pack should provide the meta info needed to create the pack, we >> could use a yaml config for example; >> type: deployment >> webapp: >> - admin.war >> - axc.war >> jaggeryapps: >> - storex >> - abcdx >> services: >> - foo.aar >> - Similarly we'll need some meta descriptors for the jars and configs >> as well, we could either use the bundles.info to parse the required jars >> or look for a better option. We might also have to define a way to specify >> configs in a profile specific manner so we can simplify the process of >> creating the bear minimum runtime. >> >> 3. Provide the option create docker-compose yaml that would take the >> created archives and deploy it in containers. Ideally we should be able to >> provide a fully functional docker based distributed deployment (with host >> entries, databases etc) OOTB. >> >> If we are to go with this approach, we need to rethink our packaging >> stricture to be able to create the profile specific pack. With this >> approach, I feel we are moving away from P2 based profiles, so maybe we can >> refer to the minimum packs as a "runtime" of product. >> >> Any thoughts, suggestions? >> >> >> On Wed, Oct 5, 2016 at 11:16 AM, Muhammed Shariq <[email protected]> wrote: >> >>> Hi, >>> >>> If we are to reduce the pack size to bear minimum and pack only the >>> essential artifacts, we can use one of the following approaches; >>> >>> 1. Build a bear minimum distribution (with only the required jars, >>> config and artifacts) at build time (maven build) >>> 2. Pack all the artifacts into one distribution (like we do now) and >>> provide a script / tool to create the bear minimum pack at runtime. >>> >>> If we go ahead with Option-1, then we will be creating multiple >>> distributions for a single product so a product like API-M will have many >>> different distributions. This I feel will make the simple case too >>> complicated, specially for a user trying to get started. >>> >>> With Option-2, we can still have the default profile as it is for the >>> simplest case, but provide users the ability to create profile specific >>> distributions for larger deployments. Users can then use these profile >>> specific distributions to create their images. >>> >>> In both cases, I feel we are moving away from using profiles as we used >>> to use them since we are creating a pack with only the required jars and >>> artifacts. >>> >>> Considering these factors, should we look to creating only the container >>> friendly bear minimum distribution (Option-1) or provide the ability to a >>> create profile specific pack from the default distribution? >>> >>> >>> On Wed, Oct 5, 2016 at 9:03 AM, Lakmal Warusawithana <[email protected]> >>> wrote: >>> >>>> >>>> >>>> On Tue, Oct 4, 2016 at 5:39 PM, Afkham Azeez <[email protected]> wrote: >>>> >>>>> At runtime, there should be profile specific packs shouldn't have >>>>> anything extra other than the bear minimum. This is to make it container >>>>> friendly as well. >>>>> >>>> >>>> Yes, reducing image size is critical to support container native >>>> architecture. >>>> >>>> >>>>> >>>>> On Tue, Oct 4, 2016 at 5:27 PM, Muhammed Shariq <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi folks, >>>>>> >>>>>> I had a chat with Sameera and Jayanga on how we can improve support >>>>>> for managing configurations for a particular profile. >>>>>> >>>>>> We were discussing the possibility of extending the ConfigResolver >>>>>> [1] concept to manage profile specific configurations. With >>>>>> ConfigResolver, >>>>>> we have the ability to override certain configurations using the >>>>>> deployment.properties file, this way we only need modify a single file to >>>>>> override configurations scattered in different files. >>>>>> >>>>>> We are looking into the possibility of using the ConfigResolver >>>>>> approach to handle configs related to a specific profile as well. The >>>>>> idea >>>>>> is to have a profile specific deployment.properties file where we can >>>>>> specify the configs related to that particular profile. This way we can >>>>>> avoid duplicating the same config file and parameteriz values using the >>>>>> properties file. >>>>>> >>>>>> If we build a pack specific to a particular profile, this might >>>>>> create complication for the simplest scenario where a user wants to >>>>>> simply >>>>>> extract the distribution and try it out. With profile specific >>>>>> distributions, users will have to deal with multiple distributions >>>>>> (configuring offsets etc) even to try out a product which is cumbersome. >>>>>> >>>>>> So I feel, having a per profile properties is a feasible mechanism to >>>>>> define profile specify configs. Shall we go ahead with this approach? >>>>>> >>>>>> Any concerns, thoughts? >>>>>> >>>>>> [1] - [Architecture] [C5] Adding deployment.properties file to >>>>>> override configurations in components >>>>>> [2] - [C5] Less configuration elements with meaningful defaults >>>>>> >>>>>> On Mon, Oct 3, 2016 at 5:11 PM, Sameera Jayasoma <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> We can categorize all the files which need to be handled by profiles >>>>>>> into three groups. >>>>>>> >>>>>>> >>>>>>> *1) OSGi repository * >>>>>>> >>>>>>> - *Location:* repository/components >>>>>>> - *Description:* Contains all the OSGi bundles, >>>>>>> dropins artifacts and other required files. This repository is >>>>>>> already >>>>>>> organized into multiple profiles(if any). >>>>>>> >>>>>>> >>>>>>> *2) Config repository* >>>>>>> >>>>>>> - *Location:* repository/conf >>>>>>> - *Description: Contains configuration files of the products. We >>>>>>> need to figure out a way handle profile specific configuration >>>>>>> files.* >>>>>>> >>>>>>> >>>>>>> *3) Artifact repository* >>>>>>> >>>>>>> - *Location*: repository/deployment >>>>>>> - *Description*: Contains deployment artifacts of the product. >>>>>>> We need to figure out a way to handle profile specific deployment >>>>>>> artifacts. >>>>>>> >>>>>>> >>>>>>> Shariq from the platform team will work on this. >>>>>>> >>>>>>> Thanks, >>>>>>> Sameera. >>>>>>> >>>>>>> On Fri, Sep 23, 2016 at 3:18 PM, Nuwan Dias <[email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 23, 2016 at 2:38 PM, Srinath Perera <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I think this happen with ESB NIO transport and Servelt transport >>>>>>>>> for webapps. ( Nuwan, is there other examples?). >>>>>>>>> >>>>>>>> >>>>>>>> In the regsitry.xml, we configure the indexers and handlers. These >>>>>>>> again are only required in certain profiles. There are also some >>>>>>>> configs in >>>>>>>> the api-manager.xml which only makes sense to enable in certain >>>>>>>> profiles. >>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 23, 2016 at 9:42 AM, Kishanthan Thangarajah < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Current issue is that all bundles and artifacts (conf files, >>>>>>>>>> webapps) are common to the server which are shared among all the >>>>>>>>>> profiles. >>>>>>>>>> We don't have a way to delete and modify them when starting up a >>>>>>>>>> profile. >>>>>>>>>> >>>>>>>>>> One other option is we could pack everything (profile specific >>>>>>>>>> artifacts) in the base distribution and provide a build script (ant) >>>>>>>>>> which >>>>>>>>>> create profile specific runtime a. >>>>>>>>>> >>>>>>>>>> We will check for the other alternatives along with this PoC and >>>>>>>>>> see. >>>>>>>>>> >>>>>>>>>> On Thu, Sep 22, 2016 at 12:27 PM, Afkham Azeez <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> We proposed an idea to build a pack based on a profile. That >>>>>>>>>>> will contain only the essential stuff. So rather than starting up a >>>>>>>>>>> runtime >>>>>>>>>>> and then loading a profile, you build a pack that contains the bare >>>>>>>>>>> minimum stuff required. Perhaps we can have a descriptor which >>>>>>>>>>> describes >>>>>>>>>>> what non-OSGi stuff are required for a profile and we can combine >>>>>>>>>>> that with >>>>>>>>>>> the OSGi bundles.info to figure out exactly what is needed. Can >>>>>>>>>>> someone in the kernel team do a quick PoC? >>>>>>>>>>> >>>>>>>>>>> On Thu, Sep 22, 2016 at 11:26 AM, Srinath Perera < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Smaeera, are these things we can fix? >>>>>>>>>>>> >>>>>>>>>>>> --Srinath >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Sep 22, 2016 at 11:23 AM, Nuwan Dias <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> This is to raise some concerns over the current server >>>>>>>>>>>>> profiles. Although we are able to control the bundles which are >>>>>>>>>>>>> loaded to >>>>>>>>>>>>> the runtime based on the -Dprofile parameter, we still lack the >>>>>>>>>>>>> ability of >>>>>>>>>>>>> removing files and modifying configuration files when the server >>>>>>>>>>>>> starts on >>>>>>>>>>>>> a profile. And this is forcing us to start unnecessary bundles at >>>>>>>>>>>>> startup. >>>>>>>>>>>>> Let me explain... >>>>>>>>>>>>> >>>>>>>>>>>>> API Manager has both webapps and a gateway in its >>>>>>>>>>>>> distribution. The synapse bundles are only required in the Gateway >>>>>>>>>>>>> profiles. However since the axis2.xml file of API Manager defines >>>>>>>>>>>>> the http >>>>>>>>>>>>> transport senders and receivers based on the Synapse passthrough >>>>>>>>>>>>> senders >>>>>>>>>>>>> and receivers, the axis2 engine will try to load the synapse >>>>>>>>>>>>> classes on >>>>>>>>>>>>> startup. Ideally if we were able to modify the axis2.xml on the >>>>>>>>>>>>> Publisher, >>>>>>>>>>>>> Store and Key Manager profiles and replace the passthrough >>>>>>>>>>>>> senders and >>>>>>>>>>>>> receivers with our standard http senders and receivers, we could >>>>>>>>>>>>> avoid >>>>>>>>>>>>> loading the synapse bundles on the Publisher, Store and Key >>>>>>>>>>>>> Manager. >>>>>>>>>>>>> >>>>>>>>>>>>> The same problem occurs for registry indexers and handlers. >>>>>>>>>>>>> Since the registry indexers and handlers are configured on the >>>>>>>>>>>>> registry.xml, even though these are only required in the >>>>>>>>>>>>> publisher and >>>>>>>>>>>>> store profiles, these bundles will be activated and running even >>>>>>>>>>>>> on the >>>>>>>>>>>>> Gateway, Key Manager and Traffic Manager. So unless we modify the >>>>>>>>>>>>> registry.xml on those nodes manually, we can't stop those bundles >>>>>>>>>>>>> from >>>>>>>>>>>>> running. >>>>>>>>>>>>> >>>>>>>>>>>>> Another problem we're facing is the inability to remove >>>>>>>>>>>>> webapps. Since all webapps in the >>>>>>>>>>>>> repository/deployment/server/webapps >>>>>>>>>>>>> and repository/deployment/server/jaggeryapps are deployed >>>>>>>>>>>>> into the runtime, unless we remove these webapps manually there >>>>>>>>>>>>> is no other >>>>>>>>>>>>> way to stop them from being deployed in unrelated profiles. >>>>>>>>>>>>> >>>>>>>>>>>>> I heard there is a discussion to bind a profile to a >>>>>>>>>>>>> container. Which would solve these problems. However it still >>>>>>>>>>>>> won't help >>>>>>>>>>>>> the "non-container" deployments. Are there ways to overcome the >>>>>>>>>>>>> above >>>>>>>>>>>>> mentioned limitations and enhance the efficiency of our profiles? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> NuwanD. >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Nuwan Dias >>>>>>>>>>>>> >>>>>>>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>>>>>>> email : [email protected] >>>>>>>>>>>>> Phone : +94 777 775 729 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> ============================ >>>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>>> http://people.apache.org/~hemapani/ >>>>>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>>> *email: **[email protected]* >>>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>>> >>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Kishanthan Thangarajah* >>>>>>>>>> Technical Lead, >>>>>>>>>> Platform Technologies Team, >>>>>>>>>> WSO2, Inc. >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> >>>>>>>>>> Mobile - +94773426635 >>>>>>>>>> Blog - *http://kishanthan.wordpress.com >>>>>>>>>> <http://kishanthan.wordpress.com>* >>>>>>>>>> Twitter - *http://twitter.com/kishanthan >>>>>>>>>> <http://twitter.com/kishanthan>* >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Kishanthan Thangarajah* >>>>>>>>>> Technical Lead, >>>>>>>>>> Platform Technologies Team, >>>>>>>>>> WSO2, Inc. >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> >>>>>>>>>> Mobile - +94773426635 >>>>>>>>>> Blog - *http://kishanthan.wordpress.com >>>>>>>>>> <http://kishanthan.wordpress.com>* >>>>>>>>>> Twitter - *http://twitter.com/kishanthan >>>>>>>>>> <http://twitter.com/kishanthan>* >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ============================ >>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>> http://people.apache.org/~hemapani/ >>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Nuwan Dias >>>>>>>> >>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>> email : [email protected] >>>>>>>> Phone : +94 777 775 729 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sameera Jayasoma, >>>>>>> Software Architect, >>>>>>> >>>>>>> WSO2, Inc. (http://wso2.com) >>>>>>> email: [email protected] >>>>>>> blog: https://medium.com/@sameera.jayasoma >>>>>>> twitter: https://twitter.com/sameerajayasoma >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> Shariq >>>>>> Associate Technical Lead >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Afkham Azeez* >>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>> * <http://www.apache.org/>* >>>>> *email: **[email protected]* <[email protected]> >>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>> <http://twitter.com/afkham_azeez> >>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>> >>>>> *Lean . Enterprise . Middleware* >>>>> >>>> >>>> >>>> >>>> -- >>>> Lakmal Warusawithana >>>> Director - Cloud Architecture; WSO2 Inc. >>>> Mobile : +94714289692 >>>> Blog : http://lakmalsview.blogspot.com/ >>>> >>>> >>> >>> >>> -- >>> Thanks, >>> Shariq >>> Associate Technical Lead >>> >> >> >> >> -- >> Thanks, >> Shariq >> Associate Technical Lead >> > > > > -- > *Kishanthan Thangarajah* > Technical Lead, > Platform Technologies Team, > WSO2, Inc. > lean.enterprise.middleware > > Mobile - +94773426635 > Blog - *http://kishanthan.wordpress.com <http://kishanthan.wordpress.com>* > Twitter - *http://twitter.com/kishanthan <http://twitter.com/kishanthan>* > -- Thanks, Shariq Associate Technical Lead
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
