Hi Nuwan, I'll include that as well.
Thanks&Regards, Sarubi On Wed, Jun 13, 2018 at 2:19 PM, Nuwan Dias <[email protected]> wrote: > In the doc, it would be better to put the logs that are being printed > while the optimization happens as an example. > > On Wed, Jun 13, 2018 at 12:06 PM, Sarubi Thillainathan <[email protected]> > wrote: > >> Hi all, >> >> PR for $Subject can be found in [1]. >> Documentation for this can be found in [2] >> >> [1] https://github.com/wso2/product-apim/pull/3350 >> [2] https://docs.google.com/document/d/1nB112HRslaqXLQA50lXdHblf >> 42QgMTIzTCAPVaOFjrA/edit?usp=sharing >> >> >> On Fri, Jun 8, 2018 at 9:14 AM, Chamila De Alwis <[email protected]> >> wrote: >> >>> Hi Nuwan, >>> >>> Thanks! I understand the context now and I completely agree that some >>> kind of convenience is required to prune the base distribution to profile >>> specific ones. It could very well be automated better than being documented >>> instructions. >>> >>> My idea was that if some kind of automation could be done, it would be >>> better suited on top of what we have already built out (ex: our knowledge >>> on Puppet, what WUM client does with pack building). On the other hand, >>> automation which considers details after a certain point could have to >>> cater for a wider spectrum of requirements, which could make the tool >>> complicated. >>> >>> +1 to make the tool to automate the prunning part of the process. >>> >>> Startup time getting affected by the `-Dprofile` property might not be >>> that much of a concern at all (other than for Containerized scenarios, >>> which could make use of pre-baked optimized packs), because instance >>> downtimes are almost always pre-planned and will be done with back up >>> nodes. So few additional seconds would not be that much of a difference. >>> >>> >>> Regards, >>> Chamila de Alwis >>> Committer and PMC Member - Apache Stratos >>> Associate Technical Lead | WSO2 >>> +94 77 220 7163 >>> Blog: https://medium.com/@chamilad >>> >>> >>> >>> >>> On Thu, Jun 7, 2018 at 1:18 AM Nuwan Dias <[email protected]> wrote: >>> >>>> While these concerns are valid, I think we need to step back and look >>>> at this. If you look at are current documentation related to profiles, you >>>> will see that we have lots of manual instructions to optimize the profiles. >>>> What this tool/command does is to simply automate all those manual >>>> instructions. So our "produce" at the end of the day will be exactly the >>>> same as it is today. >>>> >>>> This tool does modify some configs, but I wouldn't consider that as a >>>> config automation. Config automation is supposed to do configs to make the >>>> product work according to a particular deployment architecture. The configs >>>> we're modifying in this case do nothing to do with deployment architecture. >>>> All it does is to remove unnecessary processes and stuff based on the >>>> profile. I would rather consider the outcome/produce of this tool to be >>>> used as the base image on which config automation tools should run on. >>>> >>>> I don't think WUM is a problem because WUM always gives you a fresh >>>> distribution. It does not work/modify your current running instance. >>>> Therefore a WUM user has to run the tool/process on the WUM updated >>>> distribution as they do today. So the pipeline would be >>>> >>>> wum_update --> optimize --> puppet --> run >>>> >>>> As Sinthuja has pointed it would perhaps be good to create a new distro >>>> (.zip) after this process so that one can use it as the base distro. We >>>> have currently focused on making the optimization work seamlessly with the >>>> -Dprofile=x startup command, which is how a majority of of customers using >>>> profiles use the product. >>>> >>>> Thanks, >>>> NuwanD. >>>> >>>> On Thu, Jun 7, 2018 at 12:19 PM, Chamila De Alwis <[email protected]> >>>> wrote: >>>> >>>>> Thanks Sarubi for the detailed explanation! I had understood the >>>>> context wrong and this cleared up some of the doubts I had. However, I >>>>> still have a hard time grasping the whole user story. >>>>> >>>>> 1. I assume the changes you mention are the ones at [1]. If that is >>>>> the case, there are indeed configuration changes involved. These are >>>>> typically handled by a configuration automation tool. This is the overlap >>>>> I'm concerned with. With the limited information I have, I guess any >>>>> config >>>>> automation will have to consider both scenarios where this script is >>>>> present and absent. In other words, a set of scripts that work with a pack >>>>> edited by this script would not work with the shipped pack. >>>>> >>>>> 2. What would be the effect of WUM updates? If the script considers >>>>> the config changes in #1, and if any WUM updates affect those >>>>> configurations (any changes would be additions), would the script also >>>>> have >>>>> to be updated? >>>>> >>>>> 3. Going along with what Sinthuja has raised above, if new packs are >>>>> to be created, then most of the operations would be what a config >>>>> automation tool does. >>>>> >>>>> 4. What would be overlap/effect on the product installer effort that >>>>> is going on in the IE team [2] >>>>> >>>>> >>>>> [1] - https://docs.wso2.com/display/AM2xx/Product+Profiles >>>>> [2] - "Installers for WSO2 products" >>>>> >>>>> Regards, >>>>> Chamila de Alwis >>>>> Committer and PMC Member - Apache Stratos >>>>> Associate Technical Lead | WSO2 >>>>> +94 77 220 7163 >>>>> Blog: https://medium.com/@chamilad >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 6, 2018 at 11:21 PM Sinthuja Rajendran <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> On Thu, Jun 7, 2018 at 11:07 AM Sarubi Thillainathan <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Chamila, >>>>>>> >>>>>>> According to the offline discussion with Nuwan, >>>>>>> Here what we are doing is, automate configuration changes to >>>>>>> optimize API-M profiles. Up to now, it has been manually done by >>>>>>> users [1] when they wanted to run a selected profile. >>>>>>> >>>>>>> We provide two options to accomplish this, >>>>>>> 1. The user can do the optimization separately before the server >>>>>>> starts up by run the profileSetup.sh along with a particular >>>>>>> profile flag. (ex: profileSetup.sh -Dprofile=api-store ) Once it >>>>>>> has done, can use this APIM package as the root image for CM tools. >>>>>>> >>>>>> >>>>>> This creates a new distribution (.zip) file for the intended profile >>>>>> without altering the current distribution? Or it alters the current >>>>>> distribution it self? i.e, Let's say I have downloaded and extracted >>>>>> APIM, >>>>>> now I have to create api-store and api-publisher profiles. So, should I >>>>>> extract the distributions two times and run this commands seperately >>>>>> profileSetup.sh -Dprofile=api-store and profileSetup.sh >>>>>> -Dprofile=api-publisher in two extracted directories? >>>>>> >>>>>> Thanks, >>>>>> Sinthuja. >>>>>> >>>>>>> >>>>>>> 2. The user can start the wso2server.sh with a particular profile >>>>>>> along with --optimize flag. >>>>>>> (i.e) wso2server.sh --optimize -Dprofile=api-store. >>>>>>> if and only -optimize flag found we do the optimization for that >>>>>>> particular profile then start the server automatically, otherwise not. >>>>>>> >>>>>>> Please find inline comments for further details: >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 5, 2018 at 4:11 AM, Chamila De Alwis <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Sarubi, >>>>>>>> >>>>>>>> I have a couple of questions regarding the fit between the changes >>>>>>>> that this script does and the config automation tools. >>>>>>>> >>>>>>>> Is this a tool focused on evaluation part of the product or the >>>>>>>> deployment part? IMO both would require a re-evaluation of the >>>>>>>> approach. >>>>>>>> >>>>>>> >>>>>>> This is focused on deployment part of product. >>>>>>> >>>>>>>> >>>>>>>> IIUC, this script would alter the startup pack, with file additions >>>>>>>> and removals done to config files, deployable artifacts, and other >>>>>>>> library >>>>>>>> files. This could be a problematic case for continuous config >>>>>>>> automation >>>>>>>> story. >>>>>>>> >>>>>>>> Consider a scenario where Puppet (or any other tool) configures a >>>>>>>> set of instances, using the APIM pack as the root. The tool might be >>>>>>>> configured to continuously run (say, with a 30-minute interval). Is >>>>>>>> there a >>>>>>>> chance for the server to be reconfigured every time the tool is run, >>>>>>>> because the tool detects the changes that this script does at the >>>>>>>> startup, >>>>>>>> as changes that should be rolled back? >>>>>>>> >>>>>>> >>>>>>> According to the scenario that you have mentioned, Puppet (or any >>>>>>> other tool) can configures a set of instances, using optimized APIM >>>>>>> pack as >>>>>>> the root. >>>>>>> >>>>>>> >>>>>>>> In that case, it might be better to have this tool as a separate >>>>>>>> script rather than one which starts up the server as well. >>>>>>>> >>>>>>> >>>>>>> Yes, now we maintain a seperate script which will get executed based >>>>>>> on the user specification for server start-up. >>>>>>> >>>>>>>> >>>>>>>> Another point against such a tool bound to server startup would be >>>>>>>> the mixing of concerns. AFAIU, what this script does falls under what >>>>>>>> the >>>>>>>> config automation tools do. IMO WSO2 server startup scripts should only >>>>>>>> work on the bootstrapping of the server runtime, rather than the config >>>>>>>> automation part. Also, we would essentially be repeating what the >>>>>>>> config >>>>>>>> automation tools should do, but in Bash which sometimes is a lot >>>>>>>> harder. >>>>>>>> Obtaining config values, managing them meaningfully, and applying them >>>>>>>> to >>>>>>>> files is essentially what a config automation tool does. We may be >>>>>>>> re-inventing the wheel here. For an example, should this tool be >>>>>>>> concerned >>>>>>>> about the port offset at all since even dev deployments will be done >>>>>>>> using >>>>>>>> separate VMs/Containers? Or should it be worried about different >>>>>>>> values for >>>>>>>> the config options in different environments? >>>>>>>> >>>>>>>> Also, this script don't concerned about the port offset things >>>>>>> since it designed only to configure the changes for corresponding >>>>>>> profile >>>>>>> specified. >>>>>>> >>>>>>> >>>>>>>> Additionally, users who manage config automation usually adhere to >>>>>>>> one set of scripts. Puppet users almost always go for Puppet, Chef >>>>>>>> users >>>>>>>> for Chef etc. What we would be delivering might step on what they >>>>>>>> already >>>>>>>> have, or are comfortable working with, and in turn, the adaption rate >>>>>>>> might >>>>>>>> take a hit. >>>>>>>> >>>>>>>> User can go with first option, run the profile setup script >>>>>>> seperatly before server start up and get optimized API-M pack as the >>>>>>> base >>>>>>> image >>>>>>> >>>>>>> >>>>>>>> IMO you might benefit from reviewing your approach with the >>>>>>>> Installation Experience effort, as the responsibilities of this tool >>>>>>>> fall >>>>>>>> close to, if not overlap with, what the Installation Experience scripts >>>>>>>> deliver/would deliver with different providers. >>>>>>>> >>>>>>>> WDYT? >>>>>>>> >>>>>>>> >>>>>>>> Regards, >>>>>>>> Chamila de Alwis >>>>>>>> Committer and PMC Member - Apache Stratos >>>>>>>> Associate Technical Lead | WSO2 >>>>>>>> +94 77 220 7163 >>>>>>>> Blog: https://medium.com/@chamilad >>>>>>>> >>>>>>>> >>>>>>> Commnets or suggestions are highly appreciated. >>>>>>> >>>>>>> [1]https://docs.wso2.com/display/AM220/Product+Profiles >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Sarubi Thillainathan * >>>>>>> *Software Engineer - WSO2 Inc.* >>>>>>> >>>>>>> *Mobile : +94 (0) 76 68 49 101* >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Sinthuja Rajendran* >>>>>> Senior Technical Lead >>>>>> WSO2, Inc.:http://wso2.com >>>>>> >>>>>> Blog: http://sinthu-rajan.blogspot.com/ >>>>>> Mobile: +94774273955 >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> -- >>>> Nuwan Dias >>>> >>>> Software Architect - WSO2, Inc. http://wso2.com >>>> email : [email protected] >>>> Phone : +94 777 775 729 >>>> >>> >> >> >> -- >> *Sarubi Thillainathan * >> *Software Engineer - WSO2 Inc.* >> >> *Mobile : +94 (0) 76 68 49 101* >> > > > > -- > Nuwan Dias > > Software Architect - WSO2, Inc. http://wso2.com > email : [email protected] > Phone : +94 777 775 729 > -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
