In the doc, it would be better to put the logs that are being printed while the optimization happens as an example.
On Wed, Jun 13, 2018 at 12:06 PM, Sarubi Thillainathan <sar...@wso2.com> wrote: > Hi all, > > PR for $Subject can be found in [1]. > Documentation for this can be found in [2] > > [1] https://github.com/wso2/product-apim/pull/3350 > [2] https://docs.google.com/document/d/1nB112HRslaqXLQA50lXdHblf42QgM > TIzTCAPVaOFjrA/edit?usp=sharing > > > On Fri, Jun 8, 2018 at 9:14 AM, Chamila De Alwis <chami...@wso2.com> > wrote: > >> Hi Nuwan, >> >> Thanks! I understand the context now and I completely agree that some >> kind of convenience is required to prune the base distribution to profile >> specific ones. It could very well be automated better than being documented >> instructions. >> >> My idea was that if some kind of automation could be done, it would be >> better suited on top of what we have already built out (ex: our knowledge >> on Puppet, what WUM client does with pack building). On the other hand, >> automation which considers details after a certain point could have to >> cater for a wider spectrum of requirements, which could make the tool >> complicated. >> >> +1 to make the tool to automate the prunning part of the process. >> >> Startup time getting affected by the `-Dprofile` property might not be >> that much of a concern at all (other than for Containerized scenarios, >> which could make use of pre-baked optimized packs), because instance >> downtimes are almost always pre-planned and will be done with back up >> nodes. So few additional seconds would not be that much of a difference. >> >> >> Regards, >> Chamila de Alwis >> Committer and PMC Member - Apache Stratos >> Associate Technical Lead | WSO2 >> +94 77 220 7163 >> Blog: https://medium.com/@chamilad >> >> >> >> >> On Thu, Jun 7, 2018 at 1:18 AM Nuwan Dias <nuw...@wso2.com> wrote: >> >>> While these concerns are valid, I think we need to step back and look at >>> this. If you look at are current documentation related to profiles, you >>> will see that we have lots of manual instructions to optimize the profiles. >>> What this tool/command does is to simply automate all those manual >>> instructions. So our "produce" at the end of the day will be exactly the >>> same as it is today. >>> >>> This tool does modify some configs, but I wouldn't consider that as a >>> config automation. Config automation is supposed to do configs to make the >>> product work according to a particular deployment architecture. The configs >>> we're modifying in this case do nothing to do with deployment architecture. >>> All it does is to remove unnecessary processes and stuff based on the >>> profile. I would rather consider the outcome/produce of this tool to be >>> used as the base image on which config automation tools should run on. >>> >>> I don't think WUM is a problem because WUM always gives you a fresh >>> distribution. It does not work/modify your current running instance. >>> Therefore a WUM user has to run the tool/process on the WUM updated >>> distribution as they do today. So the pipeline would be >>> >>> wum_update --> optimize --> puppet --> run >>> >>> As Sinthuja has pointed it would perhaps be good to create a new distro >>> (.zip) after this process so that one can use it as the base distro. We >>> have currently focused on making the optimization work seamlessly with the >>> -Dprofile=x startup command, which is how a majority of of customers using >>> profiles use the product. >>> >>> Thanks, >>> NuwanD. >>> >>> On Thu, Jun 7, 2018 at 12:19 PM, Chamila De Alwis <chami...@wso2.com> >>> wrote: >>> >>>> Thanks Sarubi for the detailed explanation! I had understood the >>>> context wrong and this cleared up some of the doubts I had. However, I >>>> still have a hard time grasping the whole user story. >>>> >>>> 1. I assume the changes you mention are the ones at [1]. If that is the >>>> case, there are indeed configuration changes involved. These are typically >>>> handled by a configuration automation tool. This is the overlap I'm >>>> concerned with. With the limited information I have, I guess any config >>>> automation will have to consider both scenarios where this script is >>>> present and absent. In other words, a set of scripts that work with a pack >>>> edited by this script would not work with the shipped pack. >>>> >>>> 2. What would be the effect of WUM updates? If the script considers the >>>> config changes in #1, and if any WUM updates affect those configurations >>>> (any changes would be additions), would the script also have to be updated? >>>> >>>> 3. Going along with what Sinthuja has raised above, if new packs are to >>>> be created, then most of the operations would be what a config automation >>>> tool does. >>>> >>>> 4. What would be overlap/effect on the product installer effort that is >>>> going on in the IE team [2] >>>> >>>> >>>> [1] - https://docs.wso2.com/display/AM2xx/Product+Profiles >>>> [2] - "Installers for WSO2 products" >>>> >>>> Regards, >>>> Chamila de Alwis >>>> Committer and PMC Member - Apache Stratos >>>> Associate Technical Lead | WSO2 >>>> +94 77 220 7163 >>>> Blog: https://medium.com/@chamilad >>>> >>>> >>>> >>>> >>>> On Wed, Jun 6, 2018 at 11:21 PM Sinthuja Rajendran <sinth...@wso2.com> >>>> wrote: >>>> >>>>> Hi, >>>>> On Thu, Jun 7, 2018 at 11:07 AM Sarubi Thillainathan <sar...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Chamila, >>>>>> >>>>>> According to the offline discussion with Nuwan, >>>>>> Here what we are doing is, automate configuration changes to >>>>>> optimize API-M profiles. Up to now, it has been manually done by >>>>>> users [1] when they wanted to run a selected profile. >>>>>> >>>>>> We provide two options to accomplish this, >>>>>> 1. The user can do the optimization separately before the server >>>>>> starts up by run the profileSetup.sh along with a particular profile >>>>>> flag. (ex: profileSetup.sh -Dprofile=api-store ) Once it has done, >>>>>> can use this APIM package as the root image for CM tools. >>>>>> >>>>> >>>>> This creates a new distribution (.zip) file for the intended profile >>>>> without altering the current distribution? Or it alters the current >>>>> distribution it self? i.e, Let's say I have downloaded and extracted APIM, >>>>> now I have to create api-store and api-publisher profiles. So, should I >>>>> extract the distributions two times and run this commands seperately >>>>> profileSetup.sh -Dprofile=api-store and profileSetup.sh >>>>> -Dprofile=api-publisher in two extracted directories? >>>>> >>>>> Thanks, >>>>> Sinthuja. >>>>> >>>>>> >>>>>> 2. The user can start the wso2server.sh with a particular profile >>>>>> along with --optimize flag. >>>>>> (i.e) wso2server.sh --optimize -Dprofile=api-store. >>>>>> if and only -optimize flag found we do the optimization for that >>>>>> particular profile then start the server automatically, otherwise not. >>>>>> >>>>>> Please find inline comments for further details: >>>>>> >>>>>> >>>>>> On Tue, Jun 5, 2018 at 4:11 AM, Chamila De Alwis <chami...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Sarubi, >>>>>>> >>>>>>> I have a couple of questions regarding the fit between the changes >>>>>>> that this script does and the config automation tools. >>>>>>> >>>>>>> Is this a tool focused on evaluation part of the product or the >>>>>>> deployment part? IMO both would require a re-evaluation of the approach. >>>>>>> >>>>>> >>>>>> This is focused on deployment part of product. >>>>>> >>>>>>> >>>>>>> IIUC, this script would alter the startup pack, with file additions >>>>>>> and removals done to config files, deployable artifacts, and other >>>>>>> library >>>>>>> files. This could be a problematic case for continuous config automation >>>>>>> story. >>>>>>> >>>>>>> Consider a scenario where Puppet (or any other tool) configures a >>>>>>> set of instances, using the APIM pack as the root. The tool might be >>>>>>> configured to continuously run (say, with a 30-minute interval). Is >>>>>>> there a >>>>>>> chance for the server to be reconfigured every time the tool is run, >>>>>>> because the tool detects the changes that this script does at the >>>>>>> startup, >>>>>>> as changes that should be rolled back? >>>>>>> >>>>>> >>>>>> According to the scenario that you have mentioned, Puppet (or any >>>>>> other tool) can configures a set of instances, using optimized APIM pack >>>>>> as >>>>>> the root. >>>>>> >>>>>> >>>>>>> In that case, it might be better to have this tool as a separate >>>>>>> script rather than one which starts up the server as well. >>>>>>> >>>>>> >>>>>> Yes, now we maintain a seperate script which will get executed based >>>>>> on the user specification for server start-up. >>>>>> >>>>>>> >>>>>>> Another point against such a tool bound to server startup would be >>>>>>> the mixing of concerns. AFAIU, what this script does falls under what >>>>>>> the >>>>>>> config automation tools do. IMO WSO2 server startup scripts should only >>>>>>> work on the bootstrapping of the server runtime, rather than the config >>>>>>> automation part. Also, we would essentially be repeating what the config >>>>>>> automation tools should do, but in Bash which sometimes is a lot harder. >>>>>>> Obtaining config values, managing them meaningfully, and applying them >>>>>>> to >>>>>>> files is essentially what a config automation tool does. We may be >>>>>>> re-inventing the wheel here. For an example, should this tool be >>>>>>> concerned >>>>>>> about the port offset at all since even dev deployments will be done >>>>>>> using >>>>>>> separate VMs/Containers? Or should it be worried about different values >>>>>>> for >>>>>>> the config options in different environments? >>>>>>> >>>>>>> Also, this script don't concerned about the port offset things since >>>>>> it designed only to configure the changes for corresponding profile >>>>>> specified. >>>>>> >>>>>> >>>>>>> Additionally, users who manage config automation usually adhere to >>>>>>> one set of scripts. Puppet users almost always go for Puppet, Chef users >>>>>>> for Chef etc. What we would be delivering might step on what they >>>>>>> already >>>>>>> have, or are comfortable working with, and in turn, the adaption rate >>>>>>> might >>>>>>> take a hit. >>>>>>> >>>>>>> User can go with first option, run the profile setup script >>>>>> seperatly before server start up and get optimized API-M pack as the base >>>>>> image >>>>>> >>>>>> >>>>>>> IMO you might benefit from reviewing your approach with the >>>>>>> Installation Experience effort, as the responsibilities of this tool >>>>>>> fall >>>>>>> close to, if not overlap with, what the Installation Experience scripts >>>>>>> deliver/would deliver with different providers. >>>>>>> >>>>>>> WDYT? >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> Chamila de Alwis >>>>>>> Committer and PMC Member - Apache Stratos >>>>>>> Associate Technical Lead | WSO2 >>>>>>> +94 77 220 7163 >>>>>>> Blog: https://medium.com/@chamilad >>>>>>> >>>>>>> >>>>>> Commnets or suggestions are highly appreciated. >>>>>> >>>>>> [1]https://docs.wso2.com/display/AM220/Product+Profiles >>>>>> >>>>>> >>>>>> -- >>>>>> *Sarubi Thillainathan * >>>>>> *Software Engineer - WSO2 Inc.* >>>>>> >>>>>> *Mobile : +94 (0) 76 68 49 101* >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> Architecture@wso2.org >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Sinthuja Rajendran* >>>>> Senior Technical Lead >>>>> WSO2, Inc.:http://wso2.com >>>>> >>>>> Blog: http://sinthu-rajan.blogspot.com/ >>>>> Mobile: +94774273955 >>>>> >>>>> >>>>> >>> >>> >>> -- >>> Nuwan Dias >>> >>> Software Architect - WSO2, Inc. http://wso2.com >>> email : nuw...@wso2.com >>> Phone : +94 777 775 729 >>> >> > > > -- > *Sarubi Thillainathan * > *Software Engineer - WSO2 Inc.* > > *Mobile : +94 (0) 76 68 49 101* > -- Nuwan Dias Software Architect - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
