Hi all, PR for $Subject can be found in [1]. Documentation for this can be found in [2]
[1] https://github.com/wso2/product-apim/pull/3350 [2] https://docs.google.com/document/d/1nB112HRslaqXLQA50lXdHblf42QgMTIzTCAPVaOFjrA/edit?usp=sharing On Fri, Jun 8, 2018 at 9:14 AM, Chamila De Alwis <[email protected]> wrote: > Hi Nuwan, > > Thanks! I understand the context now and I completely agree that some kind > of convenience is required to prune the base distribution to profile > specific ones. It could very well be automated better than being documented > instructions. > > My idea was that if some kind of automation could be done, it would be > better suited on top of what we have already built out (ex: our knowledge > on Puppet, what WUM client does with pack building). On the other hand, > automation which considers details after a certain point could have to > cater for a wider spectrum of requirements, which could make the tool > complicated. > > +1 to make the tool to automate the prunning part of the process. > > Startup time getting affected by the `-Dprofile` property might not be > that much of a concern at all (other than for Containerized scenarios, > which could make use of pre-baked optimized packs), because instance > downtimes are almost always pre-planned and will be done with back up > nodes. So few additional seconds would not be that much of a difference. > > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Associate Technical Lead | WSO2 > +94 77 220 7163 > Blog: https://medium.com/@chamilad > > > > > On Thu, Jun 7, 2018 at 1:18 AM Nuwan Dias <[email protected]> wrote: > >> While these concerns are valid, I think we need to step back and look at >> this. If you look at are current documentation related to profiles, you >> will see that we have lots of manual instructions to optimize the profiles. >> What this tool/command does is to simply automate all those manual >> instructions. So our "produce" at the end of the day will be exactly the >> same as it is today. >> >> This tool does modify some configs, but I wouldn't consider that as a >> config automation. Config automation is supposed to do configs to make the >> product work according to a particular deployment architecture. The configs >> we're modifying in this case do nothing to do with deployment architecture. >> All it does is to remove unnecessary processes and stuff based on the >> profile. I would rather consider the outcome/produce of this tool to be >> used as the base image on which config automation tools should run on. >> >> I don't think WUM is a problem because WUM always gives you a fresh >> distribution. It does not work/modify your current running instance. >> Therefore a WUM user has to run the tool/process on the WUM updated >> distribution as they do today. So the pipeline would be >> >> wum_update --> optimize --> puppet --> run >> >> As Sinthuja has pointed it would perhaps be good to create a new distro >> (.zip) after this process so that one can use it as the base distro. We >> have currently focused on making the optimization work seamlessly with the >> -Dprofile=x startup command, which is how a majority of of customers using >> profiles use the product. >> >> Thanks, >> NuwanD. >> >> On Thu, Jun 7, 2018 at 12:19 PM, Chamila De Alwis <[email protected]> >> wrote: >> >>> Thanks Sarubi for the detailed explanation! I had understood the context >>> wrong and this cleared up some of the doubts I had. However, I still have a >>> hard time grasping the whole user story. >>> >>> 1. I assume the changes you mention are the ones at [1]. If that is the >>> case, there are indeed configuration changes involved. These are typically >>> handled by a configuration automation tool. This is the overlap I'm >>> concerned with. With the limited information I have, I guess any config >>> automation will have to consider both scenarios where this script is >>> present and absent. In other words, a set of scripts that work with a pack >>> edited by this script would not work with the shipped pack. >>> >>> 2. What would be the effect of WUM updates? If the script considers the >>> config changes in #1, and if any WUM updates affect those configurations >>> (any changes would be additions), would the script also have to be updated? >>> >>> 3. Going along with what Sinthuja has raised above, if new packs are to >>> be created, then most of the operations would be what a config automation >>> tool does. >>> >>> 4. What would be overlap/effect on the product installer effort that is >>> going on in the IE team [2] >>> >>> >>> [1] - https://docs.wso2.com/display/AM2xx/Product+Profiles >>> [2] - "Installers for WSO2 products" >>> >>> Regards, >>> Chamila de Alwis >>> Committer and PMC Member - Apache Stratos >>> Associate Technical Lead | WSO2 >>> +94 77 220 7163 >>> Blog: https://medium.com/@chamilad >>> >>> >>> >>> >>> On Wed, Jun 6, 2018 at 11:21 PM Sinthuja Rajendran <[email protected]> >>> wrote: >>> >>>> Hi, >>>> On Thu, Jun 7, 2018 at 11:07 AM Sarubi Thillainathan <[email protected]> >>>> wrote: >>>> >>>>> Hi Chamila, >>>>> >>>>> According to the offline discussion with Nuwan, >>>>> Here what we are doing is, automate configuration changes to optimize >>>>> API-M profiles. Up to now, it has been manually done by users [1] >>>>> when they wanted to run a selected profile. >>>>> >>>>> We provide two options to accomplish this, >>>>> 1. The user can do the optimization separately before the server >>>>> starts up by run the profileSetup.sh along with a particular profile >>>>> flag. (ex: profileSetup.sh -Dprofile=api-store ) Once it has done, >>>>> can use this APIM package as the root image for CM tools. >>>>> >>>> >>>> This creates a new distribution (.zip) file for the intended profile >>>> without altering the current distribution? Or it alters the current >>>> distribution it self? i.e, Let's say I have downloaded and extracted APIM, >>>> now I have to create api-store and api-publisher profiles. So, should I >>>> extract the distributions two times and run this commands seperately >>>> profileSetup.sh -Dprofile=api-store and profileSetup.sh >>>> -Dprofile=api-publisher in two extracted directories? >>>> >>>> Thanks, >>>> Sinthuja. >>>> >>>>> >>>>> 2. The user can start the wso2server.sh with a particular profile >>>>> along with --optimize flag. >>>>> (i.e) wso2server.sh --optimize -Dprofile=api-store. >>>>> if and only -optimize flag found we do the optimization for that >>>>> particular profile then start the server automatically, otherwise not. >>>>> >>>>> Please find inline comments for further details: >>>>> >>>>> >>>>> On Tue, Jun 5, 2018 at 4:11 AM, Chamila De Alwis <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Sarubi, >>>>>> >>>>>> I have a couple of questions regarding the fit between the changes >>>>>> that this script does and the config automation tools. >>>>>> >>>>>> Is this a tool focused on evaluation part of the product or the >>>>>> deployment part? IMO both would require a re-evaluation of the approach. >>>>>> >>>>> >>>>> This is focused on deployment part of product. >>>>> >>>>>> >>>>>> IIUC, this script would alter the startup pack, with file additions >>>>>> and removals done to config files, deployable artifacts, and other >>>>>> library >>>>>> files. This could be a problematic case for continuous config automation >>>>>> story. >>>>>> >>>>>> Consider a scenario where Puppet (or any other tool) configures a set >>>>>> of instances, using the APIM pack as the root. The tool might be >>>>>> configured >>>>>> to continuously run (say, with a 30-minute interval). Is there a chance >>>>>> for >>>>>> the server to be reconfigured every time the tool is run, because the >>>>>> tool >>>>>> detects the changes that this script does at the startup, as changes that >>>>>> should be rolled back? >>>>>> >>>>> >>>>> According to the scenario that you have mentioned, Puppet (or any >>>>> other tool) can configures a set of instances, using optimized APIM pack >>>>> as >>>>> the root. >>>>> >>>>> >>>>>> In that case, it might be better to have this tool as a separate >>>>>> script rather than one which starts up the server as well. >>>>>> >>>>> >>>>> Yes, now we maintain a seperate script which will get executed based >>>>> on the user specification for server start-up. >>>>> >>>>>> >>>>>> Another point against such a tool bound to server startup would be >>>>>> the mixing of concerns. AFAIU, what this script does falls under what the >>>>>> config automation tools do. IMO WSO2 server startup scripts should only >>>>>> work on the bootstrapping of the server runtime, rather than the config >>>>>> automation part. Also, we would essentially be repeating what the config >>>>>> automation tools should do, but in Bash which sometimes is a lot harder. >>>>>> Obtaining config values, managing them meaningfully, and applying them to >>>>>> files is essentially what a config automation tool does. We may be >>>>>> re-inventing the wheel here. For an example, should this tool be >>>>>> concerned >>>>>> about the port offset at all since even dev deployments will be done >>>>>> using >>>>>> separate VMs/Containers? Or should it be worried about different values >>>>>> for >>>>>> the config options in different environments? >>>>>> >>>>>> Also, this script don't concerned about the port offset things since >>>>> it designed only to configure the changes for corresponding profile >>>>> specified. >>>>> >>>>> >>>>>> Additionally, users who manage config automation usually adhere to >>>>>> one set of scripts. Puppet users almost always go for Puppet, Chef users >>>>>> for Chef etc. What we would be delivering might step on what they already >>>>>> have, or are comfortable working with, and in turn, the adaption rate >>>>>> might >>>>>> take a hit. >>>>>> >>>>>> User can go with first option, run the profile setup script seperatly >>>>> before server start up and get optimized API-M pack as the base image >>>>> >>>>> >>>>>> IMO you might benefit from reviewing your approach with the >>>>>> Installation Experience effort, as the responsibilities of this tool fall >>>>>> close to, if not overlap with, what the Installation Experience scripts >>>>>> deliver/would deliver with different providers. >>>>>> >>>>>> WDYT? >>>>>> >>>>>> >>>>>> Regards, >>>>>> Chamila de Alwis >>>>>> Committer and PMC Member - Apache Stratos >>>>>> Associate Technical Lead | WSO2 >>>>>> +94 77 220 7163 >>>>>> Blog: https://medium.com/@chamilad >>>>>> >>>>>> >>>>> Commnets or suggestions are highly appreciated. >>>>> >>>>> [1]https://docs.wso2.com/display/AM220/Product+Profiles >>>>> >>>>> >>>>> -- >>>>> *Sarubi Thillainathan * >>>>> *Software Engineer - WSO2 Inc.* >>>>> >>>>> *Mobile : +94 (0) 76 68 49 101* >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>> >>>> >>>> -- >>>> *Sinthuja Rajendran* >>>> Senior Technical Lead >>>> WSO2, Inc.:http://wso2.com >>>> >>>> Blog: http://sinthu-rajan.blogspot.com/ >>>> Mobile: +94774273955 >>>> >>>> >>>> >> >> >> -- >> Nuwan Dias >> >> Software Architect - WSO2, Inc. http://wso2.com >> email : [email protected] >> Phone : +94 777 775 729 >> > -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
