Hi Nuwan, Thanks! I understand the context now and I completely agree that some kind of convenience is required to prune the base distribution to profile specific ones. It could very well be automated better than being documented instructions.
My idea was that if some kind of automation could be done, it would be better suited on top of what we have already built out (ex: our knowledge on Puppet, what WUM client does with pack building). On the other hand, automation which considers details after a certain point could have to cater for a wider spectrum of requirements, which could make the tool complicated. +1 to make the tool to automate the prunning part of the process. Startup time getting affected by the `-Dprofile` property might not be that much of a concern at all (other than for Containerized scenarios, which could make use of pre-baked optimized packs), because instance downtimes are almost always pre-planned and will be done with back up nodes. So few additional seconds would not be that much of a difference. Regards, Chamila de Alwis Committer and PMC Member - Apache Stratos Associate Technical Lead | WSO2 +94 77 220 7163 Blog: https://medium.com/@chamilad On Thu, Jun 7, 2018 at 1:18 AM Nuwan Dias <[email protected]> wrote: > While these concerns are valid, I think we need to step back and look at > this. If you look at are current documentation related to profiles, you > will see that we have lots of manual instructions to optimize the profiles. > What this tool/command does is to simply automate all those manual > instructions. So our "produce" at the end of the day will be exactly the > same as it is today. > > This tool does modify some configs, but I wouldn't consider that as a > config automation. Config automation is supposed to do configs to make the > product work according to a particular deployment architecture. The configs > we're modifying in this case do nothing to do with deployment architecture. > All it does is to remove unnecessary processes and stuff based on the > profile. I would rather consider the outcome/produce of this tool to be > used as the base image on which config automation tools should run on. > > I don't think WUM is a problem because WUM always gives you a fresh > distribution. It does not work/modify your current running instance. > Therefore a WUM user has to run the tool/process on the WUM updated > distribution as they do today. So the pipeline would be > > wum_update --> optimize --> puppet --> run > > As Sinthuja has pointed it would perhaps be good to create a new distro > (.zip) after this process so that one can use it as the base distro. We > have currently focused on making the optimization work seamlessly with the > -Dprofile=x startup command, which is how a majority of of customers using > profiles use the product. > > Thanks, > NuwanD. > > On Thu, Jun 7, 2018 at 12:19 PM, Chamila De Alwis <[email protected]> > wrote: > >> Thanks Sarubi for the detailed explanation! I had understood the context >> wrong and this cleared up some of the doubts I had. However, I still have a >> hard time grasping the whole user story. >> >> 1. I assume the changes you mention are the ones at [1]. If that is the >> case, there are indeed configuration changes involved. These are typically >> handled by a configuration automation tool. This is the overlap I'm >> concerned with. With the limited information I have, I guess any config >> automation will have to consider both scenarios where this script is >> present and absent. In other words, a set of scripts that work with a pack >> edited by this script would not work with the shipped pack. >> >> 2. What would be the effect of WUM updates? If the script considers the >> config changes in #1, and if any WUM updates affect those configurations >> (any changes would be additions), would the script also have to be updated? >> >> 3. Going along with what Sinthuja has raised above, if new packs are to >> be created, then most of the operations would be what a config automation >> tool does. >> >> 4. What would be overlap/effect on the product installer effort that is >> going on in the IE team [2] >> >> >> [1] - https://docs.wso2.com/display/AM2xx/Product+Profiles >> [2] - "Installers for WSO2 products" >> >> Regards, >> Chamila de Alwis >> Committer and PMC Member - Apache Stratos >> Associate Technical Lead | WSO2 >> +94 77 220 7163 >> Blog: https://medium.com/@chamilad >> >> >> >> >> On Wed, Jun 6, 2018 at 11:21 PM Sinthuja Rajendran <[email protected]> >> wrote: >> >>> Hi, >>> On Thu, Jun 7, 2018 at 11:07 AM Sarubi Thillainathan <[email protected]> >>> wrote: >>> >>>> Hi Chamila, >>>> >>>> According to the offline discussion with Nuwan, >>>> Here what we are doing is, automate configuration changes to optimize >>>> API-M profiles. Up to now, it has been manually done by users [1] when >>>> they wanted to run a selected profile. >>>> >>>> We provide two options to accomplish this, >>>> 1. The user can do the optimization separately before the server starts >>>> up by run the profileSetup.sh along with a particular profile flag. >>>> (ex: profileSetup.sh -Dprofile=api-store ) Once it has done, can use >>>> this APIM package as the root image for CM tools. >>>> >>> >>> This creates a new distribution (.zip) file for the intended profile >>> without altering the current distribution? Or it alters the current >>> distribution it self? i.e, Let's say I have downloaded and extracted APIM, >>> now I have to create api-store and api-publisher profiles. So, should I >>> extract the distributions two times and run this commands seperately >>> profileSetup.sh -Dprofile=api-store and profileSetup.sh >>> -Dprofile=api-publisher in two extracted directories? >>> >>> Thanks, >>> Sinthuja. >>> >>>> >>>> 2. The user can start the wso2server.sh with a particular profile along >>>> with --optimize flag. >>>> (i.e) wso2server.sh --optimize -Dprofile=api-store. >>>> if and only -optimize flag found we do the optimization for that >>>> particular profile then start the server automatically, otherwise not. >>>> >>>> Please find inline comments for further details: >>>> >>>> >>>> On Tue, Jun 5, 2018 at 4:11 AM, Chamila De Alwis <[email protected]> >>>> wrote: >>>> >>>>> Hi Sarubi, >>>>> >>>>> I have a couple of questions regarding the fit between the changes >>>>> that this script does and the config automation tools. >>>>> >>>>> Is this a tool focused on evaluation part of the product or the >>>>> deployment part? IMO both would require a re-evaluation of the approach. >>>>> >>>> >>>> This is focused on deployment part of product. >>>> >>>>> >>>>> IIUC, this script would alter the startup pack, with file additions >>>>> and removals done to config files, deployable artifacts, and other library >>>>> files. This could be a problematic case for continuous config automation >>>>> story. >>>>> >>>>> Consider a scenario where Puppet (or any other tool) configures a set >>>>> of instances, using the APIM pack as the root. The tool might be >>>>> configured >>>>> to continuously run (say, with a 30-minute interval). Is there a chance >>>>> for >>>>> the server to be reconfigured every time the tool is run, because the tool >>>>> detects the changes that this script does at the startup, as changes that >>>>> should be rolled back? >>>>> >>>> >>>> According to the scenario that you have mentioned, Puppet (or any other >>>> tool) can configures a set of instances, using optimized APIM pack as the >>>> root. >>>> >>>> >>>>> In that case, it might be better to have this tool as a separate >>>>> script rather than one which starts up the server as well. >>>>> >>>> >>>> Yes, now we maintain a seperate script which will get executed based on >>>> the user specification for server start-up. >>>> >>>>> >>>>> Another point against such a tool bound to server startup would be the >>>>> mixing of concerns. AFAIU, what this script does falls under what the >>>>> config automation tools do. IMO WSO2 server startup scripts should only >>>>> work on the bootstrapping of the server runtime, rather than the config >>>>> automation part. Also, we would essentially be repeating what the config >>>>> automation tools should do, but in Bash which sometimes is a lot harder. >>>>> Obtaining config values, managing them meaningfully, and applying them to >>>>> files is essentially what a config automation tool does. We may be >>>>> re-inventing the wheel here. For an example, should this tool be concerned >>>>> about the port offset at all since even dev deployments will be done using >>>>> separate VMs/Containers? Or should it be worried about different values >>>>> for >>>>> the config options in different environments? >>>>> >>>>> Also, this script don't concerned about the port offset things since >>>> it designed only to configure the changes for corresponding profile >>>> specified. >>>> >>>> >>>>> Additionally, users who manage config automation usually adhere to one >>>>> set of scripts. Puppet users almost always go for Puppet, Chef users for >>>>> Chef etc. What we would be delivering might step on what they already >>>>> have, >>>>> or are comfortable working with, and in turn, the adaption rate might take >>>>> a hit. >>>>> >>>>> User can go with first option, run the profile setup script seperatly >>>> before server start up and get optimized API-M pack as the base image >>>> >>>> >>>>> IMO you might benefit from reviewing your approach with the >>>>> Installation Experience effort, as the responsibilities of this tool fall >>>>> close to, if not overlap with, what the Installation Experience scripts >>>>> deliver/would deliver with different providers. >>>>> >>>>> WDYT? >>>>> >>>>> >>>>> Regards, >>>>> Chamila de Alwis >>>>> Committer and PMC Member - Apache Stratos >>>>> Associate Technical Lead | WSO2 >>>>> +94 77 220 7163 >>>>> Blog: https://medium.com/@chamilad >>>>> >>>>> >>>> Commnets or suggestions are highly appreciated. >>>> >>>> [1]https://docs.wso2.com/display/AM220/Product+Profiles >>>> >>>> >>>> -- >>>> *Sarubi Thillainathan * >>>> *Software Engineer - WSO2 Inc.* >>>> >>>> *Mobile : +94 (0) 76 68 49 101* >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>> >>> >>> -- >>> *Sinthuja Rajendran* >>> Senior Technical Lead >>> WSO2, Inc.:http://wso2.com >>> >>> Blog: http://sinthu-rajan.blogspot.com/ >>> Mobile: +94774273955 >>> >>> >>> > > > -- > Nuwan Dias > > Software Architect - WSO2, Inc. http://wso2.com > email : [email protected] > Phone : +94 777 775 729 >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
