Re: [openstack-dev] [TripleO] Plan management refactoring for Life cycle
Hi, On 09/11/2018 12:08 PM, Bogdan Dobrelya wrote: > On 9/11/18 4:43 AM, James Slagle wrote: >> On Mon, Sep 10, 2018 at 10:12 AM Jiri Tomasek >> wrote: >>> >>> Hi Mathieu, >>> >>> Thanks for bringing up the topic. There are several efforts >>> currently in progress which should lead to solving the problems >>> you're describing. We are working on introducing CLI commands which >>> would perform the deployment configuration operations on deployment >>> plan in Swift. This is a main step to finally reach CLI and GUI >>> compatibility/interoperability. CLI will perform actions to >>> configure deployment (roles, networks, environments selection, >>> parameters setting etc.) by calling Mistral workflows which store >>> the information in deployment plan in Swift. The result is that all >>> the information which define the deployment are stored in central >>> place - deployment plan in Swift and the deploy command is turned >>> into simple 'openstack overcloud deploy'. Deployment plan >>> then has plan-environment.yaml which has the list of environments >>> used and customized parameter values, roles-data.yaml which carry >>> roles definition and network-data.yaml which carry networks >>> definition. The information stored in these files (and deployment >>> plan in general) can then be treated as source of information about >>> deployment. The deployment can then be easily exported and reliably >>> replicated. >>> >>> Here is the document which we put together to identify missing >>> pieces between GUI,CLI and Mistral TripleO API. We'll use this to >>> discuss the topic at PTG this week and define work needed to be done >>> to achieve the complete interoperability. [1] >>> >>> Also there is a pending patch from Steven Hardy which aims to remove >>> CLI specific environments merging which should fix the problem with >>> tracking of the environments used with CLI deployment. [2] >>> Thank you Jirka to point me to this work. I will be happy to help in those efforts, at least for the lice cycle part (Update/Upgrade/Scale) of this big feature. I can't attend to the PTG this week unfortunately, but if you can point me out the etherpad with the resume of the session it would be very nice. I think the review from Steven aim to solve more or less the same issue than my current review. I will go through it in details, and AFAIS the last changes are old. >>> [1] >>> https://gist.github.com/jtomasek/8c2ae6118be0823784cdafebd9c0edac >>> (Apologies for inconvenient format, I'll try to update this to >>> better/editable format. Original doc: >>> https://docs.google.com/spreadsheets/d/1ERfx2rnPq6VjkJ62JlA_E6jFuHt9vVl3j95dg6-mZBM/edit?usp=sharing) >>> [2] https://review.openstack.org/#/c/448209/ >> >> >> Related to this work, I'd like to see us store the plan in git instead >> of swift. I think this would reduce some of the complexity around plan >> management, and move us closer to a simpler undercloud architecture. >> It would be nice to see each change to the plan represented as new git >> commit, so we can even see the changes to the plan as roles, networks, >> services, etc, are selected. >> >> I also think git would provide a familiar experience for both >> developers and operators who are already accustomed to devops type >> workflows. I think we could make these changes without it impact the >> API too much or, hopefully, at all. > > +42! > See also the related RFE (drafted only) [0] > > [0] https://bugs.launchpad.net/tripleo/+bug/1782139 Thanks James, Same here +1 (or 42 :)) > >> > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Plan management refactoring for Life cycle
On 9/11/18 4:43 AM, James Slagle wrote: On Mon, Sep 10, 2018 at 10:12 AM Jiri Tomasek wrote: Hi Mathieu, Thanks for bringing up the topic. There are several efforts currently in progress which should lead to solving the problems you're describing. We are working on introducing CLI commands which would perform the deployment configuration operations on deployment plan in Swift. This is a main step to finally reach CLI and GUI compatibility/interoperability. CLI will perform actions to configure deployment (roles, networks, environments selection, parameters setting etc.) by calling Mistral workflows which store the information in deployment plan in Swift. The result is that all the information which define the deployment are stored in central place - deployment plan in Swift and the deploy command is turned into simple 'openstack overcloud deploy'. Deployment plan then has plan-environment.yaml which has the list of environments used and customized parameter values, roles-data.yaml which carry roles definition and network-data.yaml which carry networks definition. The information stored in these files (and deployment plan in general) can then be treated as source of information about deployment. The deployment can then be easily exported and reliably replicated. Here is the document which we put together to identify missing pieces between GUI,CLI and Mistral TripleO API. We'll use this to discuss the topic at PTG this week and define work needed to be done to achieve the complete interoperability. [1] Also there is a pending patch from Steven Hardy which aims to remove CLI specific environments merging which should fix the problem with tracking of the environments used with CLI deployment. [2] [1] https://gist.github.com/jtomasek/8c2ae6118be0823784cdafebd9c0edac (Apologies for inconvenient format, I'll try to update this to better/editable format. Original doc: https://docs.google.com/spreadsheets/d/1ERfx2rnPq6VjkJ62JlA_E6jFuHt9vVl3j95dg6-mZBM/edit?usp=sharing) [2] https://review.openstack.org/#/c/448209/ Related to this work, I'd like to see us store the plan in git instead of swift. I think this would reduce some of the complexity around plan management, and move us closer to a simpler undercloud architecture. It would be nice to see each change to the plan represented as new git commit, so we can even see the changes to the plan as roles, networks, services, etc, are selected. I also think git would provide a familiar experience for both developers and operators who are already accustomed to devops type workflows. I think we could make these changes without it impact the API too much or, hopefully, at all. +42! See also the related RFE (drafted only) [0] [0] https://bugs.launchpad.net/tripleo/+bug/1782139 -- Best regards, Bogdan Dobrelya, Irc #bogdando __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Plan management refactoring for Life cycle
On Mon, Sep 10, 2018 at 10:12 AM Jiri Tomasek wrote: > > Hi Mathieu, > > Thanks for bringing up the topic. There are several efforts currently in > progress which should lead to solving the problems you're describing. We are > working on introducing CLI commands which would perform the deployment > configuration operations on deployment plan in Swift. This is a main step to > finally reach CLI and GUI compatibility/interoperability. CLI will perform > actions to configure deployment (roles, networks, environments selection, > parameters setting etc.) by calling Mistral workflows which store the > information in deployment plan in Swift. The result is that all the > information which define the deployment are stored in central place - > deployment plan in Swift and the deploy command is turned into simple > 'openstack overcloud deploy'. Deployment plan then has > plan-environment.yaml which has the list of environments used and customized > parameter values, roles-data.yaml which carry roles definition and > network-data.yaml which carry networks definition. The information stored in > these files (and deployment plan in general) can then be treated as source of > information about deployment. The deployment can then be easily exported and > reliably replicated. > > Here is the document which we put together to identify missing pieces between > GUI,CLI and Mistral TripleO API. We'll use this to discuss the topic at PTG > this week and define work needed to be done to achieve the complete > interoperability. [1] > > Also there is a pending patch from Steven Hardy which aims to remove CLI > specific environments merging which should fix the problem with tracking of > the environments used with CLI deployment. [2] > > [1] https://gist.github.com/jtomasek/8c2ae6118be0823784cdafebd9c0edac > (Apologies for inconvenient format, I'll try to update this to > better/editable format. Original doc: > https://docs.google.com/spreadsheets/d/1ERfx2rnPq6VjkJ62JlA_E6jFuHt9vVl3j95dg6-mZBM/edit?usp=sharing) > [2] https://review.openstack.org/#/c/448209/ Related to this work, I'd like to see us store the plan in git instead of swift. I think this would reduce some of the complexity around plan management, and move us closer to a simpler undercloud architecture. It would be nice to see each change to the plan represented as new git commit, so we can even see the changes to the plan as roles, networks, services, etc, are selected. I also think git would provide a familiar experience for both developers and operators who are already accustomed to devops type workflows. I think we could make these changes without it impact the API too much or, hopefully, at all. -- -- James Slagle -- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Plan management refactoring for Life cycle
Hi Mathieu, Thanks for bringing up the topic. There are several efforts currently in progress which should lead to solving the problems you're describing. We are working on introducing CLI commands which would perform the deployment configuration operations on deployment plan in Swift. This is a main step to finally reach CLI and GUI compatibility/interoperability. CLI will perform actions to configure deployment (roles, networks, environments selection, parameters setting etc.) by calling Mistral workflows which store the information in deployment plan in Swift. The result is that all the information which define the deployment are stored in central place - deployment plan in Swift and the deploy command is turned into simple 'openstack overcloud deploy'. Deployment plan then has plan-environment.yaml which has the list of environments used and customized parameter values, roles-data.yaml which carry roles definition and network-data.yaml which carry networks definition. The information stored in these files (and deployment plan in general) can then be treated as source of information about deployment. The deployment can then be easily exported and reliably replicated. Here is the document which we put together to identify missing pieces between GUI,CLI and Mistral TripleO API. We'll use this to discuss the topic at PTG this week and define work needed to be done to achieve the complete interoperability. [1] Also there is a pending patch from Steven Hardy which aims to remove CLI specific environments merging which should fix the problem with tracking of the environments used with CLI deployment. [2] [1] https://gist.github.com/jtomasek/8c2ae6118be0823784cdafebd9c0edac (Apologies for inconvenient format, I'll try to update this to better/editable format. Original doc: https://docs.google.com/spreadsheets/d/1ERfx2rnPq6VjkJ62JlA_E6jFuHt9vVl3j95dg6-mZBM/edit?usp=sharing ) [2] https://review.openstack.org/#/c/448209/ -- Jirka On Mon, Sep 10, 2018 at 8:05 AM mathieu bultel wrote: > Hi folks, > > Last week I wrote a BluePrint and a spec [1] to propose to change the way > we used and managed the Plan in TripleO for the Deployment and the Life > cycle (update/upgrade and scale). > > While I was working on trying to simplified the implementation of the > Update and Upgrade for a end user usage, I found very hard to follow all > the calls that the TripleO Client was doing to the HeatClient and > SwiftClient. > > I traced the calls and found that we can safely and easily decrease the > number of calls and simplified the way that we are computing & rendering > the TripleO Heat Templates files. > > I did a PoC to see what would be the problematic behind that and what we > could do without breaking the "standard" usage and all the particular > things that the current code handle (specific deployments and > configurations & so on). > > By this refactoring I'm seeing another gain for the life cycle part of > TripleO, where we used to try to make thing simpler & safer but we > constantly failed due to this complexity and all the "special cases" that > we faced during the testing. > > The result is that, when a user need to perform an update/upgrade of his > deployment, he really have to be careful, to pay a lot of attention of all > the options, -e environments files that he previously used with the risk > to make a simple mistake, and totally mess up the deployment. > > So my goals with this PoC and this BP is to try to addressed those points > by: > > simplify and reduce the number of calls between the clients, > > have a simple way for creating and updating the Plan, even by amending the > plan with only a particular files / config or Playbooks, > > store all the in formations provided by the user by uploading all the > files outsides of the plan, > > keep the track of the environment files passed to the CLI, > > trace the life cycle story of the deployment. > > So feel free to comment, add your concerns or feedback around this. > > Cheer, > > Mathieu > > [1] > > https://blueprints.launchpad.net/tripleo/+spec/tripleo-plan-management > > https://review.openstack.org/599396 > > [2] > > https://review.openstack.org/583145 > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
