Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
Our "TripleO" undercloud only requires a subset of the available services that we run int the Overcloud. So ironic, heat, mistral, zaqar, keystone, nova, swift, glance, neutron. These would mostly satisfy our needs. What I find interesting here is that there isn't much mention of jenkins, or something like it to produce the artifacts that the undercloud runs on-top of (likely from a combination of git sources, one for each project). I would have thought jenkins (or equiv, zuul...) + some deployment tool (ansible, salt, rundeck, puppet, chef...) + ironic (for new hardware ingestion) would compose most of what people are using to get there 'undercloud' working. -Josh __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Wed, 2016-08-17 at 23:42 +, arkady_kanev...@dell.com wrote: > What is the goal of undercloud? > Primarily to deploy and manage/upgrade/update overcloud. > It is not targeted for multitenancy and the only “application” > running on it is overcloud. > While it may have a couple of VMs running in undercloud it is more > convenience than actual need. > > So what are the OpenStack projects need to run in undercloud to > achieve its primary goal? Our "TripleO" undercloud only requires a subset of the available services that we run int the Overcloud. So ironic, heat, mistral, zaqar, keystone, nova, swift, glance, neutron. These would mostly satisfy our needs. > > Having robust undercloud so it can handle faults, like node or > network failures, is more important than being able to deploy all > OpenStack services on it. I think we can have all of these things. By using Heat we are a step closer to an HA undercloud. The fact that we can deploy all the other services on the undercloud too may seem irrelevant, until it doesn't. This sort of "everything can be in your undercloud" use case could be quite cool in fact. I don't think we'd force the idea on anyone though and if it takes some time for people to warm up to the latter use case that is fine. The primary points in doing all this are to help benefit the TripleO undercloud via code and template re-use. I think this stands on its own. Dan > > Arkady > -Original Message- > From: Dan Prince [mailto:dpri...@redhat.com] > Sent: Friday, August 05, 2016 6:35 AM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [TripleO] a new Undercloud install > driven by Heat > > On Fri, 2016-08-05 at 12:27 +0200, Dmitry Tantsur wrote: > > On 08/04/2016 11:48 PM, Dan Prince wrote: > > > > > > Last week I started some prototype work on what could be a new > way > > > to install the Undercloud. The driving force behind this was some > of > > > the recent "composable services" work we've done in TripleO so > > > initially I called in composable undercloud. There is an > etherpad > > > here with links to some of the patches already posted upstream > (many > > > of which stand as general imporovements on their own outside the > > > scope of what I'm talking about here). > > > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > > > The idea in short is that we could spin up a small single process > > > all- > > > in-one heat-all (engine and API) and thereby avoid things like > > > Rabbit, and MySQL. Then we can use Heat templates to drive the > > > Undercloud deployment just like we do in the Overcloud. > > I don't want to sound rude, but please no. The fact that you have > a > > hammer does not mean everything around is nails :( What problem > are > > you trying to solve by doing it? > > Several problems I think. > > One is TripleO has gradually moved away from elements. And while we > still use DIB elements for some things we no longer favor that tool > and instead rely on Heat and config management tooling to do our > stepwise deployment ordering. This leaves us using instack-undercloud > a tool built specifically to install elements locally as a means to > create our undercloud. It works... and I do think we've packaged it > nicely but it isn't the best architectural fit for where we are going > I think. I actually think that from an end/user contribution > standpoint using t-h- t could be quite nice for adding features to > the Undercloud. > > Second would be re-use. We just spent a huge amount of time in Newton > (and some in Mitaka) refactoring t-h-t around composable services. So > say you add a new composable service for Barbican in the Overcloud... > wouldn't it be nice to be able to consume the same thing in your > Undercloud as well? Right now you can't, you have to do some of the > work twice and in quite different formats I think. Sure, there is > some amount of shared puppet work but that is only part of the > picture I think. > > There are new features to think about here too. Once upon a time > TripleO supported multi-node underclouds. When we switched to > instack- undercloud we moved away from that. By switching back to > tripleo-heat- templates we could structure our templates around > abstractions like resource groups and the new 'deployed-server' trick > that allow you to create machines either locally or perhaps via > Ironic too. We could avoid Ironic entirely and always install the > Undercloud on existing servers via 'deployed-server' as well. > > Lastly,
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On 08/05/2016 01:21 PM, Steven Hardy wrote: On Fri, Aug 05, 2016 at 12:27:40PM +0200, Dmitry Tantsur wrote: On 08/04/2016 11:48 PM, Dan Prince wrote: Last week I started some prototype work on what could be a new way to install the Undercloud. The driving force behind this was some of the recent "composable services" work we've done in TripleO so initially I called in composable undercloud. There is an etherpad here with links to some of the patches already posted upstream (many of which stand as general imporovements on their own outside the scope of what I'm talking about here). https://etherpad.openstack.org/p/tripleo-composable-undercloud The idea in short is that we could spin up a small single process all- in-one heat-all (engine and API) and thereby avoid things like Rabbit, and MySQL. Then we can use Heat templates to drive the Undercloud deployment just like we do in the Overcloud. I don't want to sound rude, but please no. The fact that you have a hammer does not mean everything around is nails :( What problem are you trying to solve by doing it? I think Dan explains it pretty well in his video, and your comment indicates a fundamental misunderstanding around the entire TripleO vision, which is about symmetry and reuse between deployment tooling and the deployed cloud. The problems this would solve are several: 1. Remove divergence between undercloud and overcloud puppet implementation (instead of having an undercloud specific manifest, we reuse the *exact* same stuff we use for overcloud deployments) this; to reuse the service templates and puppet classes as they are sounds good 2. Better modularity, far easier to enable/disable services 3. Get container integration "for free" when we land it in the overcloud 4. Any introspection and debugging workflow becomes identical between the undercloud and overcloud 5. We remove dependencies on a bunch of legacy scripts which run outside of puppet 6. Whenever someone lands support for a new service in the overcloud, we automatically get undercloud support for it, completely for free. 7. Potential for much easier implementation of a multi-node undercloud Undercloud installation is already sometimes fragile, but it's probably the least fragile part right now (at least from my experience) And at the very least it's pretty obviously debuggable in most cases. THT is hard to understand and often impossible to debug. I'd prefer we move away from THT completely rather than trying to fix it in one more place where heat does not fit.. I *do* see your point about the undercloud installation being the less problematic but I part of that is because we didn't need to plug into the undercloud the same level of flexibility we demand for overcloud Now, maybe we also shouldn't make things complicated where they don't need to (see points 2 and 3) but in addition to reusing tht/puppet code, I think it would be interesting to have undercloud/ha (point 7) fwiw, I'd like to try this out myself before the summit to get a better picture. -- Giulio Fidente GPG KEY: 08D733BA | IRC: gfidente __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
What is the goal of undercloud? Primarily to deploy and manage/upgrade/update overcloud. It is not targeted for multitenancy and the only "application" running on it is overcloud. While it may have a couple of VMs running in undercloud it is more convenience than actual need. So what are the OpenStack projects need to run in undercloud to achieve its primary goal? Having robust undercloud so it can handle faults, like node or network failures, is more important than being able to deploy all OpenStack services on it. Arkady -Original Message- From: Dan Prince [mailto:dpri...@redhat.com] Sent: Friday, August 05, 2016 6:35 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat On Fri, 2016-08-05 at 12:27 +0200, Dmitry Tantsur wrote: > On 08/04/2016 11:48 PM, Dan Prince wrote: > > > > Last week I started some prototype work on what could be a new way > > to install the Undercloud. The driving force behind this was some of > > the recent "composable services" work we've done in TripleO so > > initially I called in composable undercloud. There is an etherpad > > here with links to some of the patches already posted upstream (many > > of which stand as general imporovements on their own outside the > > scope of what I'm talking about here). > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > The idea in short is that we could spin up a small single process > > all- > > in-one heat-all (engine and API) and thereby avoid things like > > Rabbit, and MySQL. Then we can use Heat templates to drive the > > Undercloud deployment just like we do in the Overcloud. > I don't want to sound rude, but please no. The fact that you have a > hammer does not mean everything around is nails :( What problem are > you trying to solve by doing it? Several problems I think. One is TripleO has gradually moved away from elements. And while we still use DIB elements for some things we no longer favor that tool and instead rely on Heat and config management tooling to do our stepwise deployment ordering. This leaves us using instack-undercloud a tool built specifically to install elements locally as a means to create our undercloud. It works... and I do think we've packaged it nicely but it isn't the best architectural fit for where we are going I think. I actually think that from an end/user contribution standpoint using t-h- t could be quite nice for adding features to the Undercloud. Second would be re-use. We just spent a huge amount of time in Newton (and some in Mitaka) refactoring t-h-t around composable services. So say you add a new composable service for Barbican in the Overcloud... wouldn't it be nice to be able to consume the same thing in your Undercloud as well? Right now you can't, you have to do some of the work twice and in quite different formats I think. Sure, there is some amount of shared puppet work but that is only part of the picture I think. There are new features to think about here too. Once upon a time TripleO supported multi-node underclouds. When we switched to instack- undercloud we moved away from that. By switching back to tripleo-heat- templates we could structure our templates around abstractions like resource groups and the new 'deployed-server' trick that allow you to create machines either locally or perhaps via Ironic too. We could avoid Ironic entirely and always install the Undercloud on existing servers via 'deployed-server' as well. Lastly, there is container work ongoing for the Overcloud. Again, I'd like to see us adopt a format that would allow it to be used in the Undercloud as well as opposed to having to re-implement features in the Over and Under clouds all the time. > > Undercloud installation is already sometimes fragile, but it's > probably the least fragile part right now (at least from my > experience) And at the very least it's pretty obviously debuggable in > most cases. THT is hard to understand and often impossible to debug. > I'd prefer we move away from THT completely rather than trying to fix > it in one more place where heat does not fit.. What tool did you have in mind. FWIW I started with heat because by using just Heat I was able to take the initial steps to prototype this. In my mind Mistral might be next here and in fact it already supports the single process launching idea thing. Keeping the undercloud installer as light as possible would be ideal though. Dan > > > > > > > I created a short video demonstration which goes over some of the > > history behind the approach, and shows a live demo of all of this > > working with the patc
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
Hi, There are a few additional items needed here before you can use the container work from tht in the undercloud. First, we deploy on Atomic. Atomic already has docker started when it boots. We can't use Atomic for the undercloud because there is no yum to install the clients. The clients would have to be in another container. Instead of using Atomic, Docker could be setup and configured by a script before the deployment. Second, you need to build the images locally and push them to a local Docker registry. Therefore, there needs to be additional bits that configure and set up the registry followed by cloning Kolla and building the container images. Lastly, configs are generated using puppet tags. I'm not sure every service has a _config tag in the puppet scripts currently. Thanks, -Ryan - Original Message - From: "Dan Prince" <dpri...@redhat.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Sent: Friday, August 5, 2016 7:34:32 AM Subject: Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat Lastly, there is container work ongoing for the Overcloud. Again, I'd like to see us adopt a format that would allow it to be used in the Undercloud as well as opposed to having to re-implement features in the Over and Under clouds all the time. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
Well, except for you need some non-openstack starting point, because unlike with e.g. ansible installing any openstack service(s) does not end at "dnf install ". You might like to watch Dan's demo again. It goes something like: yum install python-tripleoclient openstack undercloud deploy Done! That's pretty awesome indeed. But it also moves the user further away from the actual code running, so in case of an unobvious failure they'll have to inspect more layers. I guess I am getting at the debugability point again... At least it's good to know we're getting all the output visible, that's really great. The problems this would solve are several: 1. Remove divergence between undercloud and overcloud puppet implementation (instead of having an undercloud specific manifest, we reuse the *exact* same stuff we use for overcloud deployments) I'm not against reusing puppet bits, I'm against building the same heavy abstraction layer with heat around it. Sure, this is a valid concern to raise, and analternative to what Dan has prototyped would be to refactor the undercloud puppet manifest to use the puppet-tripleo profiles, somebody still has to do this work and it still doesn't help at all with either container integration or multi-node underclouds. So, this multi-node thing. Will it still be as easy as running one command? I guess we assume that the OS is already provisioned on all nodes, right? 2. Better modularity, far easier to enable/disable services Why? Do you expect enabling/disabling Nova, for example? In this regard undercloud is fundamentally different from overcloud: for the former we have a list of required services and a pretty light list of optional services. Actually, yes! I'd love to be able to disable Nova and instead deploy nodes directly via a mistral workflow that drives Ironic. That's why I started this: https://review.openstack.org/#/c/313048/ ++ to this However, it brings a big QE concern. If we say we support deployment with and without nova, it increases a number of things to test wrt provisioning twice. I still suspect we'll end up with one "blesses" way, and the other "probably working" ways. Which might not be so good. There are reasons such as static IPs for everything where you might want to be able to make Neutron optional, and there are already a bunch of optional services (such as all the telemetry services). Ok, every time I want to disable or add a new service I can hack on the manifest, but it's just extra work compared to reusing the exact same method we already support for overcloud deployments. 3. Get container integration "for free" when we land it in the overcloud 4. Any introspection and debugging workflow becomes identical between the undercloud and overcloud I would love a defined debugging workflow for the overcloud first.. Sure, and it's something we have to improve regardless. 5. We remove dependencies on a bunch of legacy scripts which run outside of puppet If you mean instack-undercloud element, we're getting rid of them anyway, no? Quite a few still remain, but yeah there are less than there was, which is good. I think I've seen the patches up for removing all of them (except for puppet-stack-config obviously). 6. Whenever someone lands support for a new service in the overcloud, we automatically get undercloud support for it, completely for free. Again, why? A service won't integrate itself into the deployment. And to be honest, the amount of options TripleO has already cases real world problems. I would rather see a well defined set of functionality for it.. It means it's easy to enable any service which is one less barrier to integration, I'm not really sure how that could be construed as a bad thing. 7. Potential for much easier implementation of a multi-node undercloud Ideally, I would love to see: for node in nodes: ssh $node puppet apply blah-blah Haha, this is a delightful over-simplification, but it completely ignores all of the logic to create the per-node manifests and hieradata. This is what the Heat templates already do for us, over multiple nodes by default. A bit unrelated, but while we're here... I wonder if we could stop after instances are deployed with Heat returning a set of hieredata files for nodes... Haven't thought is through, just a quick idea. Maybe we're not there, but it only means we have to improve our puppet modules. There is a layer of orchestration outside of the per-service modules which is needed here. We do that simply in the current undercloud implementation by having a hard-coded manifest, which works OK. We do that in the overcloud by orchestrating puppet via Heat over multiple nodes, which also works OK. Undercloud installation is already sometimes fragile, but it's probably the least fragile part right now (at least from my experience) And at the very least it's pretty obviously debuggable in most cases. THT is hard to
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Fri, 2016-08-05 at 13:56 +0200, Dmitry Tantsur wrote: > On 08/05/2016 01:21 PM, Steven Hardy wrote: > > > > On Fri, Aug 05, 2016 at 12:27:40PM +0200, Dmitry Tantsur wrote: > > > > > > On 08/04/2016 11:48 PM, Dan Prince wrote: > > > > > > > > Last week I started some prototype work on what could be a new > > > > way to > > > > install the Undercloud. The driving force behind this was some > > > > of the > > > > recent "composable services" work we've done in TripleO so > > > > initially I > > > > called in composable undercloud. There is an etherpad here with > > > > links > > > > to some of the patches already posted upstream (many of which > > > > stand as > > > > general imporovements on their own outside the scope of what > > > > I'm > > > > talking about here). > > > > > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > > > > > The idea in short is that we could spin up a small single > > > > process all- > > > > in-one heat-all (engine and API) and thereby avoid things like > > > > Rabbit, > > > > and MySQL. Then we can use Heat templates to drive the > > > > Undercloud > > > > deployment just like we do in the Overcloud. > > > I don't want to sound rude, but please no. The fact that you have > > > a hammer > > > does not mean everything around is nails :( What problem are you > > > trying to > > > solve by doing it? > > I think Dan explains it pretty well in his video, and your comment > > indicates a fundamental misunderstanding around the entire TripleO > > vision, > > which is about symmetry and reuse between deployment tooling and > > the > > deployed cloud. > Well, except for you need some non-openstack starting point, because > unlike with e.g. ansible installing any openstack service(s) does > not > end at "dnf install ". > > > > > > > The problems this would solve are several: > > > > 1. Remove divergence between undercloud and overcloud puppet > > implementation > > (instead of having an undercloud specific manifest, we reuse the > > *exact* > > same stuff we use for overcloud deployments) > I'm not against reusing puppet bits, I'm against building the same > heavy > abstraction layer with heat around it. What do you mean by heavy exactly. The entire point here was to demonstrate that this can work and *is* actually quite lightweight I think. We are already building an abstraction layer. So why not just use it in 2 places instead of one. > > > > > > > 2. Better modularity, far easier to enable/disable services > Why? Do you expect enabling/disabling Nova, for example? In this > regard > undercloud is fundamentally different from overcloud: for the former > we > have a list of required services and a pretty light list of optional > services. I think this is a very narrow view of the Undercloud and ignores the fact that continually adding booleans to enable or disable features is not scalable. Using the same composability and deployment framework we have developed for the Overcloud might make better sense to me. There is also real potential here to re-use this as a means to install other package based types of setups. An "anything is an undercloud" sort of approach could be the next logic step... all of this for free because we are building abstractions to install these things in the Overcloud as well. > > > > > > > 3. Get container integration "for free" when we land it in the > > overcloud > > > > 4. Any introspection and debugging workflow becomes identical > > between the > > undercloud and overcloud > I would love a defined debugging workflow for the overcloud first.. The nice thing about demo I showed for debugging is all the output comes back to the console. Heat, os-collect-config, puppet, etc. all there at your fingertips. Set 'debug=True' and you have everything you need I think. After building it I've quite enjoyed how fast it is to test and debug creating a prototype undercloud.yaml. > > > > > > > 5. We remove dependencies on a bunch of legacy scripts which run > > outside of > > puppet > If you mean instack-undercloud element, we're getting rid of them > anyway, no? We mean all of the elements. Besides a few bootstrapping things we have gradually moved towards using Heat hooks to run things as opposed to the traditional os-apply-config/os-refresh-config hooks. This provides better signalling back to heat and arguably makes debugging much easier when something fails too. > > > > > > > 6. Whenever someone lands support for a new service in the > > overcloud, we > > automatically get undercloud support for it, completely for free. > Again, why? A service won't integrate itself into the deployment. And > to > be honest, the amount of options TripleO has already cases real > world > problems. I would rather see a well defined set of functionality for > it.. > > > > > > > 7. Potential for much easier implementation of a multi-node > > undercloud > Ideally, I would love to see: > > for node in nodes: > ssh
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Fri, Aug 05, 2016 at 01:56:32PM +0200, Dmitry Tantsur wrote: > On 08/05/2016 01:21 PM, Steven Hardy wrote: > > On Fri, Aug 05, 2016 at 12:27:40PM +0200, Dmitry Tantsur wrote: > > > On 08/04/2016 11:48 PM, Dan Prince wrote: > > > > Last week I started some prototype work on what could be a new way to > > > > install the Undercloud. The driving force behind this was some of the > > > > recent "composable services" work we've done in TripleO so initially I > > > > called in composable undercloud. There is an etherpad here with links > > > > to some of the patches already posted upstream (many of which stand as > > > > general imporovements on their own outside the scope of what I'm > > > > talking about here). > > > > > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > > > > > The idea in short is that we could spin up a small single process all- > > > > in-one heat-all (engine and API) and thereby avoid things like Rabbit, > > > > and MySQL. Then we can use Heat templates to drive the Undercloud > > > > deployment just like we do in the Overcloud. > > > > > > I don't want to sound rude, but please no. The fact that you have a hammer > > > does not mean everything around is nails :( What problem are you trying to > > > solve by doing it? > > > > I think Dan explains it pretty well in his video, and your comment > > indicates a fundamental misunderstanding around the entire TripleO vision, > > which is about symmetry and reuse between deployment tooling and the > > deployed cloud. > > Well, except for you need some non-openstack starting point, because unlike > with e.g. ansible installing any openstack service(s) does not end at "dnf > install ". You might like to watch Dan's demo again. It goes something like: yum install python-tripleoclient openstack undercloud deploy Done! > > The problems this would solve are several: > > > > 1. Remove divergence between undercloud and overcloud puppet implementation > > (instead of having an undercloud specific manifest, we reuse the *exact* > > same stuff we use for overcloud deployments) > > I'm not against reusing puppet bits, I'm against building the same heavy > abstraction layer with heat around it. Sure, this is a valid concern to raise, and analternative to what Dan has prototyped would be to refactor the undercloud puppet manifest to use the puppet-tripleo profiles, somebody still has to do this work and it still doesn't help at all with either container integration or multi-node underclouds. > > 2. Better modularity, far easier to enable/disable services > > Why? Do you expect enabling/disabling Nova, for example? In this regard > undercloud is fundamentally different from overcloud: for the former we have > a list of required services and a pretty light list of optional services. Actually, yes! I'd love to be able to disable Nova and instead deploy nodes directly via a mistral workflow that drives Ironic. That's why I started this: https://review.openstack.org/#/c/313048/ There are reasons such as static IPs for everything where you might want to be able to make Neutron optional, and there are already a bunch of optional services (such as all the telemetry services). Ok, every time I want to disable or add a new service I can hack on the manifest, but it's just extra work compared to reusing the exact same method we already support for overcloud deployments. > > 3. Get container integration "for free" when we land it in the overcloud > > > > 4. Any introspection and debugging workflow becomes identical between the > > undercloud and overcloud > > I would love a defined debugging workflow for the overcloud first.. Sure, and it's something we have to improve regardless. > > 5. We remove dependencies on a bunch of legacy scripts which run outside of > > puppet > > If you mean instack-undercloud element, we're getting rid of them anyway, > no? Quite a few still remain, but yeah there are less than there was, which is good. > > 6. Whenever someone lands support for a new service in the overcloud, we > > automatically get undercloud support for it, completely for free. > > Again, why? A service won't integrate itself into the deployment. And to be > honest, the amount of options TripleO has already cases real world problems. > I would rather see a well defined set of functionality for it.. It means it's easy to enable any service which is one less barrier to integration, I'm not really sure how that could be construed as a bad thing. > > 7. Potential for much easier implementation of a multi-node undercloud > > Ideally, I would love to see: > > for node in nodes: >ssh $node puppet apply blah-blah Haha, this is a delightful over-simplification, but it completely ignores all of the logic to create the per-node manifests and hieradata. This is what the Heat templates already do for us, over multiple nodes by default. > Maybe we're not there, but it only means we have to improve our puppet >
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
Hi, On Fri, Aug 5, 2016 at 7:56 PM, Dmitry Tantsurwrote: > Ideally, I would love to see: > > for node in nodes: >ssh $node puppet apply blah-blah > > Maybe we're not there, but it only means we have to improve our puppet > modules. This is the same thought I had, Shouldn't the config be just a call to a manifest? The undercloud should be a simple install process with some config (or image based deployment). Using Heat to deploy the undercloud means involves bootstrapping a Heat environment. I believe Ansible feels like a much better fit for this. What would the user/administrator want? Is customization of the undercloud something realistically happening? regards, Gerard -- Gerard Braad | http://gbraad.nl [ Doing Open Source Matters ] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Fri, 2016-08-05 at 13:39 +0200, Thomas Herve wrote: > On Thu, Aug 4, 2016 at 11:48 PM, Dan Prince> wrote: > > > > Last week I started some prototype work on what could be a new way > > to > > install the Undercloud. The driving force behind this was some of > > the > > recent "composable services" work we've done in TripleO so > > initially I > > called in composable undercloud. There is an etherpad here with > > links > > to some of the patches already posted upstream (many of which stand > > as > > general imporovements on their own outside the scope of what I'm > > talking about here). > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > The idea in short is that we could spin up a small single process > > all- > > in-one heat-all (engine and API) and thereby avoid things like > > Rabbit, > > and MySQL. > I saw those patches coming, I'm interested in the all-in-one > approach, > if only for testing purpose. I hope to be able to propose a solution > with broker-less RPC instead of fake RPC at some point, but it's a > good first step. > > I'm a bit more intrigued by the no-auth patch. It seems that Heat > would rely heavily on Keystone interactions even after initial > authentication, so I wonder how that work. As it seems you would need > to push the same approach to Ironic, have you considered starting > Keystone instead? It's a simple WSGI service, and can work with > SQLite > as well I believe. You are correct. Noauth wasn't enough. I had to add a bit more to make OS::Heat::SoftwareDeployments happy to get the templates I showed in the demo working. Surprisingly though if I avoid Heat OS::Heat::SoftwareDeployments and only used OS:Heat::SoftwareConfig's in my templates no extra keystone auth was needed. This is because heat only creates the extra Keystone user, trust, etc. when realizing the software deployments I think. I started with this which should work for multiple projects besides just Heat: https://review.openstack.org/#/c/351351/2/tripleoclient/fake _keystone.py I'd be happy to swap in full Keystone if people prefer but that would be more memory, and setup. Keystone dropped it's eventlet runner recently so we'd have to fork another WSGI process to run it I think somewhere in an out of the way (non-default ports, etc) fashion. I was trying to keep the project list minimal so I went and stubbed in only what was functionally needed for this here with an eye that we'd actually (at some point) make heat support true noauth again. Dan > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On 08/05/2016 01:34 PM, Dan Prince wrote: On Fri, 2016-08-05 at 12:27 +0200, Dmitry Tantsur wrote: On 08/04/2016 11:48 PM, Dan Prince wrote: Last week I started some prototype work on what could be a new way to install the Undercloud. The driving force behind this was some of the recent "composable services" work we've done in TripleO so initially I called in composable undercloud. There is an etherpad here with links to some of the patches already posted upstream (many of which stand as general imporovements on their own outside the scope of what I'm talking about here). https://etherpad.openstack.org/p/tripleo-composable-undercloud The idea in short is that we could spin up a small single process all- in-one heat-all (engine and API) and thereby avoid things like Rabbit, and MySQL. Then we can use Heat templates to drive the Undercloud deployment just like we do in the Overcloud. I don't want to sound rude, but please no. The fact that you have a hammer does not mean everything around is nails :( What problem are you trying to solve by doing it? Several problems I think. One is TripleO has gradually moved away from elements. And while we still use DIB elements for some things we no longer favor that tool and instead rely on Heat and config management tooling to do our stepwise deployment ordering. This leaves us using instack-undercloud a tool built specifically to install elements locally as a means to create our undercloud. It works... and I do think we've packaged it nicely but it isn't the best architectural fit for where we are going I think. I actually think that from an end/user contribution standpoint using t-h- t could be quite nice for adding features to the Undercloud. I don't quite get how it is better than finally moving to puppet only and stop using elements. Second would be re-use. We just spent a huge amount of time in Newton (and some in Mitaka) refactoring t-h-t around composable services. So say you add a new composable service for Barbican in the Overcloud... wouldn't it be nice to be able to consume the same thing in your Undercloud as well? Right now you can't, you have to do some of the work twice and in quite different formats I think. Sure, there is some amount of shared puppet work but that is only part of the picture I think. I've already responded to Steve's email, so a tl;dr here: I'm not sure why you want to add random services to undercloud. Have you seen an installer ever benefiting from e.g. adding a FileSystem-as-a-Service or Database-as-a-Service solution? There are new features to think about here too. Once upon a time TripleO supported multi-node underclouds. When we switched to instack- undercloud we moved away from that. By switching back to tripleo-heat- templates we could structure our templates around abstractions like resource groups and the new 'deployed-server' trick that allow you to create machines either locally or perhaps via Ironic too. We could avoid Ironic entirely and always install the Undercloud on existing servers via 'deployed-server' as well. A side note: if we do use Ironic for this purpose, I would expect some help with pushing the Ironic composable service through. And the ironic-inspector's one, which I haven't even started. I'm still struggling to understand what entity is going to install this bootstrapping Heat instance. Are we bringing back seed? Lastly, there is container work ongoing for the Overcloud. Again, I'd like to see us adopt a format that would allow it to be used in the Undercloud as well as opposed to having to re-implement features in the Over and Under clouds all the time. Undercloud installation is already sometimes fragile, but it's probably the least fragile part right now (at least from my experience) And at the very least it's pretty obviously debuggable in most cases. THT is hard to understand and often impossible to debug. I'd prefer we move away from THT completely rather than trying to fix it in one more place where heat does not fit.. What tool did you have in mind. FWIW I started with heat because by using just Heat I was able to take the initial steps to prototype this. In my mind Mistral might be next here and in fact it already supports the single process launching idea thing. Keeping the undercloud installer as light as possible would be ideal though. I don't have a really huge experience with both, but for me Mistral seems much cleaner and easier to understand. That, of course, won't allow you to use reuse the existing heat templates (which may be good or bad depending on your point of view). Dan I created a short video demonstration which goes over some of the history behind the approach, and shows a live demo of all of this working with the patches above: https://www.youtube.com/watch?v=y1qMDLAf26Q Thoughts? Would it be cool to have a session to discuss this more in Barcelona? Dan Prince (dprince)
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On 08/05/2016 01:21 PM, Steven Hardy wrote: On Fri, Aug 05, 2016 at 12:27:40PM +0200, Dmitry Tantsur wrote: On 08/04/2016 11:48 PM, Dan Prince wrote: Last week I started some prototype work on what could be a new way to install the Undercloud. The driving force behind this was some of the recent "composable services" work we've done in TripleO so initially I called in composable undercloud. There is an etherpad here with links to some of the patches already posted upstream (many of which stand as general imporovements on their own outside the scope of what I'm talking about here). https://etherpad.openstack.org/p/tripleo-composable-undercloud The idea in short is that we could spin up a small single process all- in-one heat-all (engine and API) and thereby avoid things like Rabbit, and MySQL. Then we can use Heat templates to drive the Undercloud deployment just like we do in the Overcloud. I don't want to sound rude, but please no. The fact that you have a hammer does not mean everything around is nails :( What problem are you trying to solve by doing it? I think Dan explains it pretty well in his video, and your comment indicates a fundamental misunderstanding around the entire TripleO vision, which is about symmetry and reuse between deployment tooling and the deployed cloud. Well, except for you need some non-openstack starting point, because unlike with e.g. ansible installing any openstack service(s) does not end at "dnf install ". The problems this would solve are several: 1. Remove divergence between undercloud and overcloud puppet implementation (instead of having an undercloud specific manifest, we reuse the *exact* same stuff we use for overcloud deployments) I'm not against reusing puppet bits, I'm against building the same heavy abstraction layer with heat around it. 2. Better modularity, far easier to enable/disable services Why? Do you expect enabling/disabling Nova, for example? In this regard undercloud is fundamentally different from overcloud: for the former we have a list of required services and a pretty light list of optional services. 3. Get container integration "for free" when we land it in the overcloud 4. Any introspection and debugging workflow becomes identical between the undercloud and overcloud I would love a defined debugging workflow for the overcloud first.. 5. We remove dependencies on a bunch of legacy scripts which run outside of puppet If you mean instack-undercloud element, we're getting rid of them anyway, no? 6. Whenever someone lands support for a new service in the overcloud, we automatically get undercloud support for it, completely for free. Again, why? A service won't integrate itself into the deployment. And to be honest, the amount of options TripleO has already cases real world problems. I would rather see a well defined set of functionality for it.. 7. Potential for much easier implementation of a multi-node undercloud Ideally, I would love to see: for node in nodes: ssh $node puppet apply blah-blah Maybe we're not there, but it only means we have to improve our puppet modules. Undercloud installation is already sometimes fragile, but it's probably the least fragile part right now (at least from my experience) And at the very least it's pretty obviously debuggable in most cases. THT is hard to understand and often impossible to debug. I'd prefer we move away from THT completely rather than trying to fix it in one more place where heat does not fit.. These are some strong but unqualified assertions, so it's hard to really respond. We'll talk about "unqualified" assertions the next time I'll try to get answers on #tripleo after seeing error messages like "controller_step42 failed with code 1" ;) Yes, there is complexity, but it's a set of abstractions which actually work pretty well for us, so there is value in having just one set of abstractions used everywhere vs special-casing the undercloud. There should be a point where we stop. What entity is going to install heat to install undercloud (did I just say "seed")? What will provide HA for it? Authentication, templates storage and versioning? How do you reuse the same abstractions (that's the whole point after all)? Re moving away from THT completely, this is not a useful statement - yes, there are alternative tools, but if you were to remove THT and just use some other tool with Ironic, the result would simply not be TripleO. There would be zero migration/upgrade path for existing users and all third-party integrations (and our API/UI) would break. I don't agree it would not be TripleO. OpenStack does not end on heat templates, some deployments don't even use heat. FWIW I think this prototyping work is very interesting, and I'm certainly keen to get wider (more constructive) feedback and see where it leads. Thanks, Steve __ OpenStack
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Thu, Aug 4, 2016 at 11:48 PM, Dan Princewrote: > Last week I started some prototype work on what could be a new way to > install the Undercloud. The driving force behind this was some of the > recent "composable services" work we've done in TripleO so initially I > called in composable undercloud. There is an etherpad here with links > to some of the patches already posted upstream (many of which stand as > general imporovements on their own outside the scope of what I'm > talking about here). > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > The idea in short is that we could spin up a small single process all- > in-one heat-all (engine and API) and thereby avoid things like Rabbit, > and MySQL. I saw those patches coming, I'm interested in the all-in-one approach, if only for testing purpose. I hope to be able to propose a solution with broker-less RPC instead of fake RPC at some point, but it's a good first step. I'm a bit more intrigued by the no-auth patch. It seems that Heat would rely heavily on Keystone interactions even after initial authentication, so I wonder how that work. As it seems you would need to push the same approach to Ironic, have you considered starting Keystone instead? It's a simple WSGI service, and can work with SQLite as well I believe. -- Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Fri, 2016-08-05 at 12:27 +0200, Dmitry Tantsur wrote: > On 08/04/2016 11:48 PM, Dan Prince wrote: > > > > Last week I started some prototype work on what could be a new way > > to > > install the Undercloud. The driving force behind this was some of > > the > > recent "composable services" work we've done in TripleO so > > initially I > > called in composable undercloud. There is an etherpad here with > > links > > to some of the patches already posted upstream (many of which stand > > as > > general imporovements on their own outside the scope of what I'm > > talking about here). > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > The idea in short is that we could spin up a small single process > > all- > > in-one heat-all (engine and API) and thereby avoid things like > > Rabbit, > > and MySQL. Then we can use Heat templates to drive the Undercloud > > deployment just like we do in the Overcloud. > I don't want to sound rude, but please no. The fact that you have a > hammer does not mean everything around is nails :( What problem are > you > trying to solve by doing it? Several problems I think. One is TripleO has gradually moved away from elements. And while we still use DIB elements for some things we no longer favor that tool and instead rely on Heat and config management tooling to do our stepwise deployment ordering. This leaves us using instack-undercloud a tool built specifically to install elements locally as a means to create our undercloud. It works... and I do think we've packaged it nicely but it isn't the best architectural fit for where we are going I think. I actually think that from an end/user contribution standpoint using t-h- t could be quite nice for adding features to the Undercloud. Second would be re-use. We just spent a huge amount of time in Newton (and some in Mitaka) refactoring t-h-t around composable services. So say you add a new composable service for Barbican in the Overcloud... wouldn't it be nice to be able to consume the same thing in your Undercloud as well? Right now you can't, you have to do some of the work twice and in quite different formats I think. Sure, there is some amount of shared puppet work but that is only part of the picture I think. There are new features to think about here too. Once upon a time TripleO supported multi-node underclouds. When we switched to instack- undercloud we moved away from that. By switching back to tripleo-heat- templates we could structure our templates around abstractions like resource groups and the new 'deployed-server' trick that allow you to create machines either locally or perhaps via Ironic too. We could avoid Ironic entirely and always install the Undercloud on existing servers via 'deployed-server' as well. Lastly, there is container work ongoing for the Overcloud. Again, I'd like to see us adopt a format that would allow it to be used in the Undercloud as well as opposed to having to re-implement features in the Over and Under clouds all the time. > > Undercloud installation is already sometimes fragile, but it's > probably > the least fragile part right now (at least from my experience) And > at > the very least it's pretty obviously debuggable in most cases. THT > is > hard to understand and often impossible to debug. I'd prefer we move > away from THT completely rather than trying to fix it in one more > place > where heat does not fit.. What tool did you have in mind. FWIW I started with heat because by using just Heat I was able to take the initial steps to prototype this. In my mind Mistral might be next here and in fact it already supports the single process launching idea thing. Keeping the undercloud installer as light as possible would be ideal though. Dan > > > > > > > I created a short video demonstration which goes over some of the > > history behind the approach, and shows a live demo of all of this > > working with the patches above: > > > > https://www.youtube.com/watch?v=y1qMDLAf26Q > > > > Thoughts? Would it be cool to have a session to discuss this more > > in > > Barcelona? > > > > Dan Prince (dprince) > > > > ___ > > ___ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu > > bscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _ > _ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On Fri, Aug 05, 2016 at 12:27:40PM +0200, Dmitry Tantsur wrote: > On 08/04/2016 11:48 PM, Dan Prince wrote: > > Last week I started some prototype work on what could be a new way to > > install the Undercloud. The driving force behind this was some of the > > recent "composable services" work we've done in TripleO so initially I > > called in composable undercloud. There is an etherpad here with links > > to some of the patches already posted upstream (many of which stand as > > general imporovements on their own outside the scope of what I'm > > talking about here). > > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud > > > > The idea in short is that we could spin up a small single process all- > > in-one heat-all (engine and API) and thereby avoid things like Rabbit, > > and MySQL. Then we can use Heat templates to drive the Undercloud > > deployment just like we do in the Overcloud. > > I don't want to sound rude, but please no. The fact that you have a hammer > does not mean everything around is nails :( What problem are you trying to > solve by doing it? I think Dan explains it pretty well in his video, and your comment indicates a fundamental misunderstanding around the entire TripleO vision, which is about symmetry and reuse between deployment tooling and the deployed cloud. The problems this would solve are several: 1. Remove divergence between undercloud and overcloud puppet implementation (instead of having an undercloud specific manifest, we reuse the *exact* same stuff we use for overcloud deployments) 2. Better modularity, far easier to enable/disable services 3. Get container integration "for free" when we land it in the overcloud 4. Any introspection and debugging workflow becomes identical between the undercloud and overcloud 5. We remove dependencies on a bunch of legacy scripts which run outside of puppet 6. Whenever someone lands support for a new service in the overcloud, we automatically get undercloud support for it, completely for free. 7. Potential for much easier implementation of a multi-node undercloud > Undercloud installation is already sometimes fragile, but it's probably the > least fragile part right now (at least from my experience) And at the very > least it's pretty obviously debuggable in most cases. THT is hard to > understand and often impossible to debug. I'd prefer we move away from THT > completely rather than trying to fix it in one more place where heat does > not fit.. These are some strong but unqualified assertions, so it's hard to really respond. Yes, there is complexity, but it's a set of abstractions which actually work pretty well for us, so there is value in having just one set of abstractions used everywhere vs special-casing the undercloud. Re moving away from THT completely, this is not a useful statement - yes, there are alternative tools, but if you were to remove THT and just use some other tool with Ironic, the result would simply not be TripleO. There would be zero migration/upgrade path for existing users and all third-party integrations (and our API/UI) would break. FWIW I think this prototyping work is very interesting, and I'm certainly keen to get wider (more constructive) feedback and see where it leads. Thanks, Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] a new Undercloud install driven by Heat
On 08/04/2016 11:48 PM, Dan Prince wrote: Last week I started some prototype work on what could be a new way to install the Undercloud. The driving force behind this was some of the recent "composable services" work we've done in TripleO so initially I called in composable undercloud. There is an etherpad here with links to some of the patches already posted upstream (many of which stand as general imporovements on their own outside the scope of what I'm talking about here). https://etherpad.openstack.org/p/tripleo-composable-undercloud The idea in short is that we could spin up a small single process all- in-one heat-all (engine and API) and thereby avoid things like Rabbit, and MySQL. Then we can use Heat templates to drive the Undercloud deployment just like we do in the Overcloud. I don't want to sound rude, but please no. The fact that you have a hammer does not mean everything around is nails :( What problem are you trying to solve by doing it? Undercloud installation is already sometimes fragile, but it's probably the least fragile part right now (at least from my experience) And at the very least it's pretty obviously debuggable in most cases. THT is hard to understand and often impossible to debug. I'd prefer we move away from THT completely rather than trying to fix it in one more place where heat does not fit.. I created a short video demonstration which goes over some of the history behind the approach, and shows a live demo of all of this working with the patches above: https://www.youtube.com/watch?v=y1qMDLAf26Q Thoughts? Would it be cool to have a session to discuss this more in Barcelona? Dan Prince (dprince) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO] a new Undercloud install driven by Heat
Last week I started some prototype work on what could be a new way to install the Undercloud. The driving force behind this was some of the recent "composable services" work we've done in TripleO so initially I called in composable undercloud. There is an etherpad here with links to some of the patches already posted upstream (many of which stand as general imporovements on their own outside the scope of what I'm talking about here). https://etherpad.openstack.org/p/tripleo-composable-undercloud The idea in short is that we could spin up a small single process all- in-one heat-all (engine and API) and thereby avoid things like Rabbit, and MySQL. Then we can use Heat templates to drive the Undercloud deployment just like we do in the Overcloud. I created a short video demonstration which goes over some of the history behind the approach, and shows a live demo of all of this working with the patches above: https://www.youtube.com/watch?v=y1qMDLAf26Q Thoughts? Would it be cool to have a session to discuss this more in Barcelona? Dan Prince (dprince) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev